Effects of temporal averaging on short-term irradiance variability under mixed sky conditions

Characterizations of short-term variability in solar radiation are required to successfully integrate large numbers of photovoltaic power systems into the electrical grid. Previous studies have used ground-based irradiance observations with a range of different temporal resolutions and a systematic analysis of the effects of temporal averaging on the representation of variability is lacking. Using highresolution surface irradiance data with original temporal resolutions between 0.01 and 1 s from six different locations in the Northern Hemisphere, we characterize the changes in representation of temporal variability resulting from time averaging. In this analysis, we condition all data to states of mixed skies, which are the most potentially problematic in terms of local PV power volatility. Statistics of clear-sky index k and its increments 1k τ (i.e., normalized surface irradiance and changes therein over specified intervals of time) are considered separately. Our results indicate that a temporal averaging time scale of around 1 s marks a transition in representing single-point irradiance variability, such that longer averages result in substantial underestimates of variability. Higher-resolution data increase the complexity of data management and quality control without appreciably improving the representation of variability. The results do not show any substantial discrepancies between locations or seasons.


Introduction
Both the installed capacity and the number of photovoltaic (PV) power systems are increasing in many regions of the world (Solar Power Europe, SPE).Unlike conven-tional power plants, PV systems are characterized by nondispatchable and highly variable power production on spatiotemporal scales ranging from days to seconds and from hundreds of kilometers to meters (Perez et al., 2016).This intrinsic PV power variability complicates electrical grid operation and may negatively impact power quality and grid stability (Stetz et al., 2015).As changes in PV power are primarily determined by cloud-induced changes in solar irradiance, a comprehensive data-driven characterization of irradiance variability can help mitigate the risks associated with the above-mentioned problems.While satellite-derived irradiance data provide large spatial coverage, their spatial and temporal resolutions are limited, so that high-resolution ground-based irradiance measurements are needed to analyze local short-term variability (Lohmann et al., 2016).
Previous PV-related studies on variability in ground-based solar radiation observations utilized data with a range of different temporal resolutions.While early studies focused on hourly to daily timescales (e.g.Liu and Jordan, 1960), later analyses were often geared towards finer temporal resolutions between 300 and 0.01 s (Table 1).The irradiance data considered in such studies were typically collected on the horizontal plane, although PV systems commonly feature tilted modules.With regards to PV power applications, it is thus important to note that variability in irradiance on an inclined plane has been shown to be higher than on the horizontal, at least on a daily basis (Perpiñán, 2009;Suri et al., 2007).
There is no consensus as to the proper temporal resolution of irradiance measurements needed to capture all relevant variability.The larger the panel-covered area of a PV Published by Copernicus Publications on behalf of the European Geosciences Union.
Table 1.Previous studies of short-term irradiance variability and the temporal resolutions considered.
Temp. res.Example references system, the less variable in general is its power output compared to a single-point irradiance measurement (van Haaren et al., 2014), especially on short sub-minute timescales (Marcos et al., 2011a, b).Thus, high temporal resolutions on the order of seconds and shorter may not be required to monitor large utility-scale PV plants (Woyte et al., 2013), while minute-averaged data may be too coarsely resolved (van Haaren et al., 2014).When considering smaller rooftop PV systems and/or partial shading (which can strongly reduce an inverter's power output as soon as a few connected modules are shaded; Belhaouas et al., 2017), previous research has shown that the temporal resolution needed to capture irradiance variability on all time scales may be as small as 0.1 s (Torres Lobera et al., 2013;Yordanov et al., 2013b) or 0.4 s (Gagné et al., 2016).
Those studies that argued the need for sub-second resolutions made this determination based on various different lines of reasoning: 1. using the 2nd temporal derivative of irradiance as a measure for instantaneous variation, defining the daily minimum (negative) value of this derivative as each day's strongest instantaneous irradiance variation, and finally calculating a hypothetical optimal averaging time based on an acceptable error of 10 Wm −2 and an assumed parabolic shape of the variation for each of a few hundred days in spring and summer (Yordanov et al., 2013b); 2. analyzing the sample standard deviation of irradiance as a function of averaging time measured during 7 h on a single summer's day (Torres Lobera et al., 2013); and 3. separately studying two variability metrics as functions of averaging time for 7 selected days (Gagné et al., 2016).
In addition to these methodical differences and relatively short datasets, each study was limited to a single geographic area, namely southern Norway (Yordanov et al., 2013b), southern Finland (Torres Lobera et al., 2013), and eastern Canada (Gagné et al., 2016).
In this paper, we combine high-resolution irradiance data originating from six different locations in the Northern Hemisphere in order to systematically assess the biases in representation of temporal irradiance variability resulting from temporally averaged measurements.The time series feature original temporal resolutions between 0.01 and 1 s, and four of the sites have records of at least one full year, enabling seasonal comparisons of short-term averaging effects for the first time.We derive estimates of clear-sky index (i.e., surface irradiance normalized by its clear-sky value) and its increments (i.e., changes over fixed time intervals), and analyze the respective time series' standard deviations as functions of averaging time scale.In this analysis, we condition all data to states of mixed skies, which are the most potentially problematic in terms of local PV power volatility.Details on all datasets and utilized methods are presented in Sect.2, with the results and discussion following in Sect.3, and conclusions in Sect. 4.
Different types of photodiode-based pyranometers were used, with raw temporal resolutions T 0 ranging from 0.01 s through 1 s.Table 2 summarizes the different measurement sites' coordinates, available recording periods, number of pyranometers N pyr , original temporal resolutions T 0 , and the respective total durations of mixed-sky conditions available for the analyses.
The Canadian data from Alderville and Varennes were measured using LI-COR LI-200S pyranometers, with samples being taken every 0.001 s, and their averages evaluated every 0.01 s.Gagné et al. (2016) provide detailed information about the locations, data acquisition units, and the local character of irradiance variability in Alderville and Varennes.We use a subset of three sensors from Alderville, providing data from January 2015 through March 2017, and two sensors from Varennes, with data being available from October 2015 through November 2016.The inter-sensor distances are on the order of 100 m each, and the two locations are about 400 km apart.
The data set collected near Jülich consists of a five-sensor subset of a larger pyranometer network, which was deployed during the HD(CP) 2 Observational Prototype Experiment (HOPE; Macke et al., 2017) between 2 April and 24 July 2013, using as many as 99 EKO ML-020VM sensors.The selected sensors are evenly distributed across the entire domain and separated by a few kilometers each.Madhavan et al. (2016) detail the HOPE campaign with respect to this pyranometer network and provide further information about data acquisition and quality control.Several studies have already used 1 s averages derived from the original 0.1 s irradiance samples to characterize the averages' spatiotemporal variability (Lohmann et al., 2016;Madhavan et al., 2017), and to evaluate sky-imager-based irradiance retrievals (Schmidt et al., 2016).For our present analyses of the five-sensor subset, we use the original 0.1 s irradiance samples.
In Oldenburg, three EKO ML-01 pyranometers have been continually recording 0.1 s irradiance samples since 28 April 2015 as part of an in-house weather observation system of the university's Energy Meteorology Group.The sensors are horizontally mounted on a university building rooftop with intersensor distances of about 15 m, and subject to bi-weekly maintenance, including verification of the horizontal orientation and cleaning of the glass dome.The quality of the measurements was verified for randomly selected periods using additional measurements from a co-located well-established thermopile pyranometer (Schmidt and Lohmann, 2018).We use data from 1 May 2015 through 31 December 2016 in this paper.
Near Kalaeloa Airport on Oahu, the National Renewable Energy Laboratory (NREL) performed a measurement campaign using 17 LI-COR LI-200 pyranometers from March 2010 though November 2011, collecting irradiance data every 1 s (Sengupta and Andreas, 2010).These data have previously been used and analyzed in several irradiance variability studies (e.g.Anvari et al., 2016;Aryaputera et al., 2015;Hinkelman, 2013;Madanchi et al., 2017;Munkhammar et al., 2017).In the present paper, we use data from the six-pyranometer subset located on the premises of Kalaeloa Airport, each separated by a few 100 m.
The data we use from Tucson were measured between 5 April and 30 June 2014 with a temporal resolution of 1 s (Lorenzo et al., 2015b).While the corresponding measurement campaign featured different photodiode-based sensors, we limit ourselves to a subset of five Apogee SP-212 pyranometers, so that all Tucson data used in this paper originate from the same type of sensor.The inter-sensor distances of the subset are on the order of 100 m.Lorenzo et al. (2015a) have previously documented details of the entire data set, and used it to evaluate solar irradiance forecasts.
For each location, Table 3 provides a basic overview of season-and cloud-elevation-specific cloud climatology based on the respective average cloud amount during the day from the Climatic Atlas of Clouds Over Land and Ocean (Eastman et al., 2014).The four mid-latitude locations feature comparable values of cloud amount across seasons and altitude ranges, while Oahu and Tucson stand out with appreciably smaller values for low-level (Tucson only) and middle-level clouds (both).Although the two locations also systematically exhibit smaller values than the others for highlevel cloud amount, these differences are not as pronounced.

Data preprocessing
All available irradiance time series feature infrequent dropout periods, during which the measured irradiance is suddenly reduced to almost 0 Wm −2 and then resumes a reasonable value after a short while (seconds to minutes).These unreasonably low readings can be in part associated with times of regular maintenance, during which a sensor's glass dome is covered with a piece of cloth.Additionally, they can be caused by e.g., birds, insects, or leaves temporarily occupying the small sensor area.To eliminate these drop-out periods from the data, we remove a time window of ±60 s around each irradiance value falling below a fixed threshold, taken to be 25 Wm −2 .
Next, we normalize all irradiance measurements G to their respective clear-sky values G clear , which we calculate using the clear-sky model described by Dumortier (1995), to obtain time series of the clear-sky index: While irradiance is subject to astronomically determined variations and trends, the clear-sky index is convenient for comparing short-term variability at different locations, and during different times of the day or year.However, k * estimates are highly uncertain for low solar elevation angles (Lave et al., 2012), and we thusly remove all data associated with elevation angles below 15 • at this point.It is important to note that the clear-sky index is not unambiguously defined, because different clear-sky models can yield different values of G clear and thus k * .Equation (1) hence introduces model-dependent uncertainties to the time series that are not present in the original irradiance measurements.In this paper, we use climatological averages of the Linke turbidity factor (Remund et al., 2003) as input to the clear-sky model.G clear thus represents a typical, and not necessarily the effective, clear-sky irradiance on a given day.Consequentially, all locations considered in this study exhibit some periods of suboptimal model performance, with results for Oahu (Tucson) subjectively tending to fit a little better (worse) than for the other locations (not shown).Although variations in k * may in part be due to variations in atmospheric composition (such as water vapor and aerosols) not accounted for by the clear-sky model, cloud-induced variability is by far the dominant source of irradiance variability on the relatively short timescales considered throughout this paper.
In order to extract mixed sky periods from the clear-sky index time series, we employ a subjective classification scheme based on dividing all time series into non-overlapping 900 s blocks, and calculating the individual blocks' clear-sky index standard deviations.If the standard deviation of k * exceeds a fixed threshold (taken to be 0.18, based on the results of Lohmann et al., 2016), we classify the corresponding 900 s block as mixed, and retain it for our analyses.Small changes to the value of this threshold do not substantially change the results.
The resulting ranges of durations of mixed sky conditions are quoted in Table 2, and strongly differ between locations.While Oahu, Oldenburg, and Alderville each provide a total of well over 1000 h of mixed-sky data, the brief campaigns around Jülich and Tucson feature mixed-sky records that are about an order of magnitude shorter.From Varennes, an intermediate number of more than 500 h of mixed-sky observations are available.

Characterizing variability
Using all data classified as mixed-sky conditions from each pyranometer of each location, we compute time se-ries of k * averages using a range of different averaging time scales 0.01 s < T < 900 s.In this calculation, we apply nonoverlapping moving windows of length T to each 900 s block and calculate the mean clear-sky index within each window.If fewer than 95 % of the highest-resolution data within a period T are available, the resulting average is classified as missing.Otherwise the missing values in the raw data are simply neglected in the computation.We found this subjective threshold of 95 % to be a good compromise between minimizing the number of missing data and maintaining the robustness of the mean estimates.Slight changes of the threshold value do not noticeably affect the results.With appreciably higher thresholds, a small number of missing highresolution data points could cause unreasonably many longterm averages to be missing; while for considerably lower thresholds, there would be too much variation in the number of data points entering the computation of means of nominal averaging time T .
Figure 1 presents two illustrative examples of k * time series under mixed sky conditions using temporal averaging scales across up to six orders of magnitude.The 1000 s period and its 100 s excerpt shown in panels (a, b) are typical examples of high temporal clear-sky index variability, while the 100 s period and its 10 s excerpt shown in panel (c, d) were specifically selected as a case featuring an unusually volatile time series with short-term variability on all time scales.In all panels, an evident decrease of temporal variability can be observed for increasing averaging periods T , with the most pronounced reductions starting at the transition from T = 1 to T = 10 s.The variance reduction for averaging periods T < 1 s is relatively large for the time series shown in (d).
To characterize clear-sky index variability as a function of averaging time scale, we first compare probability distributions of k * at each of the available sensors at all locations for averaging times T = 1 and T = 100 s.For each sensor at each location, we then estimate the sample standard deviation:  for averaging periods of 0.01 s < T < 900 s, where N T is the total number of available data points for averages of length T , and k * T denotes the corresponding arithmetic mean value of k * .In order to facilitate comparisons between locations, we also use a normalized clear-sky index standard deviation: for each time scale and location, using the standard deviation associated with T = 1 s as a normalization factor.The decline of σ k * T as a function of T quantifies the reduction of clear-sky index variability for increasing averaging time periods.Finally, we repeat the calculations of Eqs.
(2) and (3) for each season using the longer data from Alderville, Varennes, Oldenburg, and Oahu.In this seasonal analyses, winter is defined as December through February, spring as March through May, summer as June through August, and autumn as September through November.In the normalization factor in Eq. ( 3), we always use the full-year standard deviations σ k * T =1 s derived without conditioning by season.
The standard deviation of k * is independent of the observations' ordering in time, and thus does not quantify how G. M. Lohmann and A. H. Monahan: Temporal averaging of short-term irradiance variability quickly the values can change.In order to consider the effect of temporal averaging on the estimates of rates of change, we compute k * increments: 3 Results and discussion

Clear-sky index variability
For the two averaging times T = 1 and T = 100 s, Fig. 2 presents the probability density functions of clear-sky index, based on all available mixed-sky periods from the six locations.The 1 s averages feature pronounced bimodal distributions for all locations.The two modes are located slightly above k * 1 (corresponding to times of clear skies), and within a range of 0.3 k * 0.5 (corresponding to times of cloud shadow coverage), with modest differences between locations.The probability minimum between the peaks is less pronounced at Tucson than at other locations.The distinct bimodality of the distributions of 1 s averages is consistent with previous findings of, for example, Jurado et al. (1995), Lohmann et al. (2016), Munkhammar et al. (2017), Schmidt et al. (2016), Skartveit and Olseth (1992), Suehrcke and McCormick (1988), and Woyte et al. (2007a).This feature is substantially weaker for the distributions of 100 s averages.Instead, while the high-value peaks are still very pronounced (but located at somewhat smaller values compared to the 1 s averages), the low-value peaks are replaced by flat shoulders in the distribution ranging from k * 0.4 through k * 0.9.
We interpret the minor differences in the distinct k * values at which the peaks occur as results of imperfect simulations of clear-sky irradiance, and hence a biased estimation of clear-sky index as per Eq.(1), as well as the fact that each location uses distinct types of pyranometers with potentially varying calibration performances and possibly marginally tilted orientations.The very high values k * 1 being frequently recorded at all locations are consistent with shortterm cloud enhancement (see e.g.Schade et al., 2007), while the comparatively high spread and rough structures of Jülich and Tucson (especially for T = 100 s) in panels (2c) and (2f) are due to sampling variability resulting from the relatively short periods of available observations (cf.Table 2).The normalized clear-sky index standard deviation σ k * T decays as a function of averaging time T (Fig. 3).The values remain close to σ k * T 1 for values T 5 s, and then begin to rapidly decrease with increasing T .For example, around T 50 s, the normalized standard deviation has dropped to 90 % σ k * T 95 %, and around T 500 s, it has become as low as 60 % σ k * T 67 %, depending on the location.For averaging times T < 1 s, the normalized standard deviations do not go appreciably above σ k * T 1 (the highest value is σ k * T = 1.0005 for T = 0.01 s; not shown).
In general, the structures of σ k * T at the different locations and their corresponding normalization factors σ k * T =1 s compare well, with relatively small differences in the rate at which variability declines and in the absolute values of σ k * T =1 s .No direct relation is evident between the order in which the normalized standard deviation structures decline, and the order of absolute standard deviations associated with T as a function of averaging time T under mixed-sky conditions for all six available locations.The normalization factors σ k * T =1 s are quoted for each location.The spread across multiple individual single-sensor structures is indicated using semi-transparent coloring, while the solid lines are derived by averaging the singlesensor structures for each T .T = 1 s.For example, Oldenburg and Oahu feature almost identical values of σ k * T =1 s (0.37 and 0.36, respectively), but correspond to end members in terms of variability decay rates, with Oahu featuring the fastest and Oldenburg the slowest decline.Likewise, the normalized structures of Oldenburg and Jülich both decline with a similar rate, but their values of σ k * T =1 s differ by approximately 10 % (0.37 and 0.33, respectively).As discussed above, imperfectly simulated clear-sky irradiance and possible differences in pyranometer calibration may contribute to the differences in absolute standard deviations for relatively close locations (like Oldenburg and Jülich).Differences between locations are largely removed by normalization.Even though the cloud climatologies are different at the different locations (for example, the Oahu data are strongly influenced by passing trade wind cumulus clouds, while the mid-latitude locations experience a more diverse variety of different cloud and weather phenomena), the statistical structures are very similar across locations.
Using data from the four locations with measurements in all seasons, Fig. 4 shows σ k * T as a function of averaging time separately for winter, spring, summer, and autumn.The respective durations of available mixed-sky data are additionally quoted in each panel.In general, the structures are very similar to the ones derived without conditioning by season (cf.Fig. 3 3).The spread across individual single-sensor structures is indicated using semi-transparent coloring, while the solid lines are derived by averaging the single-sensor structures for each T .

decrease of σ k *
T compared to the other seasons.Moreover, σ k * T values associated with Alderville's autumn, as well as with Oldenburg's winter and spring are slightly higher than σ k * T derived from all data (i.e.σ k * T =1 s > 1), while Oldenburg's summer exhibits lower values (i.e., σ k * T =1 s < 1).No systematic pattern of seasonal variations in σ k * T exists among the different locations.These results imply that the various cloud types prevailing during the different seasons exert relatively little influence on the temporal averaging effects of irradiance variability under mixed-sky conditions.

Increment variability
For three combinations of averaging time scales and increment time steps (T = 1 and τ = 1 s; T = 1 and τ = 100 s; T = 100 and τ = 100 s), Fig. 5 shows distributions of clearsky index increments, using all available mixed-sky periods from the six locations.At all locations, the resulting distributions exhibit global maxima at k * τ = 0, with characteristic shapes.For T = 1 and τ = 1 s, the distributions are chevronshaped, with densities decreasing rapidly for increasing positive and decreasing negative increment values.For the same averaging time T = 1 s but the larger increment τ = 100 s, Figure 5. Location-specific distributions of clear-sky index increments k * τ under mixed-sky conditions for averaging times T = 1 and T = 100 s, and increment time steps τ = 100 and τ = 1 s, using a bin width of 0.05.The spread across individual single-sensor distributions is indicated using semi-transparent coloring for all bins with data from at least two sensors.The averages of the singlesensor densities are presented as solid or dashed lines for the bins that contain data from every sensor.the distributions have broad shoulders, with two local maxima around k * τ ≈ ±1.Beyond these secondary maxima the tails decrease rapidly.For the longer averaging time T = 100 s and a corresponding increment time step of τ = 100 s, the distributions are more rounded around the distinct central peak.The finding of such non-Gaussian increment distributions is consistent with, for example, Hinkelman (2013) and Perpiñán and Lorenzo (2011).The systematic changes in the distributions' shapes for different averaging intervals and increment time steps qualitatively agree with reports published by e.g., Anvari et al. (2016), van Haaren et al. (2014), Lave et al. (2012), andMarcos et al. (2011b).
With respect to the increment distributions of 1 s averages, an increase of the increment time step from τ = 1 to τ = 100 s leads to an increase of high-magnitude increment probabilities by many orders of magnitude.While τ = 1 s is too short an increment time step for strong variations between clear and cloudy states to occur frequently, a time step of τ = 100 s frequently covers these transitions.Compared to T = 1 s, averaging the time series using T = 100 s leads to a considerable underestimation of both magnitude (i.e., the distributions are narrower) and probabilities (i.e., the distributions' shoulders are less pronounced) of strong 100 s increments.
While the increment distributions are strongly affected by averaging time and increment size, they are remarkably similar at all locations considered.The relatively small differences between locations (e.g., the exact values of the two secondary maxima for T = 1 and τ = 100 s) may result from imperfections in the clear sky model, as well as sensor calibration and leveling, as discussed earlier.The different lengths of the distributions' tails are directly determined by the record lengths (short at Jülich and Tucson; longer at the other locations).The estimates of the probability density tails are less robust than estimates within the core of the distribution, because they are typically based on a comparatively limited number of observations.In Varennes, for example, only about 100 occurrences of the absolute increment value | k * τ | > 0.6 are registered among the total of more than 2 • 10 6 data points for T = τ = 1 s.
Comparing the distributions of clear-sky index for T = 1 s (Fig. 2) with those of clear-sky index increments for T = 1 and τ = 100 s (Fig. 5) allows further characterization of transitions from times of clear skies to times of cloud shadow coverage (and vice versa).From the separation of the two distinct modes of the k * distributions, a clear-sky index difference of approximately 0.7 can be inferred between cloudfree and cloud-covered states.However, the secondary peaks of the increment distributions shown in Fig. 5 indicate transitions of ±1 in the clear-sky index to occur more frequently than changes of ±0.7.This disparity is consistent with shortterm cloud enhancement, instead of a typical transition going directly from e.g., clear (k * ≈ 1) to cloudy (k * ≈ 0.3), cloud enhancement immediately precedes (or follows) cloud cover, so typical transitions occur between a cloud-enhanced state (k * 1) and a cloud-covered state (k * ≈ 0.3).Note that the unambiguous identification of cloud enhancement is not possible using our values of k * , because of the potential bias and unaccounted for variability included in the clear-sky index we use (cf.discussion in Sect.2.2).
In order to further investigate the effect of increasing averaging times on increment variability, Fig. 6 presents both absolute and normalized clear-sky index increment standard deviations as a function of increment time step.The structures of σ k * τ T (left column) are effectively indistinguishable for T = 0.01, T = 0.1, and T = 1 s, while structures associated with longer averaging periods of T = 10 s (T = 100 s) deviate slightly (notably) from their high-resolution counterparts.Note that because of differences in the resolution of the data at different locations, shorter averaging timescales can be considered at some stations (e.g., Varennes and Alderville) than at others (e.g., Oahu and Tucson).The display of normalized standard deviation σ k * τ T (Fig. 6, right column) emphasizes the structures' differences, especially for small increment time steps.For any given averaging period T , smaller increment time steps τ are associated with larger deviations of σ k * τ T from unity (smaller increment steps are more affected by averaging than larger ones).Similarly, for any given τ , shorter averaging times T coincide with values of σ k * τ T closer to one (for a given increment step, longer averaging intervals have a bigger effect on variability).
Except for the anomalously low values of Varennes-based for T = 0.1 and τ 0.5 s in panel (6d), which we further discuss in Sect.3.3, the structures of both absolute and normalized clear-sky index increment standard deviations resemble each other well for all locations.Averages associated with T ≤ 1 s typically result in normalized increment standard deviations close to unity (0.95 ≤ σ ≈ 1 for τ 10 s), while averages of T = 10 and T = 100 s systematically underestimate increment variability for all time steps.This result is a comprehensive quantification of the qualitative features seen in the example time series in Fig. 1.Imperfections in pyranometer configuration and clear sky irradiance estimation do not appreciably influence the normalized structures, and we interpret the minor differences between location-specific increment standard deviations of 100 s averages as resulting from differences in local cloud conditions.For example, Oahu's typical trade wind cumulus clouds can be expected to generate relatively high variability on small time scales, and consequently, Oahu features smaller values of σ as functions of step size τ for different averaging timescales separately for winter, spring, summer, and autumn.The structures resemble those derived without conditioning by season (cf.Fig. 6), with some quantitative differences.For example, wintertime values of σ k * τ T 0 are slightly lower compared to the results derived from all data for Alderville, Varennes, and Oahu in panels (7a, 7c, 7g), while σ k * τ T is slightly higher for Oahu and T = 100 s in panel (7h).These winter-specific differences in increment variability structures do not differ by more than ±10 % relative to the other seasons, and we consider them comparatively minor.

Peculiarity in Varennes data
Both in Figs.6d and in 7d, the normalized standard deviations of clear-sky index increments at Varennes indicate a strong reduction of variability for increment time steps of τ = 0.1 s when averaging the original high-resolution data (T 0 = 0.01 s) with an averaging time period of T = 0.1 s.Except for the winter data in Fig. 7d, the increment standard deviation is reduced to about 90 % of its original value, which (right column) as functions of increment time τ for up to five orders of magnitude of averaging time T = 0.01, T = 0.1, T = 1, T = 10, and T = 100 s under mixed-sky conditions for all six available locations.The spread across individual single-sensor structures is indicated using semi-transparent coloring, while the solid lines are derived by averaging the single-sensor structures for each τ .In most cases, the spread is so small that it is indistinguishable from the multi-sensor average.6, for structures separately estimated for winter (December, January, February; DJF), spring (March, April, May; MAM), summer (June, July, August; JJA), and autumn (September, October, November; SON).The spread across single-sensor structures is comparable to that in Fig. 6, but it is not shown here for the sake of facility of inspection. is considerably lower than the value associated with T = 1 and τ = 1 s.This behavior matches neither the results obtained from Alderville for the same time scales (cf.For the majority of mixed-sky blocks in Varennes, the values of normalized standard deviations are very close to unity and therefore comparable to those at other stations on the scales in question.For example, the rightmost bin (0.985 < σ k * τ T ≤ 1.0) alone contains 79 % of all data (note the logarithmic scaling).This result implies that the relatively low value of σ k * τ T computed over all blocks is being influenced strongly by a very small number of peculiar periods.
An example time series of k * and k * τ associated with a very small value of σ k * τ T = 0.57 is shown in panels (b) and (c) of Fig. 8.This time series demonstrates that even within such a peculiar block, only a very small number of data points are responsible for the small block-specific normalized increment standard deviation.While the clear-sky index varies considerably throughout the block, there are only 4 events for which the averaged time series of T = 0.1 s differs evidently from the original measurements of T = 0.01 s.Within a few hundredths of a second, the original k * readings drop considerably and immediately return to their previous values during these 4 events.Averaging the time series over periods of T = 0.1 s removes these events almost entirely.
A few very rapid and short-lived changes of considerable magnitude on time scales on the order of the original temporal resolution (here: T 0 = 0.01 s), result in a relatively high increment standard deviation for small τ .If each single one of these changes in the time series consists of a sufficiently small number of data points in a row, an averaging time period T 1 of an order of magnitude higher than T 0 (here: T 1 = 0.1 s) can cause the short excursions to be averaged out almost completely, resulting in a much smaller increment standard deviation for small τ compared to the original temporal resolution, and hence a low value of normalized increment standard deviation for τ T 1 .
We have not established a clear cause of these extremely short reductions in irradiance.The observed short-term peculiarities are possibly a result of some kind of small objects briefly passing the sensor, for example birds, insects, or leaves.Although the same sensor type is used in Alderville and Varennes with the same original temporal resolutions, the Alderville data do not exhibit these rapid changes in k * resulting in particularly high values of σ k * τ T .A reason for this difference may be the measuring setup and its surroundings: in Varennes, the pyranometers were mounted on tripods and scattered on grassland, with a considerable number of nearby trees, while in Alderville, the sensors were located within a multi-megawatt PV plant (Gagné et al., 2016).The absence of small values of σ k * τ T in winter at Varennes (cf.Fig. 7) may also indicate that the peculiar events are caused by insects.However, the short record of only 30 h worth of mixed-sky data from Varennes in winter does not suffice for a conclusive diagnosis of the cause of these intermittent rapid changes.

Conclusions
Based on a unique set of irradiance measurements with original temporal resolutions between 0.01 and 1 s from six locations in the Northern Hemisphere, we have characterized the effects of temporal averaging on short-term irradiance variability under mixed-sky conditions, for the entire year as well as for individual seasons.For this analysis, we have considered probability distributions of the clear-sky index and its short-term increments on a range of temporal scales between 0.01 and 900 s, and studied each particular time series standard deviation as a function of averaging time scale and increment time step.These quantities were found to be largely independent of season and location for the data considered.
The following main results thus apply to all of the available mixed-sky data we analyzed.
-Clear-sky index distributions are strongly bimodal on averaging timescales between 0.01 and 1 s.The peaks are separated by a clear-sky index difference of approximately 0.7 and respectively represent cloud-covered and cloud-free states.
-Clear-sky index increment distributions of 1 s averages and 100 s increment time steps feature secondary peaks around k * τ ≈ ±1, indicating that transitions of ±1 in the clear-sky index occur more frequently than changes of ±0.7.We interpret this result as indicating that transitions going from a cloud-enhanced state (k * 1) to a cloud-covered state (k * ≈ 0.3), and vice versa, are more common than transitions directly between clear (k * ≈ 1) and cloudy (k * ≈ 0.3) states.-For the clear-sky index, an averaging time T 5 s is sufficient to capture all k * variability, while averaging times T beyond about 10 s cause the suppression of small-scale variability to increase rapidly (e.g., 1-min averages only retain about 90 % of the original k * standard deviation).
-For clear-sky index increments, a small averaging time of T = 1 s reduces increment standard deviation by approximately 5 % on the same scale of τ = 1 s, but effectively retains all variability information for increment time steps τ 10 s.
Based on these results, we conclude that a temporal averaging time of around T ≈ 1 s marks a transition in representing single-point irradiance variability under mixed-sky conditions.Larger values of T will tend to considerably underestimate variability, while smaller T will increase the complexity of data management and quality control without appreciably improving the representation of variability.Previous studies had judged even higher temporal resolutions between 0.1 and 0.4 s to be adequate, but the respective results were either based on considerably fewer data (Gagné et al., 2016;Torres Lobera et al., 2013), or focused on individual events rather than long-term, climatological variations (Yordanov et al., 2013b).Although there are short-term periods during which an averaging time of T ≈ 1 s does not capture all changes in irradiance from one second to the next (as illustrated in Fig. 1), such periods occur infrequently based on the records that form the basis of our results.One specific consequence of our results relates to the Baseline Surface Radiation Network (BSRN;McArthur, 2005;Ohmura et al., 1998), which currently records solar irradiance in minute averages.Our results strongly indicate the value of modifying this strategy towards much higher temporal resolutions.
To carry the present research questions further, analyzing high-resolution irradiance measurements in the plane-ofarray (POA) of existing PV systems (instead of focusing ex-G.M. Lohmann and A. H. Monahan: Temporal averaging of short-term irradiance variability clusively on global horizontal irradiance) would lead to results with higher direct applicability to PV power variability (although contrasting different sites will not be straight forward for POA irradiance).Moreover, the modeling of clear-sky irradiance can be improved by using a more advanced clear-sky model, such as the recently validated Mc-Clear model (Lefèvre et al., 2013;Eissa et al., 2015;Lefèvre and Wald, 2016).Also, it may be beneficial to extend the simple increment procedure (cf.Eq. 4) by considering the semivariogram (Matheron, 1963) and variogram (Wald, 1989), or by using a wavelet-based approach such as that discussed by Gallego et al. (2013).
Data availability.The data from Alderville and Varennes were kindly provided by Alexandre Gagné at the CanmetENERGY laboratory in Varennes.Samples of the data are available online (Natural Resources Canada, 2016).The raw 0.1 s irradiance samples from Jülich are courtesy of Andreas Macke and Bomidi Lakshmi Madhavan at the Leibniz Institute for Tropospheric Research (TROPOS) in Leipzig, Germany.A complete set of corresponding 1 s averages is available online (Standardized Atmospheric Measurement Data, 2017).The data from Oahu have been made publicly available by the National Renewable Energy Laboratory (NREL) of the United States of America (Sengupta and Andreas, 2010).The data from Oldenburg (Schmidt and Lohmann, 2018) and Tucson (Lorenzo et al., 2015b)  The Supplement related to this article is available online at https://doi.org/10.5194/amt-11-3131-2018-supplement.
Competing interests.The authors declare that they have no conflict of interest.

Figure 1 .
Figure 1.Two representative examples of highly variable clear-sky index k * time series under mixed sky conditions: panels (a) and (b) show a typical example of a 1000 s period using six averaging times T , namely (a) T = 1000, T = 100, and T = 10 s, and (b) T = 10, T = 1, T = 0.1, and T = 0.01 s (note that throughout the analyses of the paper we restrict ourselves to values of T ≤ 900 s; we consider up to T = 1000 s here for illustrative purposes); panels (c) and (d) show an unusual 100 s example with evident variability down to the shortest time scales with averaging periods (a) T = 100, T = 10, and T = 1 s, and (b) T = 1, T = 0.1, and T = 0.01 s.The time series were measured in Varennes on 2 March 2016, starting at 15:01:35 EDT (a, b), and on 21 September 2016, starting at 15:06:35 EDT (c, d).The gray areas in the top row indicate the excerpts presented in the bottom row.
for different time lags τ on the distinct averaging time scales T = 0.01, T = 0.1, T = 1, T = 10, and T = 100 s.Similar to the analyses of k * variability, we first explore distributions of k * τ for T = 100 and T = 1, using τ = 100 and τ = 1 s.Then we calculate clear-sky index increment standard deviations σ k * τ T for the same five averaging times and a range of 0.01 s < τ < 900 s analogous to Eq. (2), and derive their norfour seasons using the year-round observations from Alderville, Varennes, Oldenburg, and Oahu.

Figure 2 .
Figure2.Distributions of clear-sky index k * under mixed-sky conditions for two averaging times T = 1 and T = 100 s and all six locations, estimated from histograms using a bin width of 0.05.The spread across multiple single-sensor distributions is indicated using semi-transparent coloring, while the solid lines are derived by averaging the single-sensor densities in each bin.

Figure 3 .
Figure 3. Structures of normalized clear-sky index standard deviation σ k *T as a function of averaging time T under mixed-sky conditions for all six available locations.The normalization factors σ k * T =1 s are quoted for each location.The spread across multiple individual single-sensor structures is indicated using semi-transparent coloring, while the solid lines are derived by averaging the singlesensor structures for each T .

Figure 7 .
Figure7.As in Fig.6, for structures separately estimated for winter (December, January, February; DJF), spring (March, April, May; MAM), summer (June, July, August; JJA), and autumn (September, October, November; SON).The spread across single-sensor structures is comparable to that in Fig.6, but it is not shown here for the sake of facility of inspection.
Figs. 6b and 7b), nor the general characteristics of the value of σ k * τ T associated with τ = T to decrease with increasing T .In order to investigate the issue, we separately derive σ k * τ T for each individual 900 s mixed-sky period in Varennes using T = 0.01 and T = 0.1 s, as well as τ = 0.1 s.The distribution of the resulting set of single-block normalized increment standard deviations is shown in panel (a) of Fig. 8.

Figure 8 .
Figure 8.(a) The distribution of all Varennes-based single 900 sblock values of normalized clear-sky index increment standard deviation σ k * τ have been released online under a Creative Commons Attribution-Non-Commercial 4.0 International Public License (CC BY-NC 4.0; Creative Commons, 2013).

Table 2 .
Details of the available clear-sky index data sets: location names and their respective latitudes and longitudes, the available recording periods, number of pyranometers N pyr , original temporal resolutions T 0 , and total durations of mixed-sky conditions available for the analyses.

Table 3 .
(Hahn and Warren, 2007)during the day for each season (winter: DJF, spring: MAM, summer: JJA, autumn: SON) extracted from the Climatic Atlas of Clouds Over Land and Ocean(Eastman et al., 2014)for all six locations in question.Values are grouped by cloud altitude, with the low-level including fog, stratus (St), stratocumulus (Sc), cumulus (Cu), and cumulonimbus (Cb); the middle-level consisting of nimbostratus (Ns), altostratus (As), and altocumulus (Ac); and the high-level representing all cirriform clouds combined(Hahn and Warren, 2007).For the short-term measurement campaigns near Jülich and Tucson, only those data corresponding to the study periods are quoted.Note that Jülich and Oldenburg are located within the same grid box of the cloud atlas and hence feature identical values.
), with the exception of winter periods in Alderville, Varennes, and Oahu showing a somewhat slower T [-]Averaging time T [s] Seasons Figure4.As in Fig.3, for structures separately estimated for winter (December, January, February; DJF), spring (March, April, May; MAM), summer (June, July, August; JJA), and autumn (September, October, November; SON).The respective durations of available mixed-sky data are quoted in the appropriate color for each season and location.The normalization factors are the location-specific standard deviations σ k * T =1 s derived without conditioning by season (Fig.