Moderate spectral resolution solar irradiance measurements , aerosol optical depth , and solar transmission from 360 to 1070 nm using the refurbished Rotating Shadowband Spectroradiometer ( RSS ) 5

This paper reports on a third generation rotating shadowband spectroradiometer (RSS) used to measure global and diffuse horizontal plus direct normal irradiances and transmissions at 1002 wavelengths between 360 and 1070 nm. The prism-dispersed spectral data are from the ARM Southern Great Plains site in north central Oklahoma (36.605 N, 97.486 W) and cover dates between August 2009 and February 2014. The refurbished RSS isolates the detector in a vacuum chamber with pressures near 10-7 torr. This prevents the deposition of outgassed vapors from 15 the interior of the spectrometer shell on the cooled detector that affected the operation of the first commercial RSS. Methods for (1) ensuring the correct wavelength registration of the data and (2) deriving extraterrestrial responses over the entire spectrum, including throughout strong water vapor and oxygen bands, are described. The resulting data produced are archived as ARM data records and include cloud-screened aerosol optical depths as well as spectral irradiances and solar transmissions for all three solar components. 20


Introduction
The rotating shadowband spectroradiometer was developed to provide spectrally-resolved measurements of the 25 shortwave spectrum for the Atmospheric Radiation Measurement (ARM) program (Stokes and Schwartz, 1994). The instrument measures global and diffuse horizontal irradiance by alternately shading and unshading the diffuser that serves as the 2p steradian input optic to the spectrometer; it then calculates direct normal irradiance from these and a laboratory-measured, spectrally-dependent correction for the angular (cosine) response of the receiver.

30
To date RSS data have been used in several studies, for example; to derive water vapor by fitting a model to direct spectral irradiance data (Kiedron et al., 2001), to derive water vapor in overcast conditions by using diffuse spectral irradiance (Kiedron et al., 2003), to measure the photon pathlength to help decipher the structure of clouds over the ARM site in northern Oklahoma (Min and Harrison, 2009;Min et al., 2001;Min and Clothiaux, 2003), and to better understand aerosol retrievals (Gianelli et al., 2005) using the expanded wavelength data set.
The spectrograph contains two prisms in tandem after the collimating lens to achieve a moderate spectral resolution, 50 which has a FWHM (full width at half maximum) of 0.6 nm near 360 nm and FWHM of 7 nm near 1070 nm. The chromatic aberration in this system requires that the detector, positioned after the focusing lens, be tilted to optimize the focus at all wavelengths. The light from the sun and/or sky passes through a diffusing disk that is designed to provide an approximate Lambertian (cosine) response to incoming radiation at all wavelengths. The light from the diffuser enters an integrating cavity that has an exit slit that passes light to the optical train discussed above. The 55 band that shadows the diffuser is positioned below the horizon at the beginning of every cycle where dark and then global horizontal measurements are made; it then moves to three positions near the sun and samples at each of these. Two of these stops are near and on either side of the sun, but do not block it; the mid-stop totally blocks direct sunlight. The sideband measurements are used to calculate a first order correction for excess skylight blocked by the band during the measurement with totally blocked direct sunlight. Using these measurements and pre-deployment, 60 wavelength-dependent cosine response corrections, global and diffuse horizontal and direct normal irradiances can be calculated for 1002 continuous wavelengths.
The inside of the spectrograph is maintained at a temperature near 45° C, but the detector itself is maintained at a temperature near 20° C. This provided a detector surface in the original design that was a target for condensates 65 from outgassing vapors from the interior of the spectrograph that caused the responsivity to change over time. In the original commercial instrument, this meant frequent recalibrations to keep up with the changing response of the detector. In this new design the detector is mounted on a copper cold finger and housed in a windowed vacuum chamber that is held at a pressure of around 10 -7 torr , which effectively eliminates the condensation issue (see Figure 1).

3 Operational Details
The refurbished RSS was deployed at the ARM site in northern Oklahoma (36.605° N, 97.486° W, 317 m) and began operating on 26 August 2009; the shutter stopped opening on 21 February 2014. Note that between 25 75 December 2013 and 21 February 2014 shutter operation was intermittent, thus many data were lost during these last three months of operation. The cycle of four irradiance measurements plus one dark measurement was repeated every minute starting at the top of the minute. Exposure times for each measurement were between 0.4 and 4 seconds based on the irradiance level at the start of the cycle. There is a small variation in the sampling time caused by band travel to the sun's position. In the morning the sampling is nearer the top of the minute and somewhat later 80 in the afternoon. The band motor speed is, however, about one revolution per 6 seconds, therefore, the delay in afternoon sampling is less than this.
There were some issues with band alignment, which were detected using a Fast-Fourier Transform (FFT) procedure, that resulted in data being flagged as suspect. Furthermore, the initial five weeks of data showed suspicious 85 wavelength dependencies in the AODs, perhaps caused by a few, poorly determined extraterrestrial response retrievals at the beginning of the measurement set, and, therefore, we suggest that data taken before 1 October 2009 have a large uncertainty; however, irradiances and transmissions for the first five weeks were not removed from the database and should be used with caution.

Wavelength Registration
Although the spectrograph is rigidly secured to the frame that stabilizes it, slight changes in pixel alignment (note that pixels are about 14 µm wide) can occur due to slight mechanical or optical shifts associated with the thermal environment or external movements of the instrument, for example, caused by high wind speeds. During the four 95 and a half years of measurements, pixel shifts to shorter and longer wavelengths of up to four pixels in either direction were noted. Since we wish to use the same wavelength coverage for the entire period, we were left with 1002 pixels whose response was deemed satisfactory over the complete period of record. This included wavelengths between 360.4 and 1070.1 nm.

100
To ensure proper wavelength registration, nine global horizontal spectra, typically taken near solar noon each day, were averaged. This average was compared to the global horizontal spectrum used as a standard for all days where the wavelength registration was carefully determined using pencil lamp spectra and solar absorption features. Both of these global horizontal spectra were fitted with low order spline fits that were subtracted from each to enhance the https://doi.org/10.5194/amt-2021-162 Preprint. Discussion started: 23 June 2021 c Author(s) 2021. CC BY 4.0 License. major absorption features. A cross-correlation was performed between these two residual spectra at tenth-pixel 105 increments until the correlation between spectra reached a maximum. Only wavelengths less that 672 nm were used for the spline fit and cross correlation because these absorption features were mostly in the solar spectrum, and the terrestrial spectrum only has weak absorption lines in this part of the spectrum. We did not want to cross-correlate terrestrial features on different days where the water vapor content may have influenced the cross correlation. The spectra for that day were then shifted by the amount of pixel offset from the 'standard' spectrum. This spectral shift 110 process is illustrated in Figure 2. Note that the blue-colored spectrum on the left is displaced slightly shortward relative to the extraterrestrial and standard spectrum (designated 'tothor' in the figure). The shift in the left panel of Figure 2 was two pixels. The process described above was then perform and the blue-colored spectrum replotted on the right. On the right all three spectra are well aligned. This shift for the spectra is performed only once each day since the shifts that are observed undergo slow, subtle changes.

Estimation of Extraterrestrial Response in Strong Terrestrial Absorption Bands
It is our goal to generate a continuous spectrum of the transmission over the entire wavelength span of the RSS.

120
Transmissions are calculated by dividing the measured response of the RSS by the response at the top of the atmosphere (TOA) adjusted for solar distance. Many portions of the solar spectrum can use Langley analysis to estimate the TOA response. However, measurements in the strong bands of H2O and O2 are not well-suited for Langley analysis, since a linear curve of growth is not expected for these strong molecular bands, and, consequently, the extrapolation to zero airmass underestimates the TOA at these wavelengths.
125 Reagan et al. (1987) and Bruegge et al. (1992) were among the first to perform a modified Langley analysis to derive water vapor, but as Michalsky et al. (1995) pointed out this method depends on stable water vapor over the measurement period used for the modified Langley, which is seldom the case. Consequently, a very large number of modified Langleys, which require a very long period of time and a stable instrument over that time, are typically 130 required to even approach the accuracy in extraterrestrial response that a standard Langley analysis can achieve when working outside strong molecular bands.
In this paper we used an interpolation over two strong O2 bands and three strong H2O bands. The function used for the interpolation is given by equation (1) 135 where ! " ( ) is the estimated extraterrestrial response in the molecular band at wavelength l; subscripts 1 and 2 of l indicate ! ( )'s determined from standard Langley analysis, RSS responses R(l), and extraterrestrial irradiances ET(l) at wavelengths l1,2 just before and just after each of these five molecular bands, respectively.

140
Extraterrestrial (ET) solar irradiance at RSS spectral resolution was determined using the slit function of the RSS applied to the high spectral resolution ET spectrum of Kurucz (http://rtweb.aer.com/solar_frame.html), but scaled to the low-resolution, but well-determined absolute ET spectrum of Gueymard (2006). Figure 3 displays the extraterrestrial spectrum at RSS spectral resolution with higher spectral resolution in the short wavelengths and 145 lower resolution at long wavelengths as expected for a prism spectrograph.
The response function R(l) in equation (1) was determined by dividing the calibration lamp output in W/m 2 -nm by the RSS calibration function in W/m 2 -nm per count. Figure 4 is a plot of the RSS responsivity. We only require relative response for the interpolations using equation (1).
150 Figure 5 illustrates the V0 interpolation using equation (1) when implemented for five strong molecular bands within the wavelength span of the RSS. The black line is the extraterrestrial spectrum (y-axis label); the magenta line is the scaled, uncalibrated total horizontal irradiance in counts that is our standard for wavelength registration as discussed previously; the solid green line is a scaled uncalibrated retrieval of the RSS extraterrestrial response from a morning Langley plot with the green dots representing the estimated extraterrestrial response for the strong bands of O2 and H2O. The Ha and Na lines that are in the extraterrestrial solar spectrum are identified and appear in all three spectra. The interpolations of the extraterrestrial spectrum over the O2 and H2O bands seem reasonable and will be used for extraterrestrial responses as these are expected to be plausible estimates. With these estimates we can now calculate the continuous global and diffuse horizontal and direct normal solar spectral transmissions from 360 to 1070 nm.

4 Solar Transmission Calculations and Examples
As explained in the previous paragraphs, now that we have estimates for the extraterrestrial response over the entire RSS spectral response wavelength span, we can calculate transmissions in all three components; global and diffuse 165 horizontal and direct normal. Figure 6 is a plot of transmission near solar noon on 27 October 2009, which was clear from horizon to horizon at this time of day. Compared to the structure at short wavelengths in the extraterrestrial spectral irradiance (see Figure 3), the transmissions are extremely monotonic up to 550 nm and mostly the result of Rayleigh scattering and aerosol extinction, although Rayleigh scattering and aerosol extinction contribute throughout the entire RSS spectral range with decreasing contributions at the longer wavelengths. Both Rayleigh 170 scattering (l -4 ) and aerosol scattering (often ~ l -1.3 ) are wavelength dependent with the contributions falling off with increasing wavelength. This explains the lower transmission at the shortest wavelengths for the dni and the higher transmission at the shortest wavelengths for the dhi. There is only a minor indication of imperfect wavelength registration (smaller than 0.1 nm) that gives rise to small residuals of the very strong H and K lines of singly ionized calcium (CaII) at 393.4 and 396.9 nm in the solar spectrum. The major absorption bands of O2 near 690 and 760 nm 175 and H2O near 725, 820, and 940 nm, as noted in Figure 5, can be clearly identified. Much less obvious are the O2-O2 bands near 477, 577, and 630 nm, although the latter two bands are near a weak H2O water band (577 nm) or near weak H2O and O2 bands (630 nm) that complicate their identification and separation. The slight falloff in dni at the longest wavelengths near the end of the RSS spectrum is mostly the result of another weak O2-O2 band centered near 1065 nm. Somewhat less discernible is the broad Chappuis O3 band center roughly near 610 nm. Although no 180 retrievals will be performed in this paper, the transmission in the H2O bands can be used to estimate column H2O by running a suitable radiative transfer code at the RSS spectral resolution in transmission until an optimum match to the three H2O bands is attained. Less obvious and more uncertain is the use of the Chappuis band transmission to estimate column O3. Notice that the dni transmission on this clear day is about 95% at wavelengths above 1000 nm and slightly higher for ghi.

185
Figure 7 is interesting in that this is a plot with a clear path to the sun with ghi exceeding 100% transmission at wavelengths greater than about 650 nm (except, of course, for the strong absorption features). dni is slightly lower than it was in Figure 6 and dhi is considerably higher.

190
We can understand how transmissions above 100% are possible if we consider Figure 8, where the vertical red line marks the time of day that the spectra in Figure 7 were measured. dni appears unaffected at this point in time, but a short time later clouds are clearly moving in front of the sun. Because clouds are encroaching, diffuse is enhanced by direct sunlight scattering from them, which causes enhancements in the ghi. This is a well-recognized effect in broadband solar measurements that is discussed, for example, in Vignola et al. (2020) (see p. 21 and Figure 2.14 in 195 that reference). Additionally, Figure 7 illustrates the wavelengths where the apparent transmission exceeds 100%. Figure 9 is a plot of the transmission through a totally overcast sky on 25 October 2009 at the ARM site in northern Oklahoma. That the sky was totally cloudy was confirmed using the total sky imager collocated at the site. Note that the dni is zero for all wavelengths and the ghi is hidden by the dhi plot that should and does exactly overlay it for 200 overcast conditions. The spectrum outside molecular absorption bands indicates a slight monotonic increase in transmission with wavelength. Although there is a slight decrease in transmission centered near 600 nm that appears to counter this suggestion that the continuum monotonically increases with wavelength, this is the broad, weak Chappuis O3 band. The large H2O and O2 bands identified in Figure 5 are labeled left to right starting around 690 nm. Much weaker absorption bands below 690 nm can be identified. Using Sierk et al. (2004) allows us to identify 205 the three weak bands between 550 and 690 nm with the molecules causing the absorption features. The very weak depression around 477 nm is caused by O2-O2 (aka, O4) absorption ). The feature labeled H2O? is likely a water vapor band that is in the HITRAN (2012) database. The Ca II H and K lines short of 400 nm appear as small residuals because of slightly imperfect wavelength registration, (smaller than 0.1 nm) but this is a minor issue since these are the two strongest lines in the extraterrestrial spectrum. The downturn at the longest 210 wavelengths is absorption in the O4 band that is centered near at 1065 nm although the wings of a H2O bands may be influencing this part of the spectrum as well. Figure 10 is a similar plot on 24 October 2009 with slightly higher transmission. This plot shows a more pronounced Chappuis O3 band centered near 600 nm, which is the broad depression in what looks like continuum.
215 Figure 11 is a spectral irradiance plot from 24 December 2009 for totally overcast skies that has a starkly different appearance than the earlier plots (Figures 9 and 10) for overcast days. On this day at the time of this measurement the radar indicated substantial ice content in the clouds above the site. The attenuation above 1000 nm is consistent with radiance transmission spectra for ice clouds presented in LeBlanc et al. (2015); for example, compare their Figure 3(b). Consequently, there is the potential to use these spectra to recognize ice phase and, more importantly, 220 retrieve quantitative information on ice content and size. There is clearly an incentive to study this further, although it is beyond the current focus of this paper.

225
Since transmissions have been calculated and are available, it should be straightforward to calculate aerosol optical depths (AODs) for the parts of the spectrum that are free of strong H2O and O2 absorption bands. Since we have determined estimates for extraterrestrial responses V0(l)'s, and we measure responses at the surface V(l)'s, we can calculate optical depths by solving for t in the following where m is a known, calculated airmass, t(l) is the total optical depth, and V0(l) has been adjusted for the correct solar distance for the time of observation. From the total optical depth t(l) we must remove the optical depths associated with Rayleigh scattering and Chappuis band O3 absorption to retrieve estimates for AODs where there are no molecular bands.
235 Figure 12 is a plot of optical depth versus wavelength at 11:45 local standard time for 25 November 2009 at the ARM site with Rayleigh and ozone optical depth removed. The vertical lines are positioned at the CIMEL sunphotometer wavelengths used by AERONET in this wavelength range (https://aeronet.gsfc.nasa.gov/ (Holben et al., 2001) to measure AOD. All of these appear to be in windows not affected by the strong O2 and H2O bands. As an aside, note that if we examine this and any of our similar transmission figures that the weak water band centered 240 near 505 nm could have a small influence on the AOD assigned to this wavelength even though it has long been a standard wavelength for AOD measurements (WMO/GAW, 2016).
The shorter wavelengths appear noisier because the wavelength alignment is to the nearest 0.1 pixels where the spectral resolution is the highest for this instrument, the signal-to-noise ratio is lowest at these wavelengths, and the 245 extraterrestrial spectrum is inherently more structured at these wavelengths for a prism spectrograph. Figure 13 is a log-log plot of RSS AODs (open circles) versus wavelength for the RSS pixels nearest in wavelength to the AERONET wavelengths. For the 1020 nm pixel an additional minor correction for water vapor (original marked by 'X" in Figure 13), based on the work presented by Smirnov (2004, unpublished), brings the wavelength dependence more in line with the shorter wavelength points. The slope (Angstrom exponent) determined by a linear fit to these 250 RSS data is 0.711, not uncommon for late autumn aerosols. The filled circles are the nearest in time AERONET data (within 2.5 minutes) for the AERONET wavelengths. Other than the 1020 nm points, the data are within 0.008 optical depths with similar wavelength dependencies.

255
Just as for the aerosol optical depth calculations, if we are given the transmission, it is straightforward to calculate the spectral irradiance for all three components. All that is required is to use the extraterrestrial (ET) spectral irradiance at the spectral resolution of the RSS (see Figure 3) and adjust for the solar distance at the time of the measurements. Multiplying each component (ghi, dni, and dhi) by this distance-corrected ET spectral irradiance for 260 each of the pixels yields the estimated spectral irradiance for each component at the surface. ET spectral irradiance. Transmission uncertainty is estimated at 1%, but the ET uncertainty is 2% or higher, as 265 indicated by Gueymard (2018;private communication), depending on what portion of the 360-1070 nm spectral range is under study. Of course, for cloudy periods the dni will be zero and the dhi and ghi will be the same as can be inferred from Figures 9-11. In Figure 13 note that the ghi is not the sum of dhi and dni because dni is not the direct projected onto a horizontal surface, which would be dni · cos(sza).

7 Data Availability
For those interested, the data are freely available and are archived with the ARM program and can be downloaded using https://iop.archive.arm.gov/arm-iop/0pi-data/michalsky/RSS/. Some additional notes on data quality are at ftp://aftp.cmdl.noaa.gov/user/michalsky/ in the folder 'asked_for_stuff'.