Above-aircraft cirrus cloud and aerosol optical depth from hyperspectral irradiances measured by a total-diffuse radiometer

This study develops the use of spectral total and diffuse irradiance measurements, made from a prototype hyperspectral total-diffuse sunshine pyranometer (SPN-S), to retrieve layer fine-mode aerosol (τ_aer) and total optical depths from airborne platforms. Additionally, we use spectral analysis in an attempt to partition the total optical depth into its τ_aer and cirrus cloud optical depth (τ_cld) components in the absence of coarse-mode aerosols. Two retrieval methods are developed: one leveraging information in the diffuse irradiance and the other using spectral characteristics of the transmitted direct beam, with each approach best suited for specific cloud and aerosol conditions. The SPN-S has advantages over traditional sun photometer systems, including no moving parts and a low cost. However, a significant drawback of the instrument is that it is unable to measure the direct-beam irradiance as accurately as sun photometers. To compensate for the greater measurement uncertainty in the radiometric irradiances, these retrieval techniques employ ratioed inputs or spectral information to reduce output uncertainty. This analysis uses irradiance measurements from the SPN-S and the solar spectral flux radiometer (SSFR) aboard the National Aeronautics and Space Administration's (NASA) P-3 aircraft during the 2018 deployment of the ObseRvations of Aerosols above CLouds and their intEractionS (ORACLES) campaign and the 2019 Cloud, Aerosol and Monsoon Processes Philippines Experiment (CAMP²Ex) mission to quantify above-aircraft cirrus τ_cld and derive vertical profiles of layer τ_aer. Validation of the τ_aer retrieval is accomplished by comparison with co-located measurements of direct solar irradiance made by the Sky-Scanning Sun-Tracking Atmospheric Research (4STAR) and in situ measurements of aerosol optical depth. For the aggregated 2018 ORACLES results, regression between the SPN-S-based method and sun photometer τ_aer values yields a slope of 0.96 with an R² of 0.96, while the root mean square error (RMSE) is $\mathrm{3.0}\times {\mathrm{10}}^{-\mathrm{2}}$. When comparing the retrieved τ_aer to profiles of integrated in situ measurements of optical extinction, the slope, R², and RMSE values for ORACLES are 0.90, 0.96, and $\mathrm{3.4}\times {\mathrm{10}}^{-\mathrm{2}}$, and for CAMP²Ex they are 0.94, 0.97, and $\mathrm{3.4}\times {\mathrm{10}}^{-\mathrm{2}}$, respectively.

This paper is a demonstration of methods for deriving cloud and aerosol optical properties in environments where both atmospheric constituents may be present. With improvements to the low-cost SPN-S radiometer instrument, it may be possible to extend these methods to a broader set of sampling applications, such as ground-based settings.

1 Introduction

Clouds and aerosol particles both play important roles in controlling the flux of solar radiation through the Earth's atmosphere. Despite their relevance to the broader climate system and Earth's radiative balance, significant uncertainty exists in quantifying the optical properties of atmospheric systems containing one or both constituents. Traditional passive remote sensing methods retrieve aerosol properties in the absence of clouds (Holben et al., 1998; Levey et al., 2013). This is because when clouds are thick their radiative signal is large in relation to the aerosol signal, and when clouds are thin it is difficult to separate the two signals. For automated aerosol optical depth (τ_aer) retrievals, the challenge of cloud detection and removal is a significant hurdle to overcome (Smirnov et al., 2000; Remer et al., 2012; Spencer et al., 2019; Yang et al., 2019). Advanced methods, such as the spectral deconvolution algorithm (SDA), have been developed to differentiate between fine- and coarse-mode τ_aer using spectral sun photometry data (O'Neill et al., 2003), though these techniques are limited when cirrus is present (Smirnov et al., 2018). Conversely, retrieval of cloud optical depth (τ_cld) tends to be insensitive to the aerosol loading of the local environment because clouds are often much thicker optically than aerosols. A common exception to this occurs when thin cirrus clouds are present. Reported values for mean cirrus cloud optical depth vary regionally (Giannakaki et al., 2007; Dai et al., 2019) but tend to be less than a value of unity (Kox et al., 2014; Heymsfield et al., 2017; Zhou et al., 2018). Since it is common for cirrus to have optical depths similar to those of aerosols, which typically have values less than 4 in equatorial regions (Torres et al., 2002), remote sensing of the optical depth of either constituent is complicated by the presence of the other. Other thin-cloud types, such as low-level clouds in the Arctic, can have relatively high rates of occurrence and therefore pose challenges when using remote sensing techniques to study cloud or aerosol optical properties. For the case of thin Arctic clouds, Garrett and Zhao (2013) demonstrated the utility of thermal spectral remote sensing to derive the optical properties when the clouds have an emissivity less than unity.

The necessity of retrieving both aerosol and cirrus cloud optical properties is supported by the fact that aerosols are ubiquitous throughout Earth's atmosphere, and cirrus clouds are globally prevalent (Sassen et al., 2008). Cirrus presence is especially high in equatorial regions, where their frequency of occurrence can be near 50 % (Sassen et al., 2008; Zou et al., 2020). Overlying cirrus limits remote sensing of cloud and aerosol properties by passive airborne radiometers and polarimeters (e.g., Werner et al., 2013; Stamnes et al., 2018), and ground-based sun photometer retrievals of τ_aer frequently suffer from contamination by these clouds (Chew et al., 2011). In the past, attempts have been made to account for and correct retrievals of τ_aer for the impact of cirrus (Lee et al., 2013). Efforts have also been made to use sun photometry to derive cirrus τ_cld (Kinne et al., 1997; Segal-Rosenheimer et al., 2013), but work towards joint τ_cld and τ_aer retrieval is limited.

In this paper we address some of the issues associated with remote sensing of thin-cloud and aerosol systems by leveraging the capabilities of a new hyperspectral total-diffuse radiometer, SPN-S. The advantage of this radiometer system is that it is low-cost and deployable to a wide range of environments. The device is mechanically simple, with a fixed shadow mask used to block the direct beam of the sun to make measurements of the diffuse irradiance. This shadow mask design allows for simultaneous measurements of both the total and diffuse fluxes, which is functionality that traditional rotating shadow band radiometer systems cannot obtain. The concurrent sampling of the two irradiances is useful in airborne or other dynamic settings where scenes can change rapidly. However, radiometer systems like the SPN-S have higher measurement uncertainties than sun photometer systems, which limits their ability to investigate atmospheric optical properties. To compensate for lower accuracy, we propose a method for deriving τ_cld of thin clouds using narrowband measurements of the diffuse-to-total ratio (DR). This diffuse ratio method, which is referred to as RD, is advantageous for the study of thin clouds (τ<1) because its main radiometric input is a ratio of two measured irradiances made from the same instrument, which reduces absolute calibration-induced errors. By using the ratio of two radiometric quantities as the retrieval input, the measurement uncertainty affecting the retrieval output is strongly dependent on the instrument precision, which is low in relation to the overall radiometric calibration of the system. As a result, the RD method reliably detects thin clouds because it is highly sensitive to small variations in optical depth. The concept of using a ratio of two spectral measurements to study thin clouds was previously developed by Garrett and Zhao (2013), who used a ratio of measured thermal emission to study thermodynamic phase and optical properties of thin Arctic clouds. A second method, which we refer to as the spectral direct-beam method (RS), uses measurements of the direct irradiance to develop optical depth spectra. The shape of the spectral optical depth curve contains information on the loading of fine-mode and large, coarse-mode particles in the atmosphere. In the absence of coarse-mode aerosols we show the potential to retrieve τ_aer and τ_cld values.

The two methods presented in this study overlap with previous cloud and aerosol retrieval techniques. DR has been used to study aerosol single-scattering albedo and asymmetry parameter properties using measurements from multi-filter rotating shadow band radiometers (MFRSRs; Kassianov et al., 2007; Herman et al., 1975). Although MFRSRs are the most widely used total-diffuse radiometer system, past work utilizing the instrument has relied on the direct irradiance measurement when deriving τ_aer (Michalsky et al., 2010). Likewise, the use of spectral shape of the transmitted direct beam in the RS method is similar to the SDA method developed by O'Neill et al. (2003). However, MFRSR and sun photometer systems require extensive alignment and precise operating conditions, making these past methods not applicable to airborne settings and restricting them to use at the surface. Extending these previous works and then using them in conjunction with a shadow-mask-designed radiometer allows for a broader set of applications. Specifically, the use of the SPN-S in airborne settings (or other non-stationary environments) is a novel application of a spectral total-diffuse radiometer system which allows for greater detail of the atmospheric aerosol and cloud structure to be known.

This paper describes the theoretical underpinnings of the retrievals in Sect. 2, along with a justification of which method is best suited for specific atmospheric conditions. Section 3 overviews the data used in the study from two airborne field campaigns: the 2018 deployment of the ObseRvations of Aerosols above CLouds and their intEractionS (ORACLES) campaign conducted over the Southeast Atlantic Ocean and the 2019 Cloud, Aerosol and Monsoon Processes Philippines Experiment (CAMP²Ex) mission conducted above the waters surrounding the Philippines. Section 3 follows with a detailed accounting of the retrieval algorithm implementation using the field measurements. Section 4 presents the results: first a comparison of retrieved τ_aer values to co-located τ_aer measurements made by a sun photometer as well as in situ measurements of the aerosol optical depth. Then summary statistics of the ORACLES and CAMP²Ex campaigns are shown. Section 5 is a discussion of limitations and usefulness of the two new methods. Section 6 provides a brief summary of the paper.

2 Theory and approach

The attenuated direct solar radiation in a layered model of the atmosphere is directly related to the layer optical depth by Beer's law:

$$\begin{array}{}\text{(1)}& F_{\text{dir}}=F_{\mathrm{0}}{e}^{-\mathit{\tau }/\mathit{\mu }},\end{array}$$

where F_dir is the direct-beam irradiance, F₀ is the incident irradiance at the top of the layer, μ is the air mass factor which we approximate as the cosine of the solar zenith angle (SZA), and τ is the layer optical depth. Inverting Eq. (1) is frequently invoked to derive optical depth from measurements of F_dir.

Conversely, the two-stream approximation of the radiative transfer equation can be used to relate the downwelling diffuse irradiance to the optical depth (Eq. 5.66, p. 263 in Bohren and Clothiaux, 2006):

$$\begin{array}{}\text{(2)}& {\displaystyle \frac{F_{\text{dif}}}{F_{\mathrm{0}}}}={\displaystyle \frac{\mathrm{1}}{\mathrm{1}+\mathit{\tau }(\mathrm{1}-g)/\mathrm{2}\mathit{\mu }}}-e^{-\mathit{\tau }/\mathit{\mu }},\end{array}$$

where g is the asymmetry parameter of the single atmospheric layer (the mean cosine of the scattering angle, which in the context of the two-stream approximation is a parameter that describes the relative amounts of forward and backward scattering within a layer), and F_dif is the downwelling diffuse irradiance. It is important to note that Eq. (2) assumes no absorption, and the atmospheric layer is above a black surface. By solving Eq. (1) for F₀, substituting this into Eq. (2), and consolidating all the irradiance terms to the left-hand side, we get an expression relating the diffuse ratio (DR) to optical depth:

$$\begin{array}{ll}{\displaystyle }\text{diffuse ratio (DR)}& {\displaystyle }={\displaystyle \frac{F_{\text{dif}}}{F_{\text{dir}}+F_{\text{dif}}}}\\ \text{(3)}& {\displaystyle }& {\displaystyle }=\mathrm{1}-(\mathrm{1}+\mathit{\tau }(\mathrm{1}-g)/\mathrm{2}\mathit{\mu })e^{-\mathit{\tau }/\mathit{\mu }}.\end{array}$$

In the thin-cloud limit

$$\begin{array}{}\text{(4)}& {\displaystyle \frac{\mathit{\tau }(\mathrm{1}-g)}{\mathrm{2}\mathit{\mu }}}\ll \mathrm{1},\end{array}$$

Eq. (3) simplifies to

$$\begin{array}{}\text{(5)}& \text{DR}=\mathrm{1}-e^{-\mathit{\tau }/\mathit{\mu }},\end{array}$$

and the dependence of DR on g is minimal. Crucially, Eqs. (3) and (5) are not dependent on F₀, which allows for knowledge of τ to be obtained without information (or assumptions) of the irradiance incident upon the layer. These equations give us two relationships between observables, DR and F_dir, and optical depth. Equations (1) and (3) form the basis of the two retrieval methods: (1) using DR to derive τ_cld and (2) exploiting spectral features of F_dir to partition τ into τ_cld and τ_aer components.

2.1 Diffuse method, RD – τ_cld retrieval

Equation (3) directly links DR to optical depth for cases when the extinction of the layer is solely caused by scattering. Cloud particles have minimal absorption coefficients for light at visible wavelengths (Bohren and Huffman, 1998) and therefore a single-scattering albedo (SSA) near unity. Fine-mode aerosols are commonly absorbing, while coarse-mode aerosols, such as sea salt and dust, have similar scattering characteristics to clouds, both of which limit the application of RD to samples without aerosols (the implications aerosols have on RD are discussed in more detail in Sect. 5). Given these constraints, Eq. (3) can be solved to derive cloud optical depth. We apply this model specifically to retrieve cirrus cloud optical depth because, as we show in Sect. 2.3, DR is most sensitive to changes in τ when τ is small. At optical depths ${\mathit{\tau }}_{\text{cld}}>\sim \mathrm{5}$, DR asymptotes to unity, leaving little information on τ.

At smaller cloud optical depths, when a direct beam is still present (${\mathit{\tau }}_{\text{cld}}<\sim \mathrm{5}$), DR is linked to the amount of forward scattering promoted by the cloud medium, and therefore a source of uncertainty in RD is incomplete knowledge of g (or the full scattering phase function). For cirrus clouds g typically ranges from 0.7 to 0.9 (Fu, 2007). Likewise, surface albedo, a, impacts DR through a process of multiple scattering of light between the surface and the cloud layer. Since the analytic solution in Eq. (3) does not account for the impact of surface albedo on DR we use a radiative transfer model (RTM) to accurately represent the sampled environment and implement the retrieval. In field settings, the flight-level a (a_fl) is directly inferred by ratioing downwelling and upwelling total irradiance measured by the SPN-S and the solar spectral flux radiometer (SSFR), respectively.

The retrieval steps of the RD method are simple.

1. Given a measurement of diffuse ratio, DR_mea, we solve Eq. (5) to make an initial estimate of the cloud optical depth, τ_est.

2. Measured flight-level albedo (a_fl) and τ_est are used as inputs into the RTM to simulate a diffuse ratio, DR_sim; g is set to 0.85 in the RTM.

3. DR_sim and DR_mea are compared:

   • 3.1.

     If the difference is greater than ±1 %, τ_est is adjusted by a small amount, ${\mathit{\tau }}_{\text{est}}={\mathit{\tau }}_{\text{est}}\pm \mathrm{\Delta }\mathit{\tau }$, and step 2 is repeated.

   • 3.2.

     If the difference is less than ±1 %, τ_cld=τ_est.

4. The τ_cld value has bias induced by the wide field of view (FOV) of the SPN-S, and this bias is corrected for in certain sampling settings. Details of the FOV correction are found in Sect. 3.2 and Appendix A.

The details and specifics of the measurements and RTM used in the diffuse method, RD, will be discussed in Sect. 3 of this paper.

2.2 Spectral direct-beam method, RS – τ_cld,τ_aer retrieval

The spectral direct-beam method, RS, leverages the differences in the spectral dependence of layer optical depth containing small (fine-mode) and large (coarse-mode) particles. Layers containing small particles, with sizes roughly that of the wavelength of visible light, have a strong wavelength dependence in optical depth. For fine-mode aerosols τ_aer,λ can be described by

$$\begin{array}{}\text{(6)}& {\mathit{\tau }}_{\text{aer},\mathit{\lambda }}={\mathit{\tau }}_{\text{aer},\mathrm{0}}{\left({\displaystyle \frac{\mathit{\lambda }}{{\mathit{\lambda }}_{\mathrm{0}}}}\right)}^{-\text{AE}},\end{array}$$

where the lambda subscript of the τ_aer,λ term indicates a spectral dependence on the optical depth, τ_aer,0 is the optical depth at a reference wavelength λ₀, and AE is the Ångström exponent (Ångström, 1929).

Larger particles, such as coarse-mode aerosol (e.g., sea salt or dust) and cloud hydrometeors, reside close to or in the geometric scattering regime at visible wavelengths and therefore have low AE values. This results in minimal wavelength dependence of τ of atmospheric layers containing these large particles (i.e., τ is largely spectrally flat in visible wavelengths). In an atmospheric system containing fine- but no coarse-mode aerosols, we expect the spectral total layer optical depth to be of the form

$$\begin{array}{}\text{(7)}& {\mathit{\tau }}_{\mathit{\lambda }}={\mathit{\tau }}_{\text{cld},\mathit{\lambda }}+{\mathit{\tau }}_{\text{aer},\mathit{\lambda }}={\mathit{\tau }}_{\text{cld}}+\left[{\mathit{\tau }}_{\text{aer},\mathrm{0}}{\left({\displaystyle \frac{\mathit{\lambda }}{{\mathit{\lambda }}_{\mathrm{0}}}}\right)}^{-\text{AE}}\right].\end{array}$$

In practice we use Beer's law, Eq. (1), to determine τ_λ, where this spectral optical depth is also represented by the summation of all atmospheric constituents with non-zero optical depth:

$$\begin{array}{}\text{(8)}& {\mathit{\tau }}_{\mathit{\lambda }}={\mathit{\tau }}_{\text{cld},\mathit{\lambda }}+{\mathit{\tau }}_{\text{aer},\mathit{\lambda }}+{\mathit{\tau }}_{\text{Ray},\mathit{\lambda }}+{\mathit{\tau }}_{\text{mol},\mathit{\lambda }},\end{array}$$

where τ_Ray,λ is the spectral optical depth from Rayleigh scattering, and τ_mol,λ is a term encompassing non-Rayleigh extinction from trace gas molecular and water vapor absorption sources. In the wavelength range used in this analysis there is optical depth from trace gases and water vapor, but rather than measure or calculate the τ_mol,λ term, we select wavelengths that minimize its value. Further, we apply a correction to the derived τ_λ based on measured values of τ_λ from above the aerosol layer (see Sect. 3.5.2). That is, we are calculating a layer aerosol optical depth, and in doing so we account for much of the influence of τ_mol,λ on τ_λ. Future work may want to treat the τ_mol,λ term with more detail, which would possibly allow for column and not layer τ_aer,λ to be retrieved. Then if wavelengths are selected that minimize τ_mol,λ, τ_λ is dependent on τ_cld, τ_aer,λ, and τ_Ray,λ; τ_Ray,λ can be solved for empirically and is a function of the pressure differential across the observed layer:

$$\begin{array}{}\text{(9)}& {\mathit{\tau }}_{\text{Ray},\mathit{\lambda }}=c_{\mathrm{1}}\left[{\displaystyle \frac{c_{\mathrm{2}}-c_{\mathrm{3}}{\mathit{\lambda }}^{\mathrm{2}}-c_{\mathrm{4}}{\mathit{\lambda }}^{-\mathrm{2}}}{\mathrm{1}-c_{\mathrm{5}}{\mathit{\lambda }}^{\mathrm{2}}-c_{\mathrm{6}}{\mathit{\lambda }}^{-\mathrm{2}}}}\right]\left({\displaystyle \frac{\mathrm{\Delta }P}{\mathrm{1013.25}}}\right),\end{array}$$

where ΔP is the pressure differential across the layer in millibars, $c_{\mathrm{1}}=\mathrm{2.10966}\times {\mathrm{10}}^{-\mathrm{3}}$, c₂=1.0455996, c₃=341.29061 µm⁻², $c_{\mathrm{4}}=\mathrm{9.02308508}\times {\mathrm{10}}^{-\mathrm{1}}$ µm⁻², $c_{\mathrm{5}}=\mathrm{2.7059889}\times {\mathrm{10}}^{-\mathrm{3}}$ µm⁻², and c₆=85.968563 µm⁻² (Bodhaine et al., 1999). Using Eq. (9) to calculate and account for the τ_Ray,λ term in Eq. (8), the spectral shape of the layer optical depth dictated by Eq. (7) can be described in terms of τ_cld and τ_aer,λ. An example of this is illustrated in Fig. 1 for simulated optical depths (RTM configuration is given in Sect. 3.3) for a case with a cloud only (blue lines) and cloud with aerosol (red lines). A set of selected retrieval wavelengths, designated by the yellow shading, represents regions of the spectrum where τ_mol is minimal (additionally the shaded regions correspond with usable channels from the SPN-S; more in Sect. 3). For the aerosol-free case within the selected wavelength region, the τ_λ profile, which accounts for the τ_Ray term (dashed blue line), falls nearly along a flat line of the simulated cloud optical depth value of τ_cld=0.20. When fine-mode aerosols are present, τ_λ is curved in the form of Eq. (7) and asymptotes to a value of τ_cld=0.20 at longer wavelengths. The aerosol optical depth is simply the difference in the layer and cloud optical depths at 500 nm: ${\mathit{\tau }}_{\text{aer,500\hspace{0.17em}nm}}={\mathit{\tau }}_{\text{500\hspace{0.17em}nm}}-{\mathit{\tau }}_{\text{cld}}$.

Figure 1 Simulated spectral τ_λ for two cases. Cloud only (τ_cld=0.20, τ_aer,500 nm=0.00) is denoted by the blue lines, and a case with cloud and aerosols (τ_cld=0.20, τ_aer,500 nm=0.38) is shown by the red lines. The solid black line is calculated τ_Ray,λ. Subtracting τ_Ray,λ from τ_λ yields the dashed red and blue lines, which are represented by Eq. (7). The shaded yellow regions are the wavelengths selected for use in RS. For the aerosol-free case, the layer optical depth is spectrally flat at the value of τ_cld.

In practice RS is implemented as follows:

1. Spectral measurements of direct irradiance, F_dir, are used to determine τ_mea,λ using Beer's law (Eq. 1).

2. Wavelengths in window channels are selected that minimize the τ_mol,λ term in Eq. (8).

3. τ_mea,λ is characterized above the aerosol layer and used to correct all profile samples of τ_mea,λ for the influence of τ_mol,λ.

4. τ_Ray,λ is calculated from Eq. (9) and then subtracted from the corrected τ_mea,λ.

5. A set of calculated layer optical thicknesses, τ_calc,λ, are found using Eq. (7) for a range of τ_cld, τ_aer,0, and AE values.

6. The root mean square error (RMSE) is found for each combination of τ_calc,λ and τ_mea,λ profiles. The retrieval outputs – τ_cld, τ_aer, and AE – are the values corresponding to the τ_calc,λ with the lowest RMSE.

7. A field-of-view correction is applied to the τ_cld retrieval output (see Sect. 3.2 and Appendix A).

2.3 RD and RS comparison and use selection consideration

Without knowledge of the output uncertainties in the two methods, the separation of the aerosol from the cloud radiative signal makes RS the more capable of the two retrieval methods. However, RS is based on measuring F_dir, which is a measurement that is susceptible to significant errors when made from airborne platforms. For the SPN-S, the main error sources of the irradiance measurements are the result of improper radiometric calibration, the sensor cosine response, and errors associated with the attitude of the sensor relative to the horizon. RD has the advantage of being derived from the ratio of two irradiances simultaneously measured by the same instrument. In the case of the SPN-S, the diffuse and total irradiance measurements are made by the same sensor, which allows us to assume that the radiometric uncertainties in F_dif and F_tot are correlated, and therefore the uncertainty in DR_mea is a function of the instrument precision and not the accuracy. Further, DR is minimally affected by sensor attitude errors because the F_dir term is in the denominator. However, there are additional sources of measurement uncertainty related to the SPN-S system that are important to consider, such as internal reflections caused by the shadow mask that can lead to bias in the measured diffuse irradiance. Internal reflections are a problem inherent to total-diffuse radiometer systems reliant on shadow mask (and shadow band) designs, and a detailed discussion of how these and related issues have been addressed for the SPN-S is given in Badosa et al. (2014). For the SPN-S irradiance measurements we estimate that a 4 %–6 % accuracy uncertainty results from the lamp calibration process, and up to another 2 % uncertainty stems from imperfect knowledge of the cosine response of the sensor. For the sake of this analysis, we use uncertainty values of 7 % accuracy and 0.5 % for precision (Wood et al., 2017). There are additional sources of error (e.g., changes in sensor attitude) that will be addressed in the following sections.

The performance of both retrieval methods is gauged considering the uncertainty in the inputs. First, we evaluate Eqs. (1) and (3) for a range of τ values. Figure 2 shows the profile of the direct transmittance, $T_{\text{direct}}=F_{\text{dir}}/F_{\mathrm{0}}$ (green line), given a 7.0 % uncertainty in F_dir. At high values of T_direct (low τ), the measurement uncertainty is a significant proportion of the signal, leading to substantial ambiguity in the associated value of τ. As optical depths increase, T_direct falls off exponentially at the same rate as its error, meaning the absolute error in τ is constant, and hence the fractional error decreases as the cloud layer thickens. For RD the uncertainty in DR is minimal, and so for a known value of g the retrieved τ has little ambiguity until the DR signal starts to asymptote to unity at values $\mathit{\tau }>\sim \mathrm{5}$. However, the uncertainty in the value of g is a main source of error when determining τ at low optical depths. The pink and purple lines are DR profiles for g ranging from 0.5–1.0. At low values of DR, when scattering is minimal, the uncertainty in g leads to relatively little ambiguity in the value of τ (i.e., g is not represented in Eq. 5). As the cloud optical depth increases, greater amounts of scattering occur, and the importance of g in DR becomes more pronounced, leading to poorer retrieval performance. Fortunately, the two methods for deriving τ are complementary; RD has lower uncertainty in retrieving thin-cloud τ, whereas the use of RS is justified as τ becomes larger. This threshold at which the output uncertainty in RS falls below RD is approximately when τ∼1, though this level is dependent on how well g is constrained and the level of uncertainty in the inputs. Both methods lose utility as τ becomes large, and the light becomes completely diffuse.

Figure 2 Solutions given the direct transmittance using Beer's law (Eq. 1) and the diffuse ratio (Eq. 3) for given layer optical depths. Shown are uncertainties in optical depth for irradiances associated with layer optical depths of 0.2, 1.0, and 3.0. Uncertainty in F_dir is 7.0 %, and uncertainty in DR is 0.5 %.

To further address the question of retrieval performance, we test the explicit retrieval methods used in this study (outlined in Sects. 3.4) on model-generated irradiance data. To do this we simulated a set of spectral irradiances using an RTM (see Sect. 3.3) for values of τ_cld ranging from 0 to 6. The cloud optical properties in the simulations were generated using the “hey” ice cloud parameterizations that are a part of the public libRadtran package (Yang et al., 2013; Emde et al., 2016). For this example, a cloud comprised of smooth solid-column ice crystals with an effective radius of 20 µm was inserted in the atmosphere with a base height of 10 km, and we set a=0.15. In practice, we directly infer a using SPN-S and SSFR measurements, so in this exercise we do not investigate the influence of albedo on the retrieval error. The DR profiles are calculated at three values of g: 0.70, 0.85, and 0.95. The simulated inputs, F_dir for RS and DR for RD, are then injected into both retrieval structures, and the outputs are compared to the true values of τ_cld – the values of τ_cld set in the RTM simulations.

Figure 3 (a) Simulated 500 nm irradiances at 4 km altitude, with an ice cloud situated at 10 km. Solar zenith angle is 20^∘; ice crystal effective radius is 20 µm. (b) Retrieval outputs with the simulated irradiances used as inputs. Error in F_dir is 7.0 %; error in DR is 0.5 %. The uncertainty in retrieval output is represented by the shading. For RD, this uncertainty also reflects g values of 0.70, 0.85, and 0.95. (c) Percent error in the two retrieval methods when compared to the true cloud optical depth.

Figure 3 shows the results of this experiment. The left panel is the simulated 500 nm downwelling irradiances. The middle panel shows the retrieved values of τ_cld plotted against the true τ_cld, and the right panel is the error in retrieved τ_cld value as a function of true τ_cld. The results support the analysis of the analytic functions presented in Fig. 2: when ${\mathit{\tau }}_{\text{cld}}<\sim \mathrm{1}$ the output of the RD method has less uncertainty in the retrieved optical depth, making it the superior method. When τ_cld is greater than unity, there is less uncertainty associated with the retrieved optical depth from the RS method. It is critical to note that this analysis of cloud simulations assumes an absence of aerosols. If absorbing aerosols are present, knowledge of SSA and g is needed to accurately use RD to retrieve optical depth.

Another limitation that is worth mentioning is that the output of RD falls below the identity line because of the difference in the scattering phase functions used in the simulated data and the DR calculations in the retrieval. In RD, the phase function is approximated using the Henyey–Greenstein phase function for a given value of g, while the simulated data use full phase functions for cloud layers containing solid-column ice crystal habit from the libRadtran library. We use Henyey–Greenstein in the retrieval to make it more broadly applicable to various atmospheric conditions and to avoid making assumptions about the cloud microphysics.

3 Data and implementation

This section details the data sources used in the retrievals and in validation of their outputs. Additionally, the RTM used throughout this study is described. This is followed by a detailed explanation of the procedures used to implement RS and RD with the field data.

3.1 ORACLES and CAMP²Ex campaigns

The data used in this study come from two independent field campaigns that occurred in climatically different tropical to subtropical regions of the globe. Both campaigns used the National Aeronautics and Space Administration's (NASA) P-3 aircraft equipped with a set of radiometric and in situ scientific instruments, with the goal of addressing questions surrounding cloud and aerosol impacts on atmospheric state. Specifically, ORACLES was a three-part mission focused on studying the radiative effects of biomass-burning-generated aerosols present above clouds. Sampling of the aerosol plume was done over the eastern Atlantic Ocean after it had advected off the west coast of the African continent. The experiment environment at ORACLES was stratified, with a persistent aerosol layer sitting below clear skies (i.e., minimal cirrus clouds). A stratus cloud layer of varying cloud fraction was often found below or at the bottom of the aerosol layer. While cirrus did tend to occur in the sampling region, flight planning was done to avoid their presence. This environment allowed for measurements of aerosol properties to be made with minimal influence from surrounding clouds outside of the impact the low cloud layer had on albedo. The science flights took place in August–October of 2016–2018. The SPN-S was only developed in time for operation during the 2018 deployment of the ORACLES mission. A complete overview of the ORACLES campaign is given by Redemann et al. (2021).

In contrast, CAMP²Ex took place from late August through early October of 2019 in the maritime environments surrounding the island of Luzon, Philippines. This location was selected because polluted air masses from Borneo and Sumatra could be characterized as they are transported through the South China Sea and Sulu Sea into the western Pacific by the Maritime Continent's southwest monsoon flow. Sampling took place in several different air masses, which had variability in the aerosol source: local sources from industrial activity on Luzon (especially from Manila) and tanker ships to biomass burning aerosols transported from Borneo and long-range transport of aerosols from the Asian continent. Given the nature of the southwest monsoon, the cloud environment was highly dynamic, with cumulus clouds and convective cells of varying degrees of maturity present during all flight days. Unlike with ORACLES, where cirrus could be avoided, during CAMP²Ex regional deep convection led to ubiquitous cirrus. It is this high prevalence of the cirrus clouds during the mission which provided much of the motivation for developing the spectral approaches for determining τ_cld and τ_aer. CAMP²Ex also sampled for 10 d after the monsoon transition, leading to lower cirrus optical depths and sampling of the heavily polluted Asian air masses. For an overview of the CAMP²Ex mission, refer to Reid et al. (2022).

Regarding the sampling methods, note that during both campaigns we employed a “square-spiral” sampling technique to profile layers of the atmosphere up to two times per science flight. This is where the P-3 aircraft descends through the atmosphere, from high-altitude flight to near-surface, by making a box pattern. The box pattern consists of four legs of descent, with wings held level (i.e., pitch and roll kept as close to 0^∘ as possible), connected by 90^∘ descending turns that are banked. The goal is to minimize the influence aircraft attitude has on the radiometer's position relative to the horizon, thereby reducing the magnitude of the attitude correction that is needed in post-processing of the radiometric data. In comparison to a series of traditional flat “radiation legs”, the “square-spiral” method allows for relatively rapid profiling of the atmosphere, which is useful in dynamic environments such as the ones encountered in CAMP²Ex. At CAMP²Ex near-surface clouds were encountered in most spirals, and so for this study we restricted spiral profiles to altitudes greater than 0.4 km. At ORACLES the stratus deck was mostly found below 1 km, and we limit sampling to above this altitude.

3.2 SPN-S irradiances and retrieval input data

The SPN-S is a modified version of the commercially available broadband SPN1 pyranometer. Instead of the detector head directing the sampled light to a set of seven thermopiles, this spectral version uses a seven-spectrometer array to measure irradiance from 350 to 1000 nm at 1 nm spectral resolution and 1 Hz temporal resolution. Detailed characterization of SPN1 and how the measurements of total and diffuse irradiance are made is described in Badosa et al. (2014). The version of the SPN-S used in this study is most similar to the Spectrometer 1 system described in Wood et al. (2017), with modifications done to the instrument chassis to allow it to be mounted on top of the P-3 fuselage. We follow the procedure laid out in Wood et al. (2017) for deriving the spectral direct irradiance from the measurements of total and diffuse irradiance.

Our procedure for calibration and attitude correction deviates from the methods described in Wood et al. (2017). The SPN-S was calibrated against a tungsten “FEL” lamp that is traceable to a National Institute of Standards and Technology (NIST) standard. The power of the FEL lamp is significantly lower than the power of the solar radiation at the sampling sites, and this led to issues with measuring irradiances at the shorter and longer wavelengths of the SPN-S's capabilities. For our analysis we only use sampled radiation from 460 to 900 nm due to these calibration constraints.

The direct-beam and total irradiances were corrected for the pitch and roll of the aircraft in accordance with the attitude correction method outlined in Long et al. (2010). Due to time limitations of the P-3 aircraft and cloud cover constraints, we could not fly the recommended “box” and “diagonal” patterns needed to determine pitch and roll offset angles of the mounted SPN-S. Rather, we manually went through all the flight data from the ORACLES campaign, identified heading changes in the aircraft that occurred under clear-sky conditions (DR_500 nm<0.1), and used the collective set of these heading changes as a substitute for the “box” and “diagonal” patterns in the Long et al. (2010) method. Since the SPN-S was mounted on the P-3 in the same configuration for the CAMP²Ex mission as it was during ORACLES, we assume the offset angles are constant across the two missions. To minimize the impact aircraft attitude has on the direct irradiance measurements we restrict our analysis to when both aircraft pitch and roll were within $\pm \mathrm{3}{}^{\circ }$.

To measure diffuse irradiance, the SPN-S uses a shadow mask to block the direct beam of the sun. The field of view (FOV) blocked by the sun is an area larger than the solar disk, and this wide FOV leads to understatement of the diffuse irradiance under sky conditions where there is a significant amount of light scattered in the direction of the direct beam (conversely, direct transmittance is overstated). This is similar to FOV issues encountered by sun photometer systems (Segal-Rosenheimer et al., 2013). Thin cirrus cloud layers are associated with strong forward scattering, and the induced bias in the measured irradiances will cause underestimation of τ_cld by both the RD and RS methods. We correct the τ_cld outputs of both retrievals for errors associated with the FOV of the SPN-S. This correction is done by developing an empirical relationship from radiance simulations that associates the magnitude of the irradiance bias induced by the FOV error to the true τ_cld; the details of the correction are explained in Appendix A.

RD requires measurements of a_fl, which is derived from flux measurements made by two sensors. The spectral total upwelling irradiance, F_up, is measured by a nadir-mounted solar spectral flux radiometer (SSFR; Pilewskie et al., 2003). The SSFR is a moderate-resolution total irradiance spectrometer system with a spectral range of 350–2100 nm. Like the SPN-S, the SSFR is radiometrically calibrated against a NIST-traceable lamp standard before and after each field campaign. Throughout the duration of each campaign, a series of field calibrations were used to monitor and correct for variations in the primary radiometric calibration. Since the upwelling irradiance is diffuse, the signal is only minorly impacted by the angular response of the SSFR light collector. A spectrally dependent factor that accounts for the angular response of the SSFR light collector to diffuse irradiance is determined through laboratory investigation. This factor has a value near unity, and it is used to correct the upwelling irradiance measurements to account for this angular dependence of the measured signal; a_fl is then determined by ratioing F_up to F_tot, the latter of which is measured by the SPN-S.

Altitude, pressure, sun position, and navigation data are measured at 1 Hz aboard the P-3 as described in Bennett (2020).

3.3 Radiative transfer model

The simulated DR values used in RD are made using the Discrete Ordinates Radiative Transfer Program (DISORT) 2.0 (Stamnes et al., 2000). We use the libRadtran library version 2.0.1 to configure and run DISORT 2.0 (Emde et al., 2016), with the base configuration set as follows.

• Molecular absorption is done using LOWTRAN 7 (Pierluissi and Pang, 1985).

• Pressure, temperature, and gas mixing ratio profiles – including ozone and water vapor – are set using the tropical atmosphere profile from Anderson et al. (1986).

• The solar source is defined using 1 nm resolution top-of-atmosphere flux from Kurucz (1994).

• A slit function with a 6 nm full width at half maximum is used on the output of spectral calculations.

• Solar zenith angle and model output elevation were set to values corresponding to P-3 position and time of day.

• Flight-level spectral albedo, a_fl, is determined from the SPN-S and SSFR irradiance measurements.

We simulated cirrus clouds by modifying the standard aerosol configuration in libRadtran, which generates scattering phase functions from Henyey–Greenstein. A cloud layer was inserted between 10–11 km, and then we adjusted the layer optical depth to the appropriate τ_cld. In this configuration, SSA was set to 1, and g=0.85 was used as a baseline value of the asymmetry parameter. All other simulated irradiances mentioned in the paper (e.g., Fig. 1) used a similar model configuration with deviations noted in the text.

3.4 Validation data

The Spectrometer for Sky-Scanning Sun-Tracking Atmospheric Research (4STAR; Dunagan et al., 2013) is an airborne sun photometer that makes direct-beam measurements of τ_aer above the aircraft using spectrometers (similar to the SSFR) spanning 350 to 1650 nm, with τ_aer reported at 24 wavelengths outside of gas absorption bands. Before and after each field deployment, 4STAR is calibrated at the Mauna Loa Observatory using Langley extrapolation methods. Additionally, in-flight high-altitude calibration measurements are used as a calibration verification and adjustment throughout deployment (LeBlanc et al., 2020). The resulting uncertainty in 4STAR-measured τ_aer is as low as 0.007 at the 501 nm channel.

4STAR operated only during the ORACLES campaign so we also compared the derived τ_aer values from the retrievals against in situ measurements of optical extinction (τ_in situ). For CAMP²Ex, total dry aerosol scattering was measured in situ by a TSI-3563 nephelometer at relative humidity (RH) less than 40 %. To account for aerosol humidification, a parallel TSI-3563 nephelometer was operated at an RH of 82 %±10 % and was used to derive the scattering hygroscopicity factor (i.e., f(RH); Ziemba et al., 2013). Scattering coefficients for each measured wavelength (i.e., 450, 550, and 700 nm) at ambient RH are then calculated using dry scattering coefficients, f(RH), and ambient RH by the diode laser hygrometer (DHL; Diskin et al., 2002). Optical absorption coefficients were measured by a Radiance Research three-wavelength particle soot absorption photometer (PSAP) at 430, 532, and 660 nm, with the sample dried by heating the air to 40 ^∘C. To develop altitude profiles of τ_in situ, ambient extinction coefficients are computed as the sum of dry absorption coefficient and ambient scattering coefficient after correcting both to 500 nm wavelength using measured scattering and absorption Ångström exponents and Eq. (6). The estimated uncertainties for the scattering and absorption coefficients are 30 % and 15 %, respectively. For the square spiral maneuvers performed by the P-3, this total ambient extinction was integrated with respect to altitude to generate a τ_in situ profile. Before integrating, 10 s averaging was applied to the time series to reduce the influence of noise artifacts on the profile. At ORACLES, TSI-3563 nephelometers and a Radiance Research PSAP were also flown, and measurements were processed in the same fashion as those from CAMP²Ex, with the caveat that a pair of Radiance Research M100 nephelometers were humidity-controlled and used to determine ambient scattering at 540 nm and f(RH). At ORACLES, the Radiance Research M100 nephelometer data were used to derive ambient extinction at times when the ambient TSI-3563 nephelometer was not operational. All in situ measurements on the P-3 aircraft for both CAMP²Ex and ORACLES were made behind an isokinetic shrouded solid-diffuser inlet (McNaughton et al., 2007) and are reported at ambient temperature and pressure.

3.5 Retrieval implementation

3.5.1 Diffuse method, RD

Implementation of RD is straightforward, and we followed the steps outlined in Sect. 2.1 of this paper. For the 500 nm SPN-S channel, DR_mea is calculated directly from the measured F_dif and F_tot. The estimate of the optical depth, τ_est, is made from DR_mea using Eq. (5); τ_est is input into the RTM along with a_fl to obtain a simulated value of DR_sim. If $|{\text{DR}}_{\text{mea}}/{\text{DR}}_{\text{sim}}-\mathrm{1}|\le \mathrm{0.01}$, the τ_est is the reported τ_cld value of the retrieval. If ${\text{DR}}_{\text{mea}}/{\text{DR}}_{\text{sim}}>\mathrm{1.01}$, τ_est is increased by a value of 0.01, and if ${\text{DR}}_{\text{mea}}/{\text{DR}}_{\text{sim}}<\mathrm{0.99}$, τ_est is decreased by a value of 0.01. The RTM is run again, and the new DR_sim is compared to DR_mea. The process iterates until the condition $\left|{\text{DR}}_{\text{mea}}/{\text{DR}}_{\text{sim}}-\mathrm{1}\right|\le \mathrm{0.01}$ is met.

We ran RD at three wavelengths to obtain τ_cld at 500, 670, and 870 nm. Since cirrus clouds are made of ice crystals, we assume the layer AE≈0 and therefore the τ_cld at the three wavelengths should be similar if only clouds are present. Significant spectral variation in τ_cld – >5 % deviation from the 500 nm retrieval to the 870 nm value – indicates that aerosols are present in the sampling layer. In such a case, careful interpretation of the retrieval output is advised.

3.5.2 Spectral direct-beam method, RS

The first step in implementing RS is to use Eq. (1) to derive the spectrally dependent total measured optical depth above the aircraft, τ_tot,λ. The μ term in Eq. (1) is calculated from the reported solar zenith angles logged in the P-3 flight records. F₀ in this calculation is determined using RTM simulations of F_dir at 14 km, which is an altitude above the cirrus cloud layer. This is done for all F_dir measurements contained within a square spiral where the 500 nm DR_mea<0.95, which generally corresponds to τ∼3 (see Fig. 2). This threshold on the diffuse ratio is used to ensure that there is measurable direct-beam signal, and above this level, measurement uncertainty in F_dir makes it untenable to use RS. Using the atmospheric pressure measurements made from the P-3, τ_Ray,λ is calculated by Eq. (9), and then these values are subtracted from τ_tot,λ:

$$\begin{array}{}\text{(10)}& {\mathit{\tau }}_{\mathit{\lambda }}={\mathit{\tau }}_{\text{tot},\mathit{\lambda }}-{\mathit{\tau }}_{\text{Ray},\mathit{\lambda }}.\end{array}$$

We do this for a limited set of the SPN-S wavelengths, λ_retrieval={(460–540), (665–684), (746–755), (772–785), (860–879)} nm. The wavelengths used in the retrieval were determined by studying simulated spectra of optical depths, such as those in Fig. 1, and then selecting spectral regions where there is minimal influence of molecular absorption. The presence of some extinction by trace gases is acceptable at the selected wavelengths because we correct the τ_λ spectra for these influences using aerosol-free samples from the top of the spiral (see the following paragraphs for details of this step). This correction does not account for changes in column trace gas concentrations as the aerosol layer is profiled, which is a limitation of this method. Our analysis was bounded between 460 and 880 nm due to the SPN-S calibration issue described in Sect. 3.2.

Figure 4 Two examples of the spectral optical depth used in RS to determine τ_cld and τ_aer. Both examples are from the spiral on 5 October 2018 of the ORACLES mission. (1) A clear-sky case, colored in blue, from a sample taken at an altitude 4.6 km. (2) In red, a sample from 1 km, which is below the aerosol layer. The dark red and blue dots are τ_adj,λ, and they correspond to the dashed red and blue lines in Fig. 1. The faded red and blue dots are ${\mathit{\tau }}_{\mathit{\lambda }}={\mathit{\tau }}_{\text{tot},\mathit{\lambda }}-{\mathit{\tau }}_{\text{Ray},\mathit{\lambda }}$ values from all SPN-S channels from 460 to 880 nm, including those not used in the retrieval. The blue and red lines represent the best-fit lines for the RS outputs. The green line is τ_4STAR as measured at 1 km altitude.

The determination of F₀, the presence of trace gases, and calibration inconsistencies with the prototype SPN-S system led to complexities in implementing RS. Flight-to-flight changes in atmospheric conditions and the SPN-S calibration caused the derived τ_λ profiles to have a spurious spectral shape, deviating from flat under clear-sky or cloud-only conditions (the faint blue dots in Fig. 4 are a good example of errors in τ_λ). These errors had a constant magnitude within a flight and were not proportional to signal magnitude. Issues related to the dark-current correction may have been responsible for some of the observed behavior, though we are unsure. Additionally, the use of the RTM to derive F₀, which is traceable to the solar spectrum defined by Kurucz (1994), is a source of error in optical depth calculations. Nonetheless, we were able to account for these errors by using τ_λ samples from high-altitude flights to adjust the measured optical depth profiles. To do this, spectral profiles of τ_λ from aerosol-free regions of the tops of the P-3 spirals were collected. The spiral tops are assumed to be aerosol-free, and this assumption is checked by manually observing if the 4STAR aerosol optical depth (τ_4STAR) is <0.05 or for the CAMP²Ex case if the integrated in situ extinction (τ_in situ) has a significant gradient in this region of the spiral profile. Spirals with tops that are within the aerosol layer were excluded from the study. The spectral optical depth correction, τ_crr,λ, is the mean optical depth that each channel is from a reference channel at 500 nm for the selected spiral top τ_λ spectra:

$$\begin{array}{}\text{(11)}& {\mathit{\tau }}_{\text{crr},\mathit{\lambda }}=\stackrel{\mathrm{‾}}{{\mathit{\tau }}_{\mathit{\lambda }}-{\mathit{\tau }}_{\text{500\hspace{0.17em}nm}}}.\end{array}$$

The optical depth correction is applied to all τ_λ by subtraction:

$$\begin{array}{}\text{(12)}& {\mathit{\tau }}_{\text{adj},\mathit{\lambda }}={\mathit{\tau }}_{\mathit{\lambda }}-{\mathit{\tau }}_{\text{crr},\mathit{\lambda }}.\end{array}$$

Applying this correction is beneficial since Eq. (7) assumes τ_λ has a smooth exponential shape, and correcting τ_λ to a line allows for better representation of the data by the model. Due to this need to correct τ_λ, we restrict use of RS to square spiral samples and do not use it for time series analysis of full science flights. In a sense, we are deriving layer optical depth with RS, and the spiral tops are used to characterize the flux at the top of the layer, and deviations at lower altitude are accounted for by the τ_aer term. RD in comparison is temporally and spatially independent from itself and can be used at any point along the flight track when DR is sufficiently small; more details about the utility of both methods are discussed in Sect. 5. Further, scattering by low- and mid-altitude clouds can interfere with interpretation of F_dir, and therefore spirals with significant numbers of these clouds were excluded from this study (e.g., if the P-3 entered a cloud during the spiral, the spiral was not analyzed).

Each corrected optical depth profile, τ_adj,λ, is compared to a series of curves generated using Eq. (7) for defined sets of τ_cld, τ_aer, and AE; τ_cld is varied from 0 to 5 at 0.01 resolution, τ_aer is varied from 0 to 1.5 with 0.01 resolution, and AE ranged from 1 to 2.0 with 0.1 resolution. The RMSE between each curve and the τ_adj,λ sample is computed. The fit parameters – τ_cld, τ_aer, AE – corresponding to the curve with the lowest RMSE value are designated the outputs of the retrieval. Figure 4 shows two examples of RS for a clear-sky case (blue line) and a case within an aerosol layer (red line). The solid filled dots represent τ_adj,λ, and the impact of τ_crr,λ is noticeable when comparing to the faded dots, which are the uncorrected optical depth profiles, τ_λ. The solid lines represent Eq. (7) with the fit parameters set to the retrieval output in these two cases.

4 Results

We first present the results of the ORACLES campaign followed by the CAMP²Ex results. For each campaign we first show the retrieval performance for a one-square-spiral case study, and then we give the aggregate statistics for the campaign.

4.1 ORACLES 5 October 2018 profile

Between 09:52:00 and 10:15:00 UTC, the P-3 flew a square spiral centered around a latitude and longitude of $-\mathrm{9}{}^{\circ }{\mathrm{33}}^{\prime }{\mathrm{51.50}}^{\prime \prime }$, $\mathrm{5}{}^{\circ }{\mathrm{57}}^{\prime }{\mathrm{15.20}}^{\prime \prime }$, with a spiral midpoint SZA of 22.8^∘. The profile started at an altitude of 5.74 km and ended near the surface at 0.33 km. No clouds were present above the spiral start height, and this absence of cirrus was typical of the spirals conducted during the 2018 deployment of the ORACLES campaign. Despite this lack of cloud, we applied RD in addition to RS to compare the outputs of the two methods. The left panel of Fig. 5 shows both the measured 500 nm diffuse ratio, DR${}_{\text{mea}}^{\text{500\hspace{0.17em}nm}}$, and 500 nm direct transmittance, T^500 nm, with their associated uncertainty. Starting just below the 4 km level, the magnitude of the slopes of both DR${}_{\text{mea}}^{\text{500\hspace{0.17em}nm}}$ and T^500 nm steepen, signaling the presence of an aerosol layer. The profiles remain predictable in shape all the way to the surface, with monotonic change and no erratic points to indicate clouds along the profile path.

Figure 5 ORACLES spiral on 5 October 2018. (a) DR${}_{\text{mea}}^{\text{500\hspace{0.17em}nm}}$ and T^500 nm with the associated measurement uncertainty shown by the shading; (b) τ_aer profiles derived from RS, 4STAR, in situ, and RD.

The right panel of Fig. 5 shows the altitude profiles of 500 nm τ_aer from RS (τ_RS,aer, green), 4STAR (τ_4STAR, red), in situ (τ_in situ, cyan), and RD (τ_RD, blue). Expectedly, the inability of RD to account for absorption within the aerosol layer leads to lower values of τ_aer than the other methods below 3 km. Above the aerosol layer, RD detects a non-zero optical depth of ∼0.05, which is mainly a result of the relatively high retrieval uncertainty in the RS method when observing small optical depths (see Figs. 2 and 3). This is similar to the measurements made by 4STAR, which commonly detects non-zero optical depths at the top of the spiral. However, in the case of 4STAR, these optical depths are mostly due to the presence of stratospheric aerosols (Kremser et al., 2016), though there may also be contribution from small amounts of tropospheric aerosol as well. In the case of 4STAR, the τ_aer measured at the top of the spiral is accounted for by subtracting off a constant value that is equal to the mean τ_4STAR value above 5.5 km. Visual comparison between τ_RS,aer and τ_4STAR profiles shows agreement with τ_4STAR falling within the reported uncertainty in τ_RS,aer for all samples outside of a few τ_4STAR outlier cases. With respect to the τ_in situ profile, both the τ_RS,aer and τ_4STAR agree well with the in-situ-measured extinction.

4.2 2018 ORACLES mission statistics

There were 14 square spirals flown by the P-3 during the 2018 deployment of the ORACLES mission. The cumulated τ_aer data from this set of spirals are used in regression analysis, with 4STAR serving as the reference measurement. The left panel of Fig. 6 shows scatterplots of τ_4STAR vs. τ_RS,aer and τ_in situ. Regression for τ_RS,aer gives R²=0.96, slope =0.96, and intercept $=\mathrm{1.4}\times {\mathrm{10}}^{-\mathrm{3}}$, and RMSE is $\mathrm{3.0}\times {\mathrm{10}}^{-\mathrm{2}}$. The high R² value and low RMSE indicate that the SPN-S-based aerosol optical depth retrieval performs well in comparison to 4STAR. Regression for τ_in situ vs. τ_4STAR yields R²=0.96, slope =0.90, and intercept $=-\mathrm{2.1}\times {\mathrm{10}}^{-\mathrm{4}}$, and RMSE is $\mathrm{3.3}\times {\mathrm{10}}^{-\mathrm{2}}$. The good agreement between τ_4STAR and τ_in situ gives us confidence that integrating the in situ measurements of extinction can serve as a basis for comparison for the CAMP²Ex cases where 4STAR (or an equivalent system) was not operated. The right panel shows percent errors for τ_RS,aer and τ_in situ for binned τ_4STAR values. At small optical depths, uncertainty is a significant fraction of the total optical depth signal, and this leads to large relative errors. As τ_4STAR increases, the fractional error between τ_4STAR and τ_RS,aer decreases.

Figure 6 Aggregated τ_aer statistics using τ_4STAR as a baseline for the ORACLES campaign. (a) Scatterplots of τ_4STAR vs. τ_RS,aer and τ_in situ. Regression or τ_RS,aer gives R²=0.96, slope =0.96, and intercept $=\mathrm{1.4}\times {\mathrm{10}}^{-\mathrm{3}}$, and RMSE is $\mathrm{3.0}\times {\mathrm{10}}^{-\mathrm{2}}$. Regression for τ_in situ yields R²=0.96, slope =0.90, and intercept $=-\mathrm{2.1}\times {\mathrm{10}}^{-\mathrm{4}}$, and RMSE is $\mathrm{3.3}\times {\mathrm{10}}^{-\mathrm{2}}$. (b) Percent error in τ_RS,aer and τ_in situ relative to τ_4STAR; τ_4STAR values are sorted into bins with 0.025 optical depth width, and the corresponding statistics for τ_RS,aer and τ_in situ are reported. Dots represent the median value; tips of the line are the 25th and 75th quantiles of the τ_RS,aer and τ_in situ distributions within each bin.

Figure 7 compares the total optical depths derived using RS and RD, where ${\mathit{\tau }}_{\text{RS,tot}}={\mathit{\tau }}_{\text{RS,aer}}+{\mathit{\tau }}_{\text{RS,cld}}$. For these ORACLES cases conducted under cloud-free skies, the retrieval of τ_RD is caused by aerosols. The regression slope (slope =0.84) deviates significantly from unity, which is expected because RD accounts for extinction only due to scattering. From our analysis in Sect. 2.3 we anticipate the regression to give a slope of less than 1, with the value being linked to g and SSA of the aerosol layer (the green line in Fig. 7 is the modeled relationship for g=0.85 and SSA=1). The aerosols sampled at ORACLES were absorbing, with mid-visible SSA values near 0.85 (Cochrane et al., 2019), and this sink of radiation by the aerosol layer and the errors in the scattering phase function used in RD are the main causes for τ being underestimated by the RD method. A final point worth noting is that there are numerous instances when τ_RD has a non-zero value, yet τ_RS,tot does not register an optical depth. This is not surprising given the higher sensitivity to small changes in τ of the RD output.

Figure 7 Aggregated optical depth data from ORACLES campaign τ_RS,tot vs. τ_RD. Regression gives R²=0.95, slope =0.84, and intercept $=\mathrm{7.9}\times {\mathrm{10}}^{-\mathrm{3}}$. The green line is the predicted τ_RD for a given τ_RS,tot value, which is determined by RTM simulations (i.e., the dashed red line in the center panel of Fig. 3), for g=0.85.

4.3 CAMP²Ex 17 September 2019 profile

From 00:57:00 to 01:27:00 UTC on 17 September 2019 the P-3 flew a square spiral centered at a latitude and longitude of $\mathrm{13}{}^{\circ }{\mathrm{55}}^{\prime }{\mathrm{52.32}}^{\prime \prime }$, $\mathrm{125}{}^{\circ }{\mathrm{27}}^{\prime }{\mathrm{41.76}}^{\prime \prime }$, with a spiral midpoint SZA of 36.1^∘. The spiral profiled from an altitude of 5.59 km to near the surface at 0.40 km. In contrast to the ORACLES case study, there were cirrus clouds present above the spiral location, and their radiative signature can be seen in the deviation in both the DR${}_{\text{mea}}^{\text{500\hspace{0.17em}nm}}$ and T^500 nm profiles from a monotonic curve above 3 km (left panel of Fig. 8). Additionally, the dip in T^500 nm value (seen as a spike in DR${}_{\text{mea}}^{\text{500\hspace{0.17em}nm}}$) around 2 km indicates the influence of a cloud in the vicinity of the P-3. The variation in DR${}_{\text{mea}}^{\text{500\hspace{0.17em}nm}}$ and T^500 nm due to clouds causes difficulty when visually attempting to discern the start of the aerosol layer on the graph, but near 2.5 km the magnitudes of the T^500 nm and DR${}_{\text{mea}}^{\text{500\hspace{0.17em}nm}}$ slopes increase, indicating the presence of aerosols.

Figure 8 CAMP²Ex square spiral on 17 September 2019. (a) DR${}_{\text{mea}}^{\text{500\hspace{0.17em}nm}}$ and T^500 nm as a function of P-3 altitude with the associated measurement uncertainty shown by the shading; (b) τ_aer profiles derived from RS and in situ; (c) τ_cld values for RD at 500, 670, and 870 nm and RS as a function of altitude.

The middle panel of Fig. 8 shows the altitude profiles of τ_RS,aer and τ_in situ. There is good agreement between the shape of the two curves; however τ_RS,aer is consistently higher than τ_in situ for altitudes below 2.5 km. We are not sure of the specific reasons why τ_RS,aer has higher values than τ_in situ in this case. This bias was not found regularly in the other CAMP²Ex cases, which indicates that instrument error, associated with either the SPN-S or the in situ measurements, may be responsible for the observed differences in τ_aer.

The right panel of Fig. 8 shows cloud optical depth derived from the RS and RD methods, τ_RS,cld and τ_RD, respectively, as the P-3 profiled the atmosphere; τ_RD is shown at 500, 670, and 870 nm. Above the aerosol layer (>2.5 km) there is good consistency between the three wavelengths of τ_RD. The 870 nm channel has slightly greater variance than the other two, and this relates to increased noise levels and stray-light issues within the spectrometer that we observed for SPN-S channels past 850 nm. Above the aerosol layer τ_RD has less variance in optical depth values than τ_RS,cld, though the majority of samples of τ_RS,cld and τ_RD fall within their uncertainties. An imperfect attitude correction of the SPN-S appears to be the main driver of this discrepancy between the retrieved τ_RS,cld and τ_RD values. Since fluctuations in sensor attitude more severely impact the direct irradiance than the diffuse ratio, changes in the P-3 heading during the square spiral will influence τ_RS,cld more than τ_RD. One benefit of having two methods for retrieving τ_cld is that sources of error that are a result of the experimental setup can be identified. We do not account for the uncertainty attributed to changes in sensor attitude on RS because there are engineering approaches, such as stabilizing platforms, and study methods that can be implemented which can reduce the impact aircraft attitude has on the measurement in future deployments of the SPN-S.

Within the aerosol layer (<2.5 km), spectrally dependent absorption of the aerosol causes the three wavelengths of τ_RD to diverge, with the longer wavelengths less influenced by the aerosols. While RD is limited when sampling in the aerosol layer because SSA is not known, the wavelength dependence of τ_RD can be used to determine if aerosols are present. For τ_RS,cld, the retrieved values from within the aerosol layer fall within the range of cloud optical depth observed above the aerosol layer.

4.4 2019 CAMP2Ex mission statistics

There were 18 spirals flown by the P-3 during the CAMP²Ex campaign. Figure 9 shows the relationship of τ_RS,aer vs. τ_in situ, and the corresponding regression gives R²=0.97, slope =0.94, and intercept $=\mathrm{2.4}\times {\mathrm{10}}^{-\mathrm{4}}$, while RMSE is $\mathrm{3.4}\times {\mathrm{10}}^{-\mathrm{2}}$. These aggregated results are consistent with the comparison between τ_4STAR and τ_in situ done for the ORACLES campaign, where τ_in situ also had a slight low bias in relation to the 4STAR-derived optical depths (slope =0.90). The relatively clean air with lower aerosol extinction and more varied source regions sampled during the CAMP²Ex campaign (Hilario et al., 2021) restrict our ability to fully validate τ_RS,aer under the cirrus conditions because the amount of sampling done at high τ_aer was limited. Additional future work to further examine the retrieval of τ_aer under a broader range of optical depths when cirrus clouds are present is needed. This is critical in light of the found limitations of the SDA method under cirrus conditions (Smirnov et al., 2018). However, there is reason to expect the RS method will be able to retrieve τ_aer under cirrus conditions with lower uncertainty than has been possible using past methods. For example, the SDA method is developed for use with ground-based measurements, and therefore it derives a column optical depth. When retrieving column τ_aer it is difficult to account for the forward scattering of light by cirrus, which makes it difficult to accurately constrain τ_aer. In the RS method, because the irradiance at the top of the layer is directly characterized, the forward scattering by cirrus is accounted for when deriving τ_aer. Regardless, the limited retrieved τ_RS,aer values at CAMP²Ex are consistent with the relationship between τ_in situ and τ_4STAR observed at ORACLES.

Figure 9 Comparison of τ_RS,aer vs. τ_in situ from the CAMP²Ex campaign.

Figure 10 CAMP²Ex. (a) Above-aerosol-layer comparison: τ_RS,cld vs. τ_RD. (b) All data total-optical-depth comparison: τ_RS,tot vs. τ_RD. The dashed green line represents the expected τ_RD given a value of τ_RS,tot assuming g=0.85.

Like with the ORACLES results, we compare τ_RS,tot to τ_RD for two cases: (1) above the aerosol layer, which is shown in the left panel of Fig. 10, and (2) the optical depths for complete spiral profiles (i.e., data from above and within the aerosol layer), which are shown in the right panel of Fig. 10. At higher optical depths, the relationship between τ_RS,cld and τ_RD (left panel) resembles the predicted one (see Sect. 2.3), with the sampled points clustering mostly along the 1 : 1 line. There is a grouping of points near τ_RS,cld=1 that have lower values of τ_RD, which are depressing the regression slope from unity (slope =0.94). A possible explanation for these points is clouds in the vicinity of the P-3 scattering light into the diffuse sensor of the SPN-S, biassing F_dif high, the result of which is low τ_RD,cld. Comparing the relationship between the total set of samples (τ_RS,tot vs. τ_RS; right panel), the points generally fall below the 1 : 1 line (slope =0.89). Here, the addition of absorbing aerosols is the likely cause of the low τ_RD values in relation to τ_RS,tot. However, errors in the scattering phase function used in the RTM and the influence of mid-spiral clouds may also be partially responsible for the lower values of τ_RD and the resulting low regression slope.

4.5 RD cirrus characterization along flight track

The inputs of RD are absolute, and therefore the method can be deployed to derive τ_cld along P-3 flight tracks. We demonstrate this with an example from the science flight on 6 September 2019 from 03:00 to 05:00 UTC. To reduce the computational time required to run the retrieval, the 1 Hz SPN-S and SSFR irradiance data are subsampled at 0.1 Hz before being processed. For this sampling period SZA ranged from 10.0 to 17.6^∘. The top panel of Fig. 11 shows the P-3 altitude and DR${}_{\text{mea}}^{\text{500\hspace{0.17em}nm}}$, while τ_cld at 500 and 670 nm is shown in the bottom panel, along with the ratio of the two optical depths. The presented data are not filtered for any criteria on DR. When DR is near unity the retrieval performs poorly, resulting in spikes in the retrieved value of τ_cld; two examples of these events are seen at 03:30 and 04:30. When DR<0.9, such as the start of the flight track from 03:00 to 03:28 UTC or the section near the end, 04:45 to 04:56 UTC, there is consistency between ${\mathit{\tau }}_{\text{cld}}^{\text{500\hspace{0.17em}nm}}$ and ${\mathit{\tau }}_{\text{cld}}^{\text{670\hspace{0.17em}nm}}$, indicating successful retrieval of τ_cld. At lower altitudes, from 03:38 to 04:15 UTC, the cloudier environment frequently causes the irradiance to become completely diffuse, and the retrieval fails. During the low flight leg, 04:00 to 04:10 UTC, the ratio of ${\mathit{\tau }}_{\text{cld}}^{\text{500\hspace{0.17em}nm}}/{\mathit{\tau }}_{\text{cld}}^{\text{670\hspace{0.17em}nm}}$ is high, indicating the possible presence of aerosols. A complete analysis of CAMP²Ex cirrus cloud optical and radiative properties is provided in Hong et al. (2022).

Figure 11 Time series of 6 September 2019, partial flight track of (a) P-3 Altitude and DR${}_{\text{mea}}^{\text{500\hspace{0.17em}nm}}$, (b) ${\mathit{\tau }}_{\text{RD,cld}}^{\text{500\hspace{0.17em}nm}}$ and ${\mathit{\tau }}_{\text{RD,cld}}^{\text{500\hspace{0.17em}nm}}$. The ratio, ${\mathit{\tau }}_{\text{RD,cld}}^{\text{500\hspace{0.17em}nm}}/{\mathit{\tau }}_{\text{RD,cld}}^{\text{670\hspace{0.17em}nm}}$ is also shown.

5 Discussion

The application of the SPN-S radiometer to deriving overlying cloud and aerosol optical depth is promising, but there are tradeoffs that must be considered when comparing these methods to existing standards. If the objective is to identify and classify the optical properties of thin clouds (e.g., τ_cld<1), RD is a robust choice because the small uncertainty in DR allows for a highly sensitive retrieval. At optical depths greater than 1, the assumptions underpinning RD, especially knowledge of the scattering phase function, lose validity, causing significant errors in the retrieval output (see Fig. 3). Since the relationship between the retrieved τ_cld and its error can be largely explained for a given value of g, it may be possible to correct for the discrepancy between predicted and true cloud optical depth under conditions where g is constrained or there is accurate knowledge of the phase function. However, the capabilities of RD will always be limited to thin clouds because beyond an optical depth of about 5 the DR signal loses much of its sensitivity to changes in τ_cld. Of course, this limit at which RD can accurately derive τ_cld will be dictated by the performance of the sensor measuring DR. Another limitation of RD is that, without knowledge of the layer SSA, the effects of absorption on DR are not quantifiable, thus making the retrieval of limited use when aerosols are present. The caveat to this is the development of a simple aerosol flag, where spectral dependencies in RD output can be attributed to an absorber, such as aerosols, being present in the sampled layer.

The benefit of RS is that the retrieval has the potential to separate the cloud and aerosol radiative signals from each other. The tradeoff with the existing sun photometry standards of measuring τ_aer is greater uncertainty that stems from the difficulty in accurately measuring F_dir with a radiometer. Errors are induced into F_dir measurements by several mechanisms: changes in sensor attitude, calibration shifts over time, cosine response errors, temperature effects, etc. We attempt to reduce the uncertainty associated with these errors by sampling profiles of the atmosphere and using the high-altitude aerosol-free samples to correct for calibration errors and variations in atmospheric composition, but the need for this correction currently limits the application of the method to profiles of atmospheric layers. While it is unlikely that the SPN-S system will ever be able to obtain the precision and accuracy of a system like 4STAR, there remains obvious room for improvement. Better characterization of the offset angles of the mounted SPN-S will reduce the error related to changing aircraft attitude. (During ORACLES and CAMP²Ex, the SPN-S was not a priority instrument, and flying the necessary flight patterns to determine the offset angles via the Long et al., 2010, method was not possible.) More sophisticated radiometric calibrations will also improve retrieval performance, be it through laboratory comparisons with traceable lamp standards or using Langley techniques and intercomparison with known benchmarks. The seven-detector-head design of the SPN-S introduces technical challenges when calibrating using a lamp, and this being a prototype instrument, some of the challenges are still being worked through.

Another source of uncertainty in optical depth retrievals relying on direct and diffuse measurements made from the SPN-S is the wide FOV associated with shadow band radiometer systems, which results in an overestimation of the direct transmittance. Biased measurements of transmittances due to the sensor FOV being wider than the solar disk are an issue associated with sun photometers as well; however the FOV of the SPN-S system is greater than commonly used sun photometers (∼5–10^∘ vs. $\sim \mathrm{1}{}^{\circ }$), making FOV correction necessary for accurate optical depth retrievals (di Sarra et al., 2015; Wood et al., 2017). Empirical methods can be used to account for the discrepancy in sensor FOV, but these methods are reliant on large numbers of co-located measurements of τ_aer with a sun photometer under varying aerosol or cloud conditions. These empirical relationships are specific to the individual sensors themselves, meaning that we cannot apply previously generated FOV corrections here. In the specific cases used in this study, comparison with the SPN-S and 4STAR did not indicate substantial FOV bias that warranted correcting τ_aer, so we did not apply those techniques. However, forward scattering in the direction of the direct beam is most severe under thin ice clouds, and so we used RTM simulations of the diffuse radiance to account for these effects when retrieving τ_cld using both RS and RD (details contained in Appendix A).

RS is prone to errors when differentiating a signal between τ_cld and τ_aer if certain aerosol types are present. Coarse-mode aerosols, such as dust particles, can have AE values near zero (Eck et al., 1999) and therefore have minimal spectral dependence on their optical depth. That is, larger aerosol particles have extinction wavelength profiles similar to cirrus clouds. In cases where large aerosol particles are present, the τ_cld term in Eq. (7) is also dependent on the coarse-mode aerosol optical depth. The aerosol type and clouds present when sampling will determine the extent to which large aerosols impact the retrieval output. For example, if the study region is cloud-free, optical depths due to coarse-mode aerosols will still be able to be measured using RS by evaluating the τ_cld term of the retrieval output. In fact, RS could be extended to do aerosol-mode analysis under cloud-free skies in a manner similar to the SDA method used in sun photometry (O'Neill et al., 2003).

Advances in the capability of the SPN-S are ongoing, and improvements to the system have already been made since the prototype version was deployed to ORACLES and CAMP²Ex. Most notably, software improvements now allow for sequential sampling to occur at multiple spectrometer integration times. The advantage of this technique is that both bright and dark parts of the irradiance spectrum can be resolved nearly simultaneously, giving much better instrument performance at the tails of its spectral range (<450 and >900 nm). Having greater spectral range in which to evaluate Eq. (7) will allow differentiating between cloud and aerosol optical depths to be done with greater confidence. Likewise, with the improved calibration mentioned earlier in this section it may be possible to eliminate the need for spiral patterns and the associated optical depth correction. Instrument upgrades, along with measuring and accounting for extinction from trace gases (i.e., better representation of τ_mol,λ), open the door for applying RS to a broader set of sampling types, such as along full flight tracks or to ground-based deployments. Making the mentioned improvements to the SPN-S and using it in ground-based settings would produce useful data to use in the characterization of the retrieval uncertainty. All of these retrieval methods need more future work to better validate and understand their outputs, some of which is currently ongoing. Hong et al. (2022) will compare retrieved τ_cld to similar results from spaceborne remote sensors.

There are several more advantages of SPN-S system-based methods for cirrus cloud and aerosol studies to note. The SPN-S is a total-diffuse radiometer that has no moving parts, which makes it an inexpensive and user-friendly instrument to operate in a wide variety of settings. Airborne sun photometer or lidar systems tend to be mechanically and technologically complex, resulting in significant overhead when operating them in field settings. Moreover, while the spectral analysis techniques of RS can be applied to any set of spectral direct irradiance measurements (for example, RS can be applied to spectral sun photometer irradiances), a well-characterized SPN-S system has inherent advantages. The SPN-S measurement of hyperspectral total irradiance provides a more complete view of the radiative environment rather than informing about only the optical depth and direct irradiance that sun-staring sun photometry measurements provide. An example of how this might be beneficial is for the determination of heating rates in the atmosphere. With a measured spectral τ_aer profile, radiometric-based approaches have been advanced that allow aerosol-intensive properties to be derived (Cochrane et al., 2019), from which it is possible to determine aerosol heating rates and radiative effects (Cochrane et al., 2022). SPN-S measurements may allow for similar studies to be completed with a consolidated set of instruments at lower cost. Additionally, aerosol-intensive properties can be studied using the DR measurement in manners similar to Kassianov et al. (2007).

6 Conclusions

In this paper we use the capabilities of the newly developed SPN-S radiometer to implement two retrieval methods: RD is a scheme that utilizes single-channel measurements of DR and a_fl to derive cirrus cloud optical depth, while RS is a technique that exploits structure in optical depth spectrum to partition it into τ_cld and τ_aer components. Since the primary radiometric input of RD is the ratio of two measured irradiances, calibration-induced uncertainties in the system are minimized, resulting in a retrieval that is highly sensitive to small optical depths. Unquantified absorption in the atmosphere limits the utility of RD to derive τ_cld, with the caveat that the method can be used to identify the presence of aerosols by comparison of the retrieved optical depths at multiple wavelengths. On the other hand, RS is based on measurements of F_dir which have larger associated uncertainties stemming from calibration errors and the influence of changing sensor attitude. This makes RS best suited for deriving τ_cld at values greater than unity where the inherent retrieval errors are a lower fraction of the output or when aerosols are present. Since τ_aer is derived from the spectral shape of the optical depth and not the absolute value at any one wavelength, the τ_aer output of the retrieval is less influenced by the measurement uncertainty than the spectrally independent τ_cld output.

We apply both methods to data from two field campaigns, ORACLES in 2018 and CAMP²Ex in 2019, to evaluate their performance. RS performed well at retrieving τ_aer by comparison to measurements made by the 4STAR sun photometer system ($\text{RMSE}=\mathrm{3.0}\times {\mathrm{10}}^{-\mathrm{2}}$) and optical depth as retrieved by in situ measurements under both clear-sky and cirrus conditions ($\text{RMSE}=\mathrm{3.4}\times {\mathrm{10}}^{-\mathrm{2}}$). There were limited cases of high aerosol loading under cirrus conditions, so the retrieval performance under these circumstances warrants further investigation. The τ_cld retrievals of both methods were evaluated against each other and behaved as our theoretical analysis predicted.

The optical depth retrieval uncertainties in these two new methods suggest that the SPN-S is not a replacement for traditional sun photometer instruments such as 4STAR. However, it is a low-cost alternative that is mechanically simple, making it logistically easier to deploy in many circumstances, such as on aircraft. Depending on the experiment, the tradeoffs in optical depth uncertainty in the SPN-S can be afforded by the accessibility it provides to reliably identifying the presence of above-aircraft clouds and aerosols. Cirrus identification is of value to other passive nadir-viewing sensors such as imaging and scanning radiometers and polarimeters. The SPN-S is also advantageous in that it measures spectrally resolved total, direct, and diffuse irradiance, which are useful quantities in the context of radiation science.

The SPN-S used in this study is a prototype, and work remains to better characterize the performance and calibration characteristics of the instrument. Much of this work is ongoing already, with software advancements having expanded the spectral range of the system, while improvements to the calibration procedures are a focus of current work. Increases in system predictability and reduced measurement uncertainty will allow for more versatility in the deployment of the methods presented in this study. For example, it may be possible to use RS for time series analysis if the calibration stability gets to the point where a profiling approach is no longer needed to correct for errors in the optical depth spectra. The combination of utility, robustness, and ease of implementation offered by the SPN-S makes it feasible to implement in a wide variety of settings: from future airborne campaigns to long-term monitoring applications at ground-based field sites.

Appendix A: FOV correction development

Forward scattering of radiation in the direction of the direct beam is a common phenomenon sun photometry techniques must account for when deriving optical depth of aerosols and thin clouds (Segal-Rosenheimer et al., 2013). Frequently the full-angle FOV (here FOV refers to the full-angle, not half-angle, FOV of the sensor) of a sensor used to measure F_dir is greater than the angular width of the sun's disk, causing diffuse light surrounding the direct beam to influence measurements. This overestimation of the direct transmittance leads to τ being underestimated when implementing standard radiative transfer techniques. While the SPN-S does not function in the same manner as a sun photometer to measure F_dir, the shadow mask used to block the direct beam when sampling F_dif has a shading area wider than the beam. That is, the shading area of the shadow mask is too large, and this leads to a low bias in measured F_dif and a high bias in derived F_dir. Under most atmospheric conditions this bias is minimal and can be ignored. However, it has been shown that when thin clouds are present, especially ice phase clouds which scatter strongly in the direction of the direct beam, the biases can lead to errors in τ retrievals up to 100 % (Segal-Rosenheimer et al., 2013).

To account for the contamination in measured F_dir by diffuse radiation, we developed corrections for the τ_cld outputs of both methods presented in this paper, RS and RD, based on simulated radiance fields in the FOV of the SPN-S. These corrections are in the form of a relationship between the optical depth inferred by the sensor to the true optical depth, and they are dependent on solar zenith angle and wavelength.

To do this, we used the MYSTIC Monte Carlo model that is a part of the libRadtran package (Emde et al., 2016) to simulate the diffuse radiance, L_dif, in the FOV of the SPN-S. The SPN-S is a prototype, and its exact FOV is unknown, but it is estimated to be around 8^∘ (Wood et al., 2017). Further, errors in FOV at angles greater than 5^∘ will have minimal impact on the optical depth correction because the majority of L_dif is located within the first 2 angular degrees from the center of the solar disk. We run a set of radiance simulations, with the sensor pointed towards the sun's position, and calculate L_dif across the arc length of the FOV, scanning across the sensor FOV area in the azimuth and zenith directions. These simulations are done for a set of ice clouds with τ_cld ranging from 0–6 at 0.1 resolution over a black surface. We limited the Monte Carlo model runs to 1000 photons each, which is a low number, but the FOV corrections are based on fits to sets of model runs, and so the error in any one simulation due to the low photon count is minimal. Figure A1 shows an example of simulated diffuse transmitted radiance, defined as ${\text{Tl}}_{\text{dif}}=L_{\text{dif}}/F_{\mathrm{0}}$ as a function of viewing angle in the azimuth. Forward scattering from thin cirrus peaks around τ_cld=1, leading to the observable spike in Tl_dif.

Figure A1 Simulations of diffuse radiance transmittance at 500 nm, ${\text{Tl}}_{\text{dif}}=L_{\text{dif}}/F_{\mathrm{0}}$ when scanning across the sensor FOV in the azimuthal direction. Line color corresponds to τ_cld value. The sun is positioned at a viewing angle of 0^∘, and the SZA is 20^∘.

We assume that the radiance field is symmetrical about both the zenith and azimuth axis and integrate the radiance fields to derive the total diffuse transmittance in the FOV of the sensor, T_dif,FOV:

$$\begin{array}{}\text{(A1)}& T_{\text{dif,FOV}}=\underset{\mathrm{0}}{\overset{\text{fov}}{\iint }}{\text{Tr}}_{\text{dif}}\mathrm{d}\mathit{\theta }\mathrm{d}\mathit{\varphi }.\end{array}$$

In practice, we split the integral along the four FOV paths we simulated using MYSTIC:

$$\begin{array}{}\text{(A2)}& \begin{array}{rl}& T_{\text{dif,FOV}}=\\ & {\displaystyle \frac{\mathit{\pi }}{\mathrm{2}}}\left[\underset{{\mathrm{0}}^{\circ }}{\overset{\mathrm{4}{}^{\circ }}{\int }}{\text{Tl}}_{\text{dif}}\mathrm{d}\mathit{\varphi }+\underset{-{\mathrm{4}}^{\circ }}{\overset{\mathrm{0}{}^{\circ }}{\int }}{\text{Tl}}_{\text{dif}}\mathrm{d}\mathit{\varphi }+\underset{{\mathrm{0}}^{\circ }}{\overset{\mathrm{4}{}^{\circ }}{\int }}{\text{Tl}}_{\text{dif}}\mathrm{d}\mathit{\theta }+\underset{-{\mathrm{4}}^{\circ }}{\overset{\mathrm{0}{}^{\circ }}{\int }}{\text{Tl}}_{\text{dif}}\mathrm{d}\mathit{\theta }\right].\end{array}\end{array}$$

T_dif,FOV is the extra transmittance in the measurement of direct-beam transmittance made by the sensor. Figure A2 shows the dependence T_dif,FOV has on τ_cld.

Figure A2 T_dif,FOV as a function of τ_cld for the 500 nm wavelength. The relationship is shown for multiple SZAs ranging from 10–70^∘.

For RS, the relationship between τ_cld and the optical depth inferred from the method, τ_i, is determined by inverting Beer's law (Eq. 1) and inserting the biased transmittance:

$$\begin{array}{}\text{(A3)}& {\mathit{\tau }}_i=-\mathit{\mu }\ln (T_{\text{dir,true}}+T_{\text{dif,FOV}}),\end{array}$$

where T_dir,true=T_dir,true(τ_cld) is the direct-beam transmittance given a true cloud optical depth. Here again we use libRadtran to simulate T_dir,true for each τ_cld value used in the simulations of T_dif,FOV. Likewise, for RD, T_dif,FOV is used to calculate the bias in DR, and then Eq. (5) is used to find τ_i:

$$\begin{array}{}\text{(A4)}& {\mathit{\tau }}_i=\mathit{\mu }\ln \left(\mathrm{1}-{\displaystyle \frac{F_{\text{dif,true}}+F_{\text{dif,FOV}}}{F_{\text{tot},\text{true}}}}\right),\end{array}$$

where F_dif,true and F_tot,true are the true irradiances given a true cloud optical depth, and F_dif,FOV is the extra diffuse irradiance the sensor sees due to the wide FOV of the shadow mask ($F_{\text{dif,FOV}}=F_{\mathrm{0}}\times T_{\text{dif,FOV}}$). Equations (A3) and (A4) form the relationships between τ_i and τ_cld, which we then fit with fourth- or sixth-degree polynomials. The resulting curves give the correction factors for the retrieval outputs, τ_RS,cld and τ_RD. The retrieval outputs are the sensor-inferred optical depths (τ_i), and we directly map these to the τ_cld values; these relationships are illustrated in Fig. A3.

Figure A3 Relationships of τ_cld vs. τ_i for RS and RD (Eqs. A3 and A4) for the 500 nm wavelength, with SZA ranging from 10–70^∘.

We derive these τ_cld correction relationships for SZA ranging from 10–70^∘ and at wavelengths of 500, 670, and 870 nm. When applying the corrections, the SZA dependence is accounted for by interpolating the derived factors to the SZA at the time of the sample. It is important to note that for the diffuse case, the correction is limited because we ignore the influence a non-zero surface albedo has on T_dif,FOV. Further, applying the correction to the output of RD when aerosols are present will cause errors, and so the correction is only applied in aerosol-free regions. We also ignore the impact ice crystal habit has on this correction because for thin cirrus the overall correction is relatively small (∼0 %–20 %), and any error crystal habit induces will therefore have minimal impact on the final retrieved τ_cld value.