Aerosols are ubiquitous throughout the Earth's atmosphere, and they play a crucial role in modulating the flux of solar radiation that reaches the Earth's surface. The energy distribution within a scene that contains aerosols depends not only on the amount of incoming solar radiation, aerosol optical depth (AOD), and type, but also on the albedo beneath the aerosols. Depending on the type of aerosol, the incoming radiation will be absorbed or scattered in a certain ratio, described by the single scattering albedo (SSA), while the direction (forward or backward) of the scattered radiation can be approximated by the asymmetry parameter (g). Aerosol absorption and scattering change the radiative balance relative to the aerosol-free atmosphere. This perturbation is called the direct aerosol radiative effect (DARE). The scene albedo below an aerosol layer, whether from clouds, ocean, or land, can determine whether the layer has a negative (positive) DARE, resulting in a cooling (warming) effect at the top of the atmosphere (Twomey, 1977; Russell et al., 2002). Aerosols injected into the global climate system by human activity since the beginning of industrialization may offset up to 50 % of the warming due to anthropogenic greenhouse gas emissions (Myhre et al., 2013). However, the uncertainty of this offset is large, in part due to observational challenges: radiative forcing by anthropogenic aerosol–radiation interactions could range from -1 to +0.2 W m-2 (Fig. 8.15 in Myhre et al., 2013).

Deriving the direct effect of aerosols on the radiation budget, ignoring for the moment the impact on radiative balance due to aerosol influences on cloud properties and lifetime, is difficult since DARE is derived from the difference between radiative fluxes in the presence of and absence of aerosol. It is impossible to observe both states simultaneously, and therefore, DARE is not directly measurable and, in most cases, requires a radiative transfer model (RTM) initialized with observational or model inputs of aerosol AOD, SSA, and g as well as the spectral reflectance or albedo below the aerosol layer. The DARE calculations are limited by the accuracy of the observations and the model accuracy itself. For conditions where absorbing aerosols overlie inhomogeneous cloud fields, determining DARE is even more challenging since the calculations require both the aerosol properties as well as the cloud properties, primarily the cloud spectral albedo. The cloud radiative signal can be relatively large compared to that of aerosol particles. Therefore, it can be difficult to isolate the aerosol radiative effect from that of clouds, especially when the cloud albedo varies in the sampling region.

Obtaining the necessary cloud and aerosol parameters from satellite instruments provides the flexibility to estimate DARE in nearly any region. Until recently, aerosol and cloud properties could not typically be measured from the same satellite when the aerosol occurs above the clouds, and the strategy to estimate DARE for these conditions was to combine properties from multiple satellites (e.g., Chand et al., 2009; Meyer et al., 2013; Zhang et al., 2016; Sayer at al. 2016; Kacenelenbogen et al., 2019; Oikawa et al., 2018; Korras-Carraca et al., 2019). The problem with this approach, however, is that biases in the cloud and aerosol properties translate into biases in DARE if left unaccounted for (Meyer et al., 2013). For example, many DARE studies utilize Moderate Resolution Imaging Spectroradiometer (MODIS) cloud optical thickness (COT) and effective droplet size (translated into cloud albedo, which cannot be directly measured from space) and/or AOD from the active lidar instrument Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP). However, MODIS cloud retrievals can be biased when absorbing aerosols are present above cloud (Haywood et al., 2004; Wilcox et al., 2009; Coddington et al., 2010) and CALIOP AOD, which was known to be low-biased for daytime measurements (Kacenelenbogen et al., 2011; Winker et al., 2013; Meyer et al., 2013; Jethva et al., 2014) until the development of a new method in version 4 to derive AOD above cloud that uses the cloud returns to derive a much more accurate measure of AOD above cloud (Kim et al., 2018).

Work has been done to characterize and correct for the biases in cloud and aerosol properties in DARE estimates (Meyer et al., 2013; Zhang et al., 2016). Meyer et al. (2015) account for the satellite cloud optical property bias by developing a simultaneous retrieval of cloud optical thickness and effective radius and aerosol AOD from MODIS imagery alone, thus obtaining both aerosol and cloud properties from a single instrument that are used as inputs into DARE calculations. Jethva et al. (2013) also retrieve AOD and COT from MODIS alone, using the color ratio method to derive DARE.

Table 1 in Kacenelenbogen et al. (2019) provides a summary of DARE studies and the methods used to obtain aerosol and cloud properties, and it is clear that although methods to account for satellite AOD and COT biases have been established, the aerosol SSA and g remain difficult to obtain. Often, SSA and g are obtained from an assumed aerosol model, such as the MODIS MOD04 absorbing aerosol model used in Meyer et al. (2013, 2015) or the CALIOP aerosol sub-type models used by Zhang et al. (2016). This approach requires the correct aerosol model to be chosen, and some studies choose instead to use optical properties from an outside source. For example, Chand et al. (2009) combine CALIPSO aerosol AOD and Ångström exponent with MODIS COT, but assume a regional mean value of SSA from the Southern African Regional Science Initiative (SAFARI) 2000 campaign to derive diurnal DARE. Jethva et al. (2013) estimate DARE using the SSA obtained from Aerosol Robotic Network (AERONET) sites. Since these measurements of SSA are not taken in conjunction with the other cloud and aerosol properties, it is difficult to determine whether they are valid and consistent for the specific aerosol measured by the satellite.

Some studies, such as Peers et al. (2015) and de Graaf et al. (2012), have developed unique methods for which aerosol properties are not assumed. Peers et al. (2015) derive aerosol and cloud properties simultaneously through polarization measurements made by the Polarization and Directionality of Earth Reflectances (POLDER) instrument on the PARASOL satellite, while de Graaf et al. (2012) avoid the aerosol properties altogether and simulate a cloud-only sky and compare this to measured hyperspectral reflectances from the Scanning Imaging Absorption Spectrometer for Atmospheric Chartography (SCIAMACHY). de Graaf et al. (2019) compare DARE from these two methods along with DARE derived from OMI and MODIS for the southeastern Atlantic region, finding that DARE is correlated between all methods for moderate values of DARE. However, POLDER-derived values are higher than the other two methods for higher DARE values, which they attribute to larger cloud optical thickness retrievals by POLDER.

Aircraft observations, as opposed to satellite remote sensing, provide in situ observations of clouds and aerosols that are better suited for deriving their radiative properties, especially when the clouds are inhomogeneous. For example, an aircraft can fly through an aerosol layer to measure aerosol absorption, scattering, and SSA with in situ instruments or fly directly above a cloud layer to measure the albedo. Studies such as Pilewskie et al. (2003), Redemann et al. (2006), Schmidt et al. (2010a), Coddington et al. (2010), LeBlanc et al. (2012), and Ehrlich et al. (2017) along with the work presented here have capitalized on this versatility and developed new algorithms and instrumentation to determine aerosol and cloud properties, which can then be utilized to estimate DARE.

For example, under the specific conditions of an aerosol layer with a loading gradient above a homogenous, dark surface, Redemann et al. (2006) derived the below-layer aerosol forcing efficiency (radiative effect per mid-visible AOD) from the co-varying irradiance/AOD pairs along a leg with minimal dependence on radiative transfer calculations. This method, however, is not applicable for scenes with absorbing aerosols above clouds such as those encountered during the recent NASA ObseRvations of CLouds above Aerosols and their intEractionS (ORACLES) project (Zuidema et al., 2016). The ORACLES project conducted three aircraft campaigns in the southeastern Atlantic, providing measurements in a region with high biomass burning aerosol loading where there have been few extensive field observations to date. In this study, we combine data from multiple instruments to retrieve the aerosol and cloud properties as directly as possible in order to calculate DARE and investigate the relationship between DARE and cloud albedo. The sensitivity of DARE above the aerosol layer to the underlying surface can be described by the transition from a negative to a positive radiative effect, or cooling to warming (Russell et al., 2002). The albedo where this transition occurs, hereafter called the critical albedo, expands upon the quantities of critical reflectance and critical surface albedo that more specifically refer to the relationship between AOD and top of atmosphere reflectance (Fraser and Kaufman, 1985; Seidel and Popp, 2012). The dependence of the sign of the aerosol's radiative effect on the underlying albedo has been shown for aerosols above clouds in the southeastern Atlantic by Keil and Haywood (2003), Chand et al. (2009), and Meyer et al. (2013).

ORACLES aircraft observations make up an extensive dataset that can be used to validate current satellite methods of deriving the aerosol and cloud properties that go into calculations of DARE. To begin this process, our primary objective of this paper is to derive DARE as a function of (a) the aerosol optical properties and (b) cloud albedo from the ORACLES measurements. In Sect. 2, we describe the observations themselves and the sampling approaches used to obtain them. Section 3 describes the methods used to determine SSA and g and how we utilize the results to calculate DARE as directly as possible. Section 4 presents our findings, while Sect. 5 provides a discussion and ways in which we will explore DARE's dependence on aerosol properties in the future, along with prospective satellite validation goals.

The first two deployments of the NASA ORACLES experiment were conducted from Namibia in 2016 and from São Tomé in 2017, regions located on or just off the western coast of the African continent. The southeastern Atlantic Ocean is often covered by a seasonal stratocumulus cloud deck capped by a thick layer of biomass burning aerosols advected from the interior of the African continent, providing ideal natural conditions to assess aerosol radiative effects above various cloud scenes and improve the understanding of many aspects of cloud–aerosol interactions.

Both the NASA P-3 and the ER-2 aircraft were deployed in the 2016 campaign. The P-3 flew at approximately 5 km altitude and below, carrying a comprehensive payload of both in situ and remote sensing instruments. The ER-2 flew at high altitude, approximately 20 km, carrying remote sensing instruments such as the enhanced MODIS Airborne Simulator (eMAS) and the High Spectral Resolution Lidar 2 (HSRL-2) that collected simultaneous and collocated measurements with the P-3 during several coordinated flights. During the 2016 deployment, the P-3 completed 14 science flights in total, 5 of which were collocated with the ER-2 and 9 of which included radiation-specific sampling maneuvers. Although the ER-2 did not participate in the 2017 deployment, the P-3 payload remained nearly the same except for the addition of HSRL-2 that had been deployed on the ER-2 during the 2016 campaign. Therefore, we focus on utilizing measurements taken from the P-3, which conducted 12 science flights in total, with 5 flights dedicated to radiation-specific studies in 2017. In this study, we primarily use measurements taken by SSFR, 4STAR, and HSRL-2 to investigate two cases, 20 September 2016 and 13 August 2017, which met specific requirements such as varying scene albedos and large aerosol loading. A companion paper will present more generalized results.

The SSFR (Pilewskie et al., 2003; Schmidt and Pilewskie, 2012) is comprised of two pairs of spectrometers. Each pair consists of one spectrometer that is sensitive over the near-ultraviolet, visible and very near-infrared wavelength range, and another that is sensitive in the shortwave infrared wavelength range. The spectra are joined at 940 nm to provide a full spectral range from 350 to 2100 nm. The SSFR measures downward spectral irradiance (Fλ↓) from a zenith light collector mounted on a stabilizing platform on the upper fuselage and upward spectral irradiance (Fλ↑) from a nadir light collector fix-mounted to the aircraft. The externally mounted light collectors are connected by fiber optic cables to the spectrometer, which resides in the aircraft cabin along with the data acquisition unit. The SSFR was radiometrically calibrated with a NIST-traceable 1000 W lamp light source before and after each deployment, and relative calibration changes throughout the field campaign were monitored with a portable field standard. The light collectors consist of an integrating sphere with a circular aperture on top. They weigh the incoming radiance according to an angular response close to the cosine of the incidence angle. These light collectors have been improved over time (Kindel, 2010) to minimize the dependence on the azimuth angle of the incident radiance. However, the dependence on the polar angle, termed the cosine response, still requires careful characterization in the laboratory before and after the deployment. After applying all corrections, the uncertainty of the SSFR measurements is 3 %–5 % across the spectral range for both zenith and nadir irradiance. More importantly for this study, the precision is 0.5 %–1.0 %.

The zenith light collector of SSFR was kept horizontally aligned by counteracting the variable aircraft attitude with an Active Leveling Platform (ALP), which was developed at CU Boulder for the NASA C-130 aircraft (Smith et al., 2017) and later rebuilt for the P-3, specifically for ORACLES. ALP relies on aircraft attitude information from a dedicated inertial navigation system (INS) that monitors the aircraft attitude, specifically the pitch and roll angles. This information is sent to a real-time controller, which additionally has the ability to instead ingest data from the aircraft INS. The controller drives the two actuators of a two-axis tip–tilt stage: one axis for aircraft roll movements and one for aircraft pitch movements. As the attitude angle changes, the tip–tilt stage adjusts accordingly to maintain the SSFR at the horizontal level position within approximately 0.2∘. The nadir light collector was not actively leveled since the horizontal cloud variability introduces much more variability into the signal than any attitude changes. The upwelling irradiance is also less sensitive to pitch and roll angles than the downwelling irradiance.

For fix-mounted zenith light collectors, not only will the downward irradiance be referenced to an incorrect zenith due to the polar angle of incident light referenced to the aircraft horizon rather than true horizontal, but radiation from the lower hemisphere will also contaminate the zenith irradiance measurements if the receiving plane is not properly aligned with the horizon. This is especially problematic over bright surfaces such as snow, ice, or clouds. For ORACLES, it was important to sample the dependence of the downwelling irradiance on the aerosol conditions above. Since the aerosol-induced irradiance changes are small compared to the reflection by clouds, even minor contamination from the lower hemisphere could cause a bias in the signal. Such biases cannot be corrected in post-processing because common correction schemes assume that no radiation originates in the lower hemisphere (Bucholtz et al., 2008). ALP alleviates these problems and enables the collection of irradiance data during spiral measurements as long as pitch and roll stay within the ALP operating range of 6∘. For the reasons mentioned above, spiral data have traditionally not been useful for radiation science. In this study, they turn out to be the key for achieving our stated goals.

The 4STAR instrument provides direct-beam measurements of AOD above the aircraft at hundreds of wavelengths ranging from 350 to 1650 nm, with a subset of 24 wavelengths available in the main ORACLES data archive (ORACLES Science Team, 2017a, b, 2019). The instrument is calibrated before and after each deployment using the Langley plot technique (Schmid and Wehrli, 1995); in addition, corrections for non-uniform azimuthal dependence of the transmission of the optical fiber path were assessed after each flight calibration (Dunagan et al., 2013) and corrected for in post-processing, resulting in an average AOD uncertainty of 0.011 at 500 nm (LeBlanc et al., 2019). 4STAR also provides other quantities, for example, column water vapor and trace gas retrievals, which are not used here. HSRL-2 is a downward-pointing lidar that provides vertical profiles of aerosol backscatter and depolarization at 355, 532, and 1064 nm wavelengths. Aerosol extinction is measured at 355 and 532 nm wavelengths (Hair et al., 2008; Burton et al., 2018). When the ER-2 was collocated with the P-3; imagery from the eMAS multispectral imager (King et al., 1996; Ellis et al., 2011) provided scene context.

There are two ways to determine aerosol-intensive optical properties from irradiance and AOD. An algorithm by Schmidt et al. (2010a) uses nadir and zenith irradiance pairs above and below a layer to retrieve SSA, g, and the surface albedo. A different algorithm, by Bergstrom et al. (2010), first derives the layer absorption and scene albedo from the irradiance pairs above and below the layer and then infers SSA, assuming a fixed value for g. Both methods were applied to clear sky and require irradiance measurements above and below a layer along with the associated AOD, which are most often obtained from individual points along the upper or lower leg of a “radiation wall” as shown in Fig. 1.

The intent of the wall is to obtain scene albedo, layer absorption, or transmittance by bracketing the aerosol layer above and below when flying at multiple altitudes along a track of about 100 km length. When only one aircraft is available, it samples the required legs sequentially, taking over an hour to complete. In clear sky, an aerosol layer will likely not change substantially during this time. However, in cloudy skies such as those encountered during ORACLES, the time lag between sequential sampling of the upper and lower legs is large enough that the cloud field is likely to change. Figure 1 illustrates the sampling for only two altitudes: at the bottom of the layer (BOL) and at the top of the layer (TOL) of interest. In the case of ORACLES, the BOL leg is located just below the aerosol layer and just above the cloud layer, where the column AOD and scene albedo are measured. The TOL leg is above the aerosol layer and the cloud, from which HSRL-2 measures profiles of extinction. Many other legs, for example below and within the cloud, and within the aerosol layer, were typically flown in addition to the BOL and TOL legs.

The net irradiance (Fλnet) at any level is the difference between the downwelling and upwelling irradiance. The absorption Aλ of a layer can be determined from the difference of the net irradiance at the upper and lower boundaries (the vertical component Vλ of the flux divergence) if the horizontal flux divergence of radiation Hλ is negligible (|Hλ| < < |Vλ|). Under horizontally homogeneous conditions, we assume Aλ=Vλ, which is usually the case, giving 1aAλ=Vλ=Fλ,tolnet-Fλ,bolnetFλ,tol↓=Fλ,tol↓-Fλ,tol↑-Fλ,bol↓-Fλ,bol↑Fλ,tol↓, where Aλ and Vλ have been normalized by the incident irradiance at the top of the layer (Fλ,tol↓).

Under partially cloudy conditions, 1bAλ=Vλ-Hλ. Schmidt et al. (2010b) found that Hλ of a cloud layer is not negligible and can attain a magnitude comparable to Aλ itself. Song et al. (2016) described the physical mechanism and spectral dependence of Hλ, which was determined by bracketing the cloud layer with irradiance measurements. For ORACLES, the aerosol above clouds, rather than the cloud itself, constitutes the layer of interest, but Fig. 1 illustrates how non-zero values of Hλ may arise under inhomogeneous conditions. The aerosol retrievals are only accurate if |Hλ| < < |Vλ|, which ensures that Eq. (1a) holds. The data analysis showed that this condition was rarely met for wall measurements, but more often during spiral measurements (Sect. 3.1).

The radiation spiral, shown in Fig. 2 and illustrated conceptually in Fig. 1, provides multiple irradiance and AOD samples throughout the layer. Such a sampling pattern provides irradiance measurements at four headings throughout the column at a high vertical resolution without increasing the duration of the profile because the aircraft keeps descending or ascending during the straight segments, typically at 1000 feet (approximately 305 m) per minute. During ORACLES, spirals were on average completed over the course of 10–20 min, depending on the vertical extent of the aerosol layer. Since typical roll angles during turns were 15–30∘, exceeding the operating range of ALP, the spirals include short, straight segments of 20–30 s duration every 90∘ heading change. The small pitch and roll values during the straight segments can be corrected for in real time by the ALP and lead to a rounded square-shaped pattern as shown in Fig. 2b and d rather than a traditional, circular spiral pattern. SSFR acquires the irradiance profile over a minimal horizontal extent approximately 10 km in both latitude and longitude, reducing cloud and aerosol inhomogeneity effects, and over a much shorter time interval relative to the wall, maintaining correlation of measured irradiances throughout the spiral to the ambient cloud field. A circular spiral pattern with no straight segments with a roll angle under the roll limit would cover too large an extent, and the benefits of the square spiral pattern would be lost. Moreover, the four heading angles allow biases from mechanical mounting offsets of ALP or reflections and obscuration by the aircraft structure to be diagnosed. Acquiring a large number of samples over a relatively limited horizontal extent also reduces the impact of cloud albedo variability on the nadir irradiance. The downside of the spiral sampling is that it does not capture the spatial variability of the scene albedo, which is assessed by the radiation wall. Therefore, in order to investigate any spatial relationships between radiative effects and albedo, spiral measurements must be used in conjunction with AOD and scene albedo measurements from the radiation wall where the albedo is defined as 2albedoλ=Fλ↑Fλ↓.

To characterize the connections between DARE, aerosol properties, and scene albedo, we chose to explore cases based on (a) the availability of measurements from both a radiation wall and a radiation spiral, (b) relatively high aerosol loadings above the cloud field, and (c) a range of measured albedos. The first case is 20 September 2016, where the spiral was located approximately 2.5 degrees of longitude off the coast near the Namibia/Angola border. The cloud field for this case was homogeneous; the albedo of the BOL leg of the radiation wall ranged from 0.39 to 0.59 at 501 nm. The radiation spiral was located at the northern end of the BOL leg. The aerosol layer was geometrically and optically thick, with an AOD measured just above clouds during the spiral of 0.57 at 501 nm. The ER-2 flew in coordination with the P-3, such that eMAS imagery is available for context. Figure 2b shows an eMAS image overlaid with the flight track of the P-3 for the spiral flight pattern, along with the ER-2 flight track. The second case on 13 August 2017, located approximately 8 degrees of longitude off of the coast of northern Angola, was chosen because of the inhomogeneous cloud conditions encountered along the BOL leg of the radiation wall. We treat this leg of the radiation wall as two separate cases based on differing albedo ranges – the northern end of the wall, where the albedo values at 501 nm range from 0.06 to 0.39, and the southern end of the wall, where the albedo at 501 nm ranges from 0.29 to 0.75. The radiation spiral was located on the southernmost point, though we use the retrieval results for both the North case and South case DARE calculations. Fig. 2d shows the spiral flight path overlaid on visible imagery from SEVIRI (Spinning Enhanced Visible and Infrared Imager) onboard the geostationary Meteosat Second Generation (MSG) (Schmid, 2000). The aerosol layer was significantly thinner than that of 20 September 2016, with an AOD at 501 nm measured just above clouds during the spiral of 0.22. Table 1 lists the important parameter ranges for both spirals: UTC, latitude, longitude, solar zenith angle (SZA) and albedo at cloud top. Table 2 lists these parameters for the BOL legs for each of the three cases.

To derive accurate aerosol absorptance from SSFR measurements, irradiance pairs above and below the layer must first be obtained. For the radiation walls, the irradiance pairs are sampled from the BOL and TOL legs at coincident locations, neglecting cloud advection and cloud evolution during the elapsed time between the two. For spirals, the entire measurement profile from above cloud to above aerosol is used to establish a linear fit of the data from which irradiance pairs are derived, improving the sampling statistics compared to radiation wall irradiance pairs. Figure 3 illustrates SSFR measured nadir and zenith irradiances for aircraft attitudes within the operating range of ALP plotted against the 4STAR AOD at 532 nm as a vertical coordinate. Uncertainty bars are included for a subset of the measurements. Prior to fitting, all data are corrected to the SZA at the midpoint of the spiral to account for the minor change in solar position throughout the spiral: 3Fλ=Fλ×μ0μ, where μ=cos⁡(SZA) and μ0 is the value at the midpoint of the spiral.

To derive the BOL and TOL irradiances from the spiral using all the data, a linear regression is performed: 4aFλ↑=aλ↑+bλ↑×AOD532,4bFλ↓=aλ↓+bλ↓×AOD532, where aλ and bλ are the slope and intercept of the linear fit lines. The data points from the spiral are used collectively to establish the fit coefficients in Eqs. (4a) and (4b), which express the change in nadir and zenith irradiance with AOD. Subsequently, the irradiance values at BOL and TOL are determined from AOD532max, measured at the bottom of the layer, and AOD532min, measured at the TOL. This method is more robust than picking individual irradiance pairs from the wall because many more data points are used. The uncertainty of the fit coefficients is dominated by the variability of the data throughout the vertical profile, rather than by the radiometric uncertainty of the contributing data points, discussed in more detail in Sect. 3.4.

At a wavelength with small aerosol effects, such as 1.6 µm, shown in Fig. 3b, d, neither nadir nor zenith irradiance change significantly as the aircraft moves through the layer. Any observed non-linearity or variability in the vertical profile at this wavelength can be ascribed to spurious measurement errors: for zenith, these could be due to reflections or obstructions by the aircraft or other factors causing a transient variability in the downwelling irradiance. For nadir, this is attributed to albedo changes in the cloud field below. By contrast, the irradiance at 532 nm (Fig. 3a, c) changes considerably throughout the vertical profile. The zenith irradiance decreases with decreasing altitude due to the increasing attenuation by the aerosol layer. The nadir irradiance shows the opposite behavior, decreasing with increasing altitude. By comparison, zenith and nadir irradiance would change in lock step for a purely scattering layer because the net irradiance remains constant in the absence of absorption.

The retrieval of SSA and g is done with publicly available one-dimensional (1-D) radiative transfer model (RTM) DISORT 2.0 (Stamnes et al., 2000), with SBDART for atmospheric molecular absorption (Ricchiazzi et al., 1998) along with the standard tropical atmosphere available within the libRadtran public library (Emde et al., 2016; http://www.libradtran.org, last access: 15 November 2019). In contrast to the algorithms by Pilewskie et al. (2003), Bergstrom et al. (2007), and Schmidt et al. (2010a), the aerosol layer is located over a variable cloud scene, but otherwise the principle is the same. This work is most similar to the algorithm introduced by Schmidt et al. (2010a), for which SSA and g are retrieved simultaneously.

The RTM allows us to calculate upwelling and downwelling fluxes determined by inputs of the surface albedo and the aerosol properties of AOD, SSA, and the asymmetry parameter. The updated retrieval algorithm is based on the comparison between the calculated fluxes and the SSFR-measured fluxes. Spectral albedo from SSFR and AOD from 4STAR are used as inputs, which leaves SSA and g as the free retrieval parameters. For 20 September 2016, the SZA within the RTM is set to 21.0 and the albedo at 501 nm is 0.45, while for 13 August 2017, the SZA is set to 33.5 and the albedo at 501 nm is 0.70. Since the cloud albedo is directly measured, cloud properties such as COT and effective radius are not required – an advantage when compared to the associated remote sensing bias when obtaining it from space-borne imagery (Chen et al., 2019).

The first step in the retrieval is to condition the 4STAR AOD so that the column-integrated AOD profile decreases monotonically with altitude. Because 4STAR samples horizontal as well as vertical variability throughout the spiral, the AOD profile can sometimes deviate from a strictly monotonic decrease, which cannot be ingested by the RTM. We alleviate this problem by smoothing the AOD profile with a polynomial to eliminate minor deviations from monotonic behavior. For instances when the derived extinction becomes negative, we set the value to 0. Figure 5a (20 September 2016) and b (13 August 2017) visualize the original AOD profile and the corresponding polynomial. The unique altitude to AOD relationship is used to derive the extinction profile, also shown in Fig. 5a, b. Above the aerosol layer, any remaining AOD measured by 4STAR is assigned to a layer extending to 15 000 m (a top altitude chosen somewhat arbitrarily lacking the knowledge of the correct height distribution of the residual AOD). While a direct comparison of 4STAR above-cloud AOD (LeBlanc et al., 2019) and HSRL-derived column-integrated AOD for 532 nm is possible, it is not straightforward due to the different viewing geometries of the instruments and is not done here.

In the second step of the retrieval, the RTM calculates the upwelling and downwelling irradiance profiles for each given {SSA,g} pair within a broad, physically reasonable range. The modeled downwelling irradiance profile is rescaled such that the model results at the TOL are consistent with the measured downwelling irradiance. The scaling factor effectively allows for inaccurate values in the extraterrestrial solar flux (Kurucz, 1992), for differences in atmospheric constituents, such as aerosols above the aircraft's top altitude, or for absorbing gases not accounted for using the standard atmospheric profile. It is typically close to 1. At the BOL, the measured upwelling irradiances are also rescaled such that the model albedo is consistent with measured albedo. If the calibration for the upwelling and downwelling irradiance is consistent, the scale factors should be the same. Therefore, any retrieval with differing nadir and zenith scale factors is flagged as failed.

The third step of the retrieval determines the most probable pair of {SSA,g} and calculates the uncertainty. For each {SSA,g} pair calculation, every SSFR data point in the profile is assigned a probability according to the difference between the calculation and the measurement. The probability of {SSA,g} given the SSFR observations is determined from the Gaussian distribution that represents the measurement uncertainty. This is illustrated in Fig. 6a. The probability of that pair given the observations is determined by multiplying the individual probabilities within the profile. The {SSA,g} pair with the highest probability value is reported as the retrieval result. The {SSA,g} pair probabilities are shown as a 2-D probability density function (PDF) in Fig. 6b, where the error bars show the 1σ uncertainty for SSA and g separately, determined by the respective marginal (1-D) PDFs. Since only the SSFR uncertainty is considered within the retrieval, the 4STAR uncertainty is treated separately by performing the retrieval three times: (1) for the nominal AOD, (2) for the nominal AOD – range of uncertainty, (3) for the nominal AOD + range of uncertainty. Figure 6b shows an example retrieval at 380 nm for the three retrievals. Finally, the retrieved spectra of 4STAR wavelengths between 355 and 660 nm of SSA and g are reported, with a range of uncertainty that encompasses the three separate retrievals.

Currently, the retrieval is performed for each wavelength individually, and no spectral smoothness constraints are applied. This is an important difference compared to other methods such as the AERONET inversion method that retrieves aerosol size distributions and the real and imaginary parts of the index of refraction for various size modes (Dubovik and King, 2000).

The retrieval also allows us to calculate the absorption Ångström exponent (AAE) from the absorbing aerosol optical depth (AAOD) which, like SSA, quantifies the radiative effects and optical properties of absorbing aerosols (Pilewskie et al., 2003; Bergstrom et al., 2010). The AAE and AAOD are determined as follows: 8aAAOD=(1-SSA)×AOD,8bAAOD=AAOD500×λλ500-AAE. We compare the AAE and SSA results from our retrieval to in situ measurements from a three-wavelength nephelometer (TSI 3563) and a three-wavelength particle soot absorption photometer (PSAP) (Radiance Research). The PSAP provides AAE, while the combination of scattering from the nephelometer and absorption from the PSAP provides SSA. Average values of SSA are weighted by the extinction, specifically to obtain a column value of SSA from the spiral profiles.

We calculate the DARE at the TOL and BOL as the difference between the net irradiance with and without the aerosol layer: 9DAREλ=Fλ,aernet-Fλ,noaernet. The DARE calculations are performed with the aerosol-intensive properties (SSA and g) from the spiral profiles and with the albedo as measured by SSFR and AOD from 4STAR from a BOL leg. HSRL-2 extinction profiles are taken from the TOL leg (2017) or from the collocated ER-2 leg (2016) to capture any variability within the aerosol encountered along the wall.

Combining vertical and horizontal sampling in this way is predicated on the assumption that the aerosol-intensive properties do not change along the BOL leg, whereas albedo and AOD are expected to vary. This is a reasonable assumption as long as the legs do not cross an air-mass boundary; in situ measurements show that along the BOL leg on 20 September 2016, the SSA at 530 nm ranges from 0.80 to 0.86 (0.83±0.01, average ± standard deviation). During this same time, the PSAP instrument shows that the AAE ranges from 1.71 to 2.02 (1.87±0.05). From a radiation wall leg within the aerosol layer (12:35–12:47 UTC), the SSA ranges from 0.84 to 0.87 (0.85±0.004). The AAE ranges from 1.71 to 1.99 for this time (1.84±0.04). On 13 August (both the northern and southern sections), the SSA from the BOL leg ranges from 0.84 to 0.93 (0.87±0.02), while the AAE ranges from 0.97 to 2.1 (1.6±0.3). Within the aerosol layer (14:08–14:18 UTC), the SSA ranges from 0.88 to 0.90 (0.89±0.003) and the AAE ranges from 1.80 to 2.16 (1.92±0.07) (Dobracki et al., 2019). In light of the AAE and SSA ranges in the in situ measurements, it does not appear that the legs crossed an air-mass boundary, but the aerosol-intensive properties also cannot be considered constant. However, the measured variability in the in situ SSA is captured by the standard deviation of its retrieved counterpart and is thus propagated into an uncertainty for DARE.

Since the spectral information is available, we choose to calculate DARE spectrally (350–660 nm) as a percentage of the incoming radiation rather than as broadband values commonly reported. Within the RTM, the SZA is fixed to the mean value of the above cloud leg; for consistency, SSFR measurements are corrected to this SZA following Eq. (3) (17.9∘ for 20 September 2016, 22.1∘ for the North case of 13 August 2017, and 23.2∘ for the South case of 13 August 2017). The albedo ranges for each case are presented in Table 2, and the aerosol-intensive properties used are presented in Fig. 7. Although 0 and 1 albedo values were not actually encountered, we include them in the RTM runs and calculate the DARE to investigate the behavior at the albedo limits. The relationship between DARE and SSFR-measured albedo is nearly linear; therefore, we fit a line to the 1-D calculations to find the x intercept, which is the critical albedo.

To estimate the total DARE uncertainty, we combine the errors of the individual components: 10δDAREtotal=δDAREg2+δDAREalbedo2+δDAREAOD2+δDARESSA2‾, where each parameter uncertainty is calculated as 11aδDAREg=DAREg+δg-DAREg-δg2,11bδDAREalbedo=DAREalbedo+δalbedo-DAREalbedo-δalbedo2,11cδDAREAOD=DAREAOD+δAOD-DAREAOD-δAOD2,11dδDARESSA=DARESSA+δSSA-DARESSA-δSSA2. The uncertainties of g and SSA are obtained from their retrieval, and the AOD uncertainty is the measurement uncertainty. Since the albedo is a ratio of upwelling and downwelling irradiance, calibrated using the same apparatus, the relative precision of the measurements to each other drives the uncertainty rather than through error propagation of each calibrated accuracy. The albedo uncertainty is estimated to be approximately 1 %.

This method assumes all four individual uncertainties are uncorrelated and most likely overestimates the DARE uncertainty.

The SSA spectra from 355 to 660 nm retrieved from the radiation spirals for each case are shown in Fig. 7a, b, and Table 5 presents a comparison between SSFR-derived SSA and AAE with past results and in situ measurements from ORACLES. The 20 September 2016 case can be considered spectrally flat with a minimum SSA value of 0.83 (±0.02) at 660 nm and a maximum SSA value of 0.86 (±0.01) at 380 nm. The 13 August 2017 case shows a spectrally flat SSA with 0.83 (±0.04) at 355 nm and 0.82 (±0.07) at 660 nm. Compared to the SAFARI 2000 campaign results shown in Russell et al. (2010), the results from the two ORACLES cases are slightly lower, 0.87 at 501 nm compared to 0.85 (±0.01) (20 September 2016) and 0.82 (±0.05) (13 August 2017) at 501 nm, although the values are similar to those presented in Giles et al. (2012) for AERONET sites that experienced smoke aerosol events (Giles et al., 2002; Eck et al., 2003a, b). In situ measurements of the extinction weighted SSA from the spiral profiles and are shown in Fig. 7a, b. At 530 nm, the 20 September 2016 spiral had an average SSA of 0.86 with a standard deviation of 0.03, while the 13 August 2017 spiral had an average SSA of 0.88 with a standard deviation of 0.01. Table 4 presents a comparison at 500 and 530 nm between SSFR-derived SSA and AAE with past results and in situ measurements from ORACLES, and a detailed SSA inter-comparison can be found in Pistone et al. (2019).

Also included in Fig. 7a, b are the uncertainty estimates for each wavelength, shown as the smaller, blue error bars. The larger, black error bars illustrate what the uncertainty would be if we had derived the SSA using irradiance pairs rather than from the whole profile (i.e., if the spiral TOL and BOL values had been taken from a radiation wall). The uncertainty derivation for the radiation wall measurements requires the assumption that Hλ=0, though as we have shown, this is not the case and is described in detail in Appendix A. As can be seen in Fig. 7, the uncertainty from the walls is much larger than from the new spiral method and would be even larger if we included error due to Hλ.

Figure 7c shows the asymmetry parameter retrievals along with uncertainty estimates. The values (0.45–0.65) for the 20 September 2016 case are within the range of other estimates for the region, although the spectrum falls off more rapidly than assumed by Meyer et al. (2013). The large uncertainties for the 13 August 2017 case show that even for moderate mid-visible AOD (∼0.3), the information content with respect to this retrieval parameter is fairly low. Despite the limited information content in the SSFR stand-alone retrievals, there is some indication that the asymmetry parameter always falls off more rapidly than in previous assessments – with a value approaching zero for large wavelengths. This may be due to fewer coarse-mode aerosol particles than in previous climatologies for the region (Formenti et al., 2018).

The AAOD spectrum from which we derive AAE is shown in Fig. 7d for both cases. The AAE for the 2016 case is 1.29, while the AAE for the 2017 case is 1.44. Both AAE values are similar to the results of Bergstrom et al. (2007) and reproduced by Russell et al. (2010) from the SAFARI campaign for biomass smoke of 1.45 for wavelengths of 325 to 1000 nm. In situ measurements of AAE from the PSAP showed the average AAE values from the two spirals profiles to be 1.79 for 20 September 2016 and 1.70 13 August 2017 for the 470–660 nm wavelength range (Dobracki et al., 2019). Differences between radiatively derived and in situ measured values for both AAE and SSA may be due to differences in aerosol humidification; the irradiances measured by SSFR and the resulting aerosol properties represent the aerosol in ambient conditions (Pistone et al., 2019). The in situ instruments, however, control the relative humidity while the aerosol is measured, potentially causing discrepancies. Biases may also be present in the in situ absorption that propagates to bias in SSA, due to known issues with measuring absorption on a filter (Pistone et al., 2019). When the aerosol-intensive properties are derived using our new approach, the aerosol optical properties are radiatively consistent with the measured irradiance and the ambient optical thickness, therefore allowing us to establish a more direct estimate of DARE.

Figure 8a shows the TOL radiative effect as a percent of the incoming radiation at 501 nm as a function of the underlying albedo for 20 September 2016 and the northern and southern cases from 13 August 2017. Figure 8b shows example spectra from each case with associated error bars. A positive DARE value indicates that the aerosol warms the layer. For the 20 September 2016 case, the scene albedo, which we consider the average of all the albedo values, is 0.5 at 501 nm, with a corresponding TOL DARE of 9.6±0.9 % (percentage of incoming irradiance). For the 13 August 2017 North case, the scene albedo of 0.03 results in a TOL DARE of -0.61±2.01 %, while the scene albedo of 0.27 for 13 August 2017 South results in a TOL DARE of 5.45±1.92 %. As can be seen in Fig. 8, the DARE from the 20 September 2016 case is larger than the 13 August 2016 cases, in large part due to the higher AOD values in the 20 September 2016 case. At the BOL, DARE is always negative since the amount of radiation reaching that altitude decreases when there is an aerosol layer present due to the scattering and absorption that occurs. For this reason, we do not show the BOL DARE results visually. At the scene albedos listed above, the BOL DARE values at 501 nm are -7.27±0.9 %, -8.36±2.01 %, and -4.37±1.92 %, for 20 September 2016, 13 August 2017 North, and 13 August 2017 South, respectively. For the 2017 cases, the clouds were broken on the northern section and homogeneous on the southern section. The TOL radiative effect crosses from negative to positive with increasing albedo, illustrating that the same aerosol has a warming effect in the south and a cooling effect in the north due to the differences in the underlying cloud. This is similar to the conclusions of Keil and Haywood (2003), Chand et al. (2009), and Meyer et al. (2013), who also find that DARE decreases as the underlying clouds darken, eventually becoming negative. We find that the critical albedo is 0.21 at 501 nm for 20 September 2016 and 0.26 for 13 August 2017. Chand et al. (2009), along with Meyer et al. (2013) and many other studies, choose to normalize the radiative effect by the aerosol optical depth, a quantity known as the radiative forcing efficiency (RFE), to isolate the cloud effect from the aerosol loading on DARE. For this region, Chand et al. (2009) find that the transition point from positive to negative RFE is at the critical cloud fraction of 0.4. Since we are interested in the radiative effects as a function of both the cloud and aerosol properties, we choose not to translate DARE into RFE since it (a) removes the dependence on the aerosol loading and (b) may not linearly scale with mid-visible AOD, with evidence suggesting that the increase depends upon the cloud albedo (Cochrane et al., 2019). We can, however, convert critical albedo into critical cloud fraction and critical optical thickness. For a cloud fraction of 100 % and using the two-stream approximation (Coakley and Chylek, 1975), a critical albedo of 0.21 (0.26) corresponds to a critical optical thickness of 1.5 (1.35). Assuming the mean cloud albedo value of 0.5 used by Chand et al. (2009) (determined on the basis of July–October 5∘×5∘ mean and standard deviation of MODIS-retrieved cloud optical depths), a critical albedo value of 0.21 (0.26) and a g value of 0.56 (0.27) would translate into a critical cloud fraction of 0.42 (0.52). This is consistent with their finding of the critical cloud fraction to be 0.4. Podgorny and Ramanathan (2001), however, find a much lower critical cloud fraction even with a higher SSA. Chand et al. (2009) attribute this discrepancy to differences in cloud albedo, acknowledging that accurate cloud albedo values are crucial in determining aerosol radiative effects. In reality, one cannot simply fix the cloud albedo to a single value; the true albedo, measured from the BOL leg of the radiation wall, at 501 nm for 20 September 2016 ranges from 0.39 to 0.59, while the 13 August 2016 albedo ranges from 0.06 to 0.39. Using critical albedo instead of critical cloud fraction or optical thickness circumvents these problems.

Chand et al. (2009) find that the critical cloud fraction is particularly sensitive to the SSA and is the greatest source of explicitly estimated uncertainty in their study. In our study, the largest uncertainty contributor to the DARE calculation and, similarly, critical albedo, is case dependent, though the SSA represents a significant fraction of the error across all cases for wavelengths of 355–660 nm. This can be seen in Fig. 9, which shows an example of the uncertainty contributions of each input parameter to the DARE calculation for one point from each case. The 20 September 2016 DARE error is dominated by the SSA, while the 13 August 2017 North case is dominated by the g error. The 13 August 2017 South case has nearly equally large contributions from SSA and g. It should be noted that the uncertainty partitioning changes for different points along the radiation wall. Quantifying the individual component uncertainties, especially the albedo uncertainty, is an advancement to satellite-based studies that focus on only quantifying the aerosol parameter uncertainties. The uncertainty due to the underlying clouds in DARE calculations, while known to be important, is often not emphasized or quantified since the cloud albedo cannot be measured directly from space. Despite the differences between previous studies and our work, the results all highlight the importance for accurate optical properties of both the aerosol and underlying cloud layers, since the radiative effect of an aerosol layer so clearly depends on both.