During the ORACLES campaign, HSRL-2 profiles were observed at high-altitude, above-plume flights legs, and CCN concentration was measured in situ at lower altitudes in and around the smoke plume. Each data set has a different spatial and temporal resolution, so collocation in time and space is necessary before correlation analysis can be performed. One important consideration here is the difference in aircraft setup between 2016 and 2017–2018. In September 2016 the ER-2 flew high-altitude legs and carried the HSRL-2 while in situ instruments were located on the P-3, which flew at lower altitudes. However, in August 2017 and September–October 2018 both HSRL-2 and the in situ instruments were deployed on the P-3. In these the deployments, the P-3 often flew at above-plume altitudes to optimize sampling by the HSRL-2 and other remote sensing instruments and later sampled at lower altitudes for in situ observations. Therefore, there is a slightly longer time gap between observations from these instruments in the 2017 and 2018 deployment years.

The result of our data collocation technique is a one-to-one comparison between averaged CCN concentration values and averaged HSRL-2 observations, both observed in approximately the same time and space defined using three independent collocation criteria, as follows. For any given HSRL-2 profile, the collocation method finds CCN concentration measurements that fall within a set amount of time (dt) from when the HSRL-2 profile was measured, within a set horizontal distance (dd) from the profile, and within set vertical bins (dh). Observations that remain after each of these criteria have been applied are then averaged to allow for a one-to-one comparison. A schematic of this process is shown in Fig. 2. Note that while each year uses the same collocation method, 2017 and 2018 have a different aircraft setup that results in a longer time gap between measurements (denoted by t+Δt).

Different collocation criteria values were sensitivity tested to minimize the effects of spatiotemporal variability on the correlation between HSRL-2 observables and in situ CCN concentrations while also selecting a representative portion of the data set to analyze. These sensitivity tests were performed by varying the horizontal, vertical, and temporal criteria one at a time and evaluating their impacts on correlations between CCN–HSRL-2 backscatter and extinction. The final chosen values are given in Table 2 and will be used for all subsequent results. We hold horizontal distance constant at ±1.1 km for all years to approximately correspond to the horizontal distance over which the HSRL-2 products are aggregated and to avoid averaging multiple HSRL-2 profiles in the horizontal. As a result, each average in situ CCN concentration value corresponds to a vertical average of HSRL backscatter and extinction coefficients from a single profile. Vertical bin sizes vary between years due to slight adjustments made to optimize the correlation, and temporal collocation allowances are increased for 2017 and 2018 due to the instruments' deployment on the same aircraft, resulting in a longer time gap between high-altitude remote sensing and low-altitude in situ observations.

Another pre-analysis step is filtering the collocated data set by RH and S thresholds. While one goal of the data filtering step is consistency between each year of the analysis, a few differences in instrument settings and data availability make it necessary to slightly alter the RH and S thresholds between each year. We performed sensitivity tests for different values and ranges of RH and S, and final values are given in Table 2. In general, the sensitivity tests suggest that observations made at high ambient RH tend to weaken the CCN versus backscatter and extinction relationships due to swelling of highly hygroscopic aerosols that causes an increase in backscatter and extinction without a corresponding increase in CCN concentration. Testing of different thresholds suggests that we can mostly avoid this hygroscopic effect by filtering out observations made at RH > 40 %. Due to a limited amount of collocated data points in 2018, caused primarily by remote sensing and in situ observations from a single aircraft, the RH threshold is increased to 50% for 2018 only. A threshold of 40 % resulted in a very small resultant RH range (34 %–39 %) that corresponded only to low CCN concentrations. Therefore, the threshold was increased to 50 % to allow for more data points and a relationship over a range of CCN concentrations more like that for 2016 and 2017. Importantly, since our focus is on CCN within the smoke plume, this constraint still retains most of the data set (Table 2). However, for future analyses focused on boundary layer (relatively high ambient RH) CCN, it will be important to correct lidar measurements made at higher RH to avoid eliminating those CCN concentrations most relevant for aerosol–cloud interaction studies. In terms of S, Köhler theory (Köhler, 1936) suggests that higher S values allow for more aerosols to activate into cloud droplets. Therefore, at lower S values there may be high aerosol concentrations and lidar backscatter and extinction coefficients but decreased in situ CCN concentrations. Since slightly different values and ranges of S were used by the CCN counter each year, the resultant filtering thresholds also vary accordingly.

Another goal of data collocation and filtering steps is to maintain as representative a collocated data set as possible. As shown in Table 2, our RH and S thresholds represent between 52 %–76 % of the total collocated data set. Amidst data limitations, accounting for two different aircraft setups, and working to maximize correlations, these percentages are reasonably representative of the total data set. Another important note in relation to both the location of our collocated data points within the smoke plume and the RH filtering is the inherent exclusion of marine boundary layer (MBL) CCN concentrations. This is in part due to the nature of the SEA environment and the limitation of having few lidar observations below the semi-permanent stratocumulus deck. We thus focus on developing a relationship specifically for HSRL-2 observables and BBA above-cloud within this region. Though below-cloud CCN concentrations are of most interest in terms of aerosol–cloud interaction studies, links have also been found between above-cloud aerosol and cloud microphysical properties (Gupta et al., 2021), validating the need for increased information about above-cloud CCN concentrations as well. Furthermore, CCN concentrations were found to be significantly higher above cloud in the free troposphere than those in the MBL (Redemann et al., 2021), again validating a focus on analyzing above-cloud CCN concentrations using HSRL-2 observables.

The forthcoming analysis develops a regression between HSRL-2 observables and CCN concentration, both of which are observed quantities measured with uncertainty. Therefore, we consider uncertainties associated with both measurements and with the slope of each regression. At the time of this analysis, reported HSRL-2 uncertainties were only available for September 2016. Therefore, we have taken a spatial variability approach to estimate uncertainties for HSRL-2 data from August 2017 and September–October 2018. This method uses backscatter and extinction coefficients in the same vertical bin from five profiles before and five profiles after the HSRL-2 profile associated with each collocated data point. We analyze the distributions of backscatter and extinction across these profiles to ensure no large variations in either coefficient, i.e., that we are accurately estimating instrument uncertainty and not including a large gradient due to aerosol spatial inhomogeneity. After this step we calculate the standard deviation across all 11 profiles to use as a measure of uncertainty.

In Table 3 we present a comparison between HSRL-2 uncertainties calculated using this spatial variability method to the reported HSRL-2 uncertainties available for September 2016. In general, the mean uncertainties from both methods are on the same order of magnitude and very close in value. However, our calculated uncertainties span a wider range than reported uncertainties, suggesting that this method captures a possible upper bound to HSRL-2 uncertainties. While this method only accounts for random uncertainties in backscatter and extinction measurements, systematic uncertainty for backscatter is reported as 5 % for 355 nm and 4.1 % for 532 nm, while extinction is dominated by random error and has a small systematic error (Burton et al., 2015). Given the similar mean uncertainties and possible slight overestimation of HSRL-2 reported uncertainty (rather than consistent underestimation of error) using our spatial variability method, we use these values when considering uncertainty impacting the forthcoming regressions. Furthermore, we present this spatial variability method as a reasonable way to estimate HSRL-2 uncertainties in future studies.

Following the data collocation and filtering, we analyze the correlation between CCN concentration and HSRL-2 observables. Year-by-year analyses were done to determine ideal collocation criteria, as described in Sect. 2.2. However, the primary goal of this study is to develop a relationship between CCN and HSRL-2 backscatter and extinction coefficients. Therefore, we need to show that year-by-year analyses can be combined in a way that maintains the overall strength of the linear relationships. After analyzing year-by-year relationships and finding similar fit line slopes for any given HSRL-2 coefficient and wavelength, we consider all collocated data points in Fig. 3. Data from September 2016, August 2017, and September–October 2018 are combined to fit a relationship between CCN concentration and HSRL-2 backscatter and extinction at 355 and 532 nm. All relationships are fit using a bisector regression to account for both variables being measured with uncertainty. The complete data set is represented by 80 total data points spanning 10 d.

We show the Pearson correlation coefficient (R), Spearman rank correlation coefficient (in parentheses), root mean square error (RMSE), percentage of data within ±10 % of the linear regression line, and relative uncertainty of the slope. In addition, we plot error bars representing relative CCN uncertainty (vertical) and calculated HSRL-2 uncertainties (horizontal). The combination of data from different measurement periods across 3 years results in strong correlations (0.95–0.97) between both variables. This result suggests that our collocation and filtering methodology is reasonable and holds well for multiple observational periods. In addition, different symbols designating different years support the representativeness of the collocated data set, as no one period of observations completely stands out from another. RMSE is on the order of 100 cm-3. With a relative CCN uncertainty of 10 %, RMSE is of the same order of magnitude as a median CCN uncertainty for this data set. The amount of data within ±10 % of each respective linear regression line ranges from 38 %–51 %. These values, together with RMSE, suggest a relatively low amount of scatter around the regression line for both wavelengths and coefficients. Using Eq. (3), 3EXT=QeNπr2, where Qe is extinction efficiency (approximated as 2), N is number concentration, r is radius, and EXT/N is found using the regression slopes in Fig. 3; an approximate CCN radius is calculated to be about 0.15 µm, with an effective cross section of around 0.1 µm. These values are on the order of what is physically expected for smoke aerosols.

Another parameter that we explore in relation to CCN concentration is AI, calculated using HSRL-2 extinction coefficients as described in Sect. 2. Results found using AI for the 3-year combined data set are shown in Fig. 4. Overall, the general trends found in these results match those seen in Fig. 3 for the CCN concentration versus HSRL-2 correlation. Again, these relationships hold for the 3-year combined data set without any one year appearing as an outlier. We again show the Pearson correlation coefficient (R), Spearman rank correlation coefficient (in parentheses), root mean square error (RMSE), and amount of data within ±10 % of the linear regression line. AI calculated using extinction at 532 nm relates slightly more strongly with CCN concentration than does AI calculated using the 355 nm coefficient. This difference could be due to slightly higher uncertainty of HSRL-2 extinction at 355 nm than 532 nm (Burton et al., 2018). RMSE is slightly higher than in Fig. 3, and the amount of data within ±10 % of the linear regression line ranges from 25 %–29 %, indicating slightly more scatter than when using HSRL-2 backscatter and extinction coefficients alone. This could, in part, be due to the very small range of AI values present in the comparison. Nevertheless, the relationships given in Figs. 3 and 4 are comparable and serve as good indicators for representing CCN concentration.

The singular difference between the extinction coefficient and AI is multiplication by the Ångström exponent, an indicator of particle size. There is almost no difference found between how well extinction and AI relate to CCN concentration. The fundamental similarities in both relationships suggest that there is very little variation in the range of Ångström exponents that multiply the aerosol extinction coefficient. Therefore, there is likely only small variation in the size of aerosol being measured. Since observations included in this analysis focus on those made in the smoke plume, this result is reasonable. We are not performing this analysis for a variety of aerosol types and different sizes, but instead we focus on the small range of BBA properties within the smoke plume.

Overall, data collocation and filtering methods suggest that for time intervals between ±6–18 min, horizontal distance separations within about 2 km, and low ambient RH, CCN concentration relates very strongly with lidar observables, with Pearson correlation coefficients of 0.95 to 0.97. The relationship between in situ CCN concentrations and HSRL-2 backscatter coefficients is slightly stronger than that using extinction coefficients, which could be due to higher uncertainties associated with the extinction coefficient (Burton et al., 2016). Nevertheless, we find that both backscatter and extinction are positively and linearly correlated with in situ CCN concentrations. Resultant regression equations allow us to use lidar observables alone to estimate CCN concentrations for this aerosol type, as further illustrated in Sect. 4.3.

We also considered AI, as calculated from HSRL-2 extinction coefficients, with the expectation that an aerosol property that implicitly contains information on aerosol size may correlate better with CCN concentrations than an optical coefficient alone. However, relationships between in situ CCN concentration and AI were similar to those found using HSRL-2 coefficients alone. Hence, we do not find in this study that AI is a better parameter to represent CCN concentration than other optical properties as other studies have suggested (Liu and Li, 2014; Rosenfeld et al., 2014; Stier, 2016). Rather, we find relationships between CCN and AI are nearly identical to those of the HSRL-2 observables. This finding suggests minimal variation in the Ångström exponent and implies that our analysis focuses on a small size range of aerosol observed within the smoke plume.

We hypothesized that HSRL-2 observables at 355 nm would be more strongly related to observed CCN concentrations due to a smaller wavelength interacting with smaller aerosols. However, this was largely not the case. Many of our year-by-year analyses resulted in a slightly stronger relationship between CCN concentration and HSRL-2 backscatter and extinction at 532 nm. For looking at the 3-year combined data set, both wavelengths result in an identical correlation coefficient (Fig. 3). This discrepancy will be explored in future analyses using theoretical calculations of optical properties from Mie theory and CCN concentrations derived from κ–Köhler theory (Petters and Kreidenweis, 2007). However, results from Burton et al. (2016) in relation to the information content and sensitivity of a 3β+2α lidar such as HSRL-2 suggest a very minor difference in the sensitivity of each wavelength to aerosols with small radii, such as CCN. Therefore, our preliminary literature search into this discrepancy suggests that it may be reasonable to find little variability in the strength of the relationships between lidar observables at both wavelengths and observed CCN concentrations.

In Fig. 3, relationships between in situ CCN concentration and HSRL-2 backscatter and extinction coefficients were shown for the 3-year combined data set. While the strength of these relationships for the multi-year data set suggests that we can use this method to analyze BBA found in the ORACLES data, it is important also to recognize inherent environmental differences for each of these years. First, for each of the 3 years of this campaign, observations were made during different parts of the seasonal biomass burning cycle that occurs between June and October (Redemann et al., 2021). This difference primarily impacts the age of aerosols observed during each year, therefore potentially affecting CCN activity. Secondly, observations were made in different regions of the SEA each year. While we have shown that these differences do not appear to significantly hinder the construction of a 3-year combined data set for BBA, we will examine these differences in aerosol age and location of measurement to understand the extent of data considered when all 3 years are analyzed together.

One commonly occurring trajectory for BBA in this region suggests that the plume core is lofted over the SEA between 5–15∘ S, whereby the aerosols move westward, following a spiral path, and later descend into the boundary layer near 20∘ S and close to the coast (Adebiyi and Zuidema, 2016; Redemann et al., 2021; Ryoo et al., 2021). Though the exact trajectory changes based on month within the seasonal burning cycle, the overall pattern implies that aerosols measured farther north within the SEA domain are likely younger, while aerosols observed farther west and south tend to have traveled longer along the spiral pathway and are older aerosols. Comparing this conceptual model with the approximate locations of our collocated data points (Fig. 5) suggests that observations from 2017 and 2018 that lie farther north and west within the domain were likely measured closer to the time they were ejected from the smoke plume. In contrast, observations from 2016 lie farther south and closer to the coast, suggesting that they may be older. This conceptual model can be evaluated using measurements of f44. Higher values indicate increased amounts of carboxylic acids and imply that measured aerosols are relatively old. Using 3-year combined CCN concentration versus extinction at 355 nm as an example, we color-code data by values of f44 (Fig. 6). In general, 2017 and 2018 have lower f44 values ranging from 0.17–0.21, indicating slightly younger aerosols. Data from 2016 have a wider range of f44 values between 0.2–0.27, indicating some ages close to those from 2017 and 2018 and some that are older. The f44 data suggest that determinations of aerosol age may not always be as straightforward as depicted by the expected trajectory pathway (Fig. 5). Furthermore, though a complete f44 data set is not available for every point in our collocated data set, the values that are available suggest that we consider a wide range of aerosol ages in this analysis with no singular age breaking down the strength of the regression (Fig. 6).

Lastly, we look at observations from ORACLES in the context of other regions and aerosol types to explore the representativeness of our collocated data set. Specifically, we compare our results to those from Shinozuka et al. (2015), a study focusing on similar relationships between CCN concentration and in situ aerosol extinction coefficient of dried particles. To compare our results most accurately with those from Shinozuka et al. (2015), we use a campaign-average value for CCN concentration and in situ dry extinction coefficient at 500 nm, as calculated from in situ-measured particle scattering and absorption coefficients. These observations were made on the same instrument rack as CCN concentration and represent the extinction coefficient for dried particles pumped into aircraft instrumentation and unaffected by ambient RH, unlike observations made by HSRL-2. This comparison is given in Fig. 7, which is modified from Fig. 6 in Shinozuka et al. (2015), with results from the current study added and labeled as “Southeast Atlantic”. We overlay the campaign-average CCN concentration and 500 nm dry extinction coefficient as a scatter point, without altering the original regression lines. Campaign-average values were calculated using the entirety of the ORACLES in situ dry extinction and CCN concentration data sets (not limited by our collocation technique). The data were only filtered to exclude low CCN concentrations (CCN < 100 cm-3) collocated with low extinction coefficients (in situ extinction < 0.02 km-1) characteristic of a gap region and to match the S range (0.3 %–0.5 %) from Shinozuka et al. (2015). This filtering was performed to provide as close of a comparison as possible. Horizontal and vertical bars for each point represent the standard deviation. Overall, we find that our campaign-average CCN and in situ dry extinction values, specifically for the BBA-dominated SEA region, compare well with the relationships given for other regions and aerosol types. This comparison serves as another source of validation for the methodology of this study and suggests that performing a similar CCN concentration versus HSRL-2 observable relationship analysis for other regions and campaigns with different dominant aerosol types would be feasible.

Therefore, while the observations used in this study do not span the entire SEA, we have reason to believe that they account for different parts and ages of the smoke plume as it is advected westward. Since the in situ and lidar observations can be combined in a 3-year data set that maintains a strong linear relationship between CCN concentrations and HSRL-2 observables, we believe that these results are representative of BBA in the SEA.

As previously mentioned and taken into consideration via bisector regression, both CCN and HSRL-2 are observations made with uncertainty. Relative CCN uncertainty is 10 %, and our spatial variability method of calculating HSRL-2 uncertainties resulted in mean values of 1.4×10-4 and 7.0×10-5 km-1 sr-1 for backscatter at 355 and 532 nm, respectively, and 0.0076 and 0.0064 km-1 for extinction at 355 and 532 nm, respectively (Table 3). In addition, uncertainty is introduced as a result of the regression itself. Relative slope uncertainties range from 3.0 %–3.6 % (Fig. 3). We discuss each of these sources of error separately due to their dependence on one another. Uncertainties in both CCN concentration and HSRL-2 observations will impact uncertainty in the slope of each regression. Therefore, the assumption of each source of error being independent that is required for error propagation calculations does not hold. Rather, we present our method of deriving CCN concentration from lidar observables with such explanation of the various sources of error that will impact results.

In addition to observational and regression-based uncertainties, another possible source of error when applying this method stems from the specific characteristics of the data set used to develop the regression equations. The relationships analyzed in this study are specific to BBA in the SEA. Additionally, they are specific to ambient conditions with low RH (≤40 %–50 %), S≥0.2 %, and aerosol ages represented by f44 values between about 0.17–0.27. While these conditions are characteristic of the high-altitude SEA smoke plume, they will not hold in all regions and for all aerosol types. Therefore, without careful consideration of the ambient conditions and aerosol types to which the regressions derived here are applied, increased uncertainty will be introduced in lidar-derived CCN concentrations. Despite the strict conditions under which our regressions are applicable, we will explore their performance on a larger portion of the collocated data set in the following section.

In addition to showing the representativeness of our lidar-derived CCN method in the context of additional aerosol types and data sets, we expand our data collocation criteria to analyze the applicability of our regression equations to a larger subset of the data collected in ORACLES. In Fig. 9, the relationship between lidar-derived CCN concentration (using the regression equation for backscatter at 532 nm from Fig. 3) and in situ CCN concentration is shown. This expanded collocated data set of 460 points is attained by increasing the horizontal distance criterion to ±4.4 km for each year and leaving vertical bin size and time criteria the same (Table 2). Our sensitivity testing revealed that using a larger horizontal distance results in more scatter and noise within the in situ CCN–HSRL-2 observable linear relationship, so while this value was not used to derive the regression equations, it is used here solely to test our regression method on a larger subset of the data across September 2016, August 2017, and September/October 2018. A correlation coefficient of 0.85 for this expanded data set shows the general applicability of our regression equations to a larger data set, demonstrating that our regression equations are not over-fitted to the smaller original set of collocated data points. Comparing this to a correlation coefficient of 0.97 for the original data set also supports our original decision to limit the collocation criteria to relatively small spatiotemporal extents when developing the regressions to avoid the effects of larger spatial variability in CCN concentrations. Therefore, while our original data collocation criteria served to optimize the strength of these linear relationships, the expanded criteria are used to show the applicability of the method to a larger portion of the data.

Additionally, a primary goal of this work is the development of a method to derive CCN concentrations using HSRL-2 observations alone and particularly to increase knowledge of the vertical distribution and variation of CCN. Below, we demonstrate this ability for one ORACLES flight leg, where we derive CCN concentrations for an entire HSRL-2 curtain using the regression equation derived from the 3-year combined data set of in situ CCN concentration versus backscatter at 532 nm developed using a supersaturation range of 0.22 %–0.4 % (Fig. 3). Figure 9a shows calculated CCN concentrations from an above-cloud 15 August 2017 flight leg. Figure 9b shows CCN concentrations for this same flight leg simulated by simulated by WRF-CAM5 (Shinozuka et al., 2020) at 0.5 % supersaturation. Some features of our derived CCN concentrations agree reasonably well with the corresponding WRF-CAM5 output. However, our lidar-derived method results in a better estimate of smoke plume depth and altitude, cloud top heights, and small-scale variation of CCN concentrations. This is likely due to our lidar-derived method having a higher horizontal resolution (2 km from HSRL-2 compared to 36 km from WRF-CAM5), as well as the depiction of possible turbulence and entrainment effects that are not currently captured by the model. Other model–observation comparison studies have shown that models such as WRF-CAM5 (and others) also tend to underestimate the altitude of the smoke plume (Shinozuka et al., 2020; Doherty et al., 2022).

One potential reason for differences in the magnitude of CCN concentrations between WRF-CAM5 and our lidar-derived method includes the tendency of WRF-CAM5 and other models to represent the plume as more vertically diffuse than is generally observed (Doherty et al., 2022). This overly diffuse representation of the smoke plume can also result in an overestimation of aerosols lying above the stratocumulus deck and that get mixed into clouds, further promoting an overestimate of CCN concentration (Doherty et al., 2022). Other factors causing differences in magnitude of CCN concentration and smoke plume placement between our lidar-derived CCN concentrations and WRF-CAM5 CCN concentrations are outside the scope of this study but remain an area of future research. Rather, we use this example flight path as an illustration of a way that the lidar-derived CCN can be used to evaluate and potentially improve model performance moving forward. In addition to model performance of CCN concentration in the SEA, this same methodology of using collocated data to develop linear regressions between in situ CCN concentrations and HSRL-2 observables can be applied in the future to allow for lidar-derived CCN concentrations specific to different aerosol types and locations.