The CALIOP V3 aerosol classification algorithm uses altitude, location, surface type, estimated particulate depolarization ratio (δpest), and integrated attenuated backscatter (γ′) to identify the aerosol subtype (Omar et al., 2009). Figure 1 shows the decision tree used to determine the V3 and V4 tropospheric aerosol subtypes. The major updates implemented in the V4 tropospheric aerosol subtyping algorithm include introducing the dusty marine aerosol subtype (by adding the red-shaded region in Fig. 1), allowing all aerosol subtypes over polar regions (by removing the blue-shaded region in Fig. 1), and revising the operational definitions for the polluted continental and smoke aerosol types.

At this time the integrated attenuated color ratio (χ′=γ1064′/γ532′) is not used for aerosol subtyping in the troposphere because the low signal-to-noise ratio (SNR) for optically thin layers, especially in the daytime, makes it an inconsistent discriminator among tropospheric aerosol types. However, it is useful for stratospheric aerosol typing in which the number of types is fewer (Sect. 2.2).

In V4, the CAD algorithm is applied at all altitudes, including in the stratosphere. By contrast, previous versions only applied the CAD algorithm below the tropopause, classifying layers detected above the tropopause as stratospheric features rather than as clouds or aerosols. As a consequence, aerosol existing above the tropopause was not identified explicitly as aerosol. However, it is well documented that certain aerosol types exist in the stratosphere. Volcanic eruptions inject ash and sulfate to high altitudes (e.g., Vernier et al., 2011; Bourassa et al., 2012). Smoke due to intense combustion or from pyro-cumulonimbus events can also breach the tropopause (e.g., Fromm et al., 2005, 2010; Trentmann et al., 2006). In the polar winter, polar stratospheric clouds (PSCs) form and the PSC composed of supercooled ternary solution (STS) is an aerosol (Pitts et al., 2009). In V4, features identified by the CAD algorithm as aerosol having 532 nm attenuated backscatter centroids (Garnier et al., 2015) above the tropopause from the Modern-Era Retrospective analysis for Research and Applications version 2 (MERRA-2) reanalysis data (Gelaro et al., 2017) outside of the polar regions are classified as “stratospheric aerosols”. Distinguishing among the different types of stratospheric aerosol relies primarily on latitude, temperature, and the measured properties of each layer: γ′ and δpest at 532 nm and χ′.

V4 identifies four stratospheric aerosol subtypes: volcanic ash, sulfate/other, elevated smoke, and polar stratospheric aerosol (PSA). Volcanic ash is defined as an aspherical volcanic aerosol that depolarizes the 532 nm backscatter, whereas sulfate/other is defined primarily as a non-depolarizing volcanic aerosol. The “other” component of this aerosol type is the catchall for stratospheric aerosol layers that are either weakly scattering or cannot be classified as any other type within the stratospheric aerosol algorithm. Weakly scattering layers are not evaluated by the stratospheric aerosol subtyping algorithm because the noisy values of δpest and χ′ at low signal levels inhibit robust classifications with the threshold-based technique employed. The PSA subtype is introduced in V4 to assign a reasonable aerosol type for features detected in the polar regions during polar winter and subsequently classified as aerosol by the CAD algorithm due to their low values of χ′ and γ′. Comparison with the CALIOP L2 PSC mask product shows that PSAs are spatially correlated with the STS PSC composition class (Pitts et al., 2009; Liu et al., 2018). However, layers assigned the PSA subtype should be interpreted carefully. For in-depth studies, the CALIPSO team recommends using the CALIOP L2 PSC mask product for analyses related to PSC composition since it is a more specialized product (Pitts et al., 2009).

The stratospheric aerosol subtyping algorithm is summarized by the flowchart in Fig. 4. PSAs are identified first by determining if the stratospheric aerosol layer's latitude, season, and temperature at its 532 nm attenuated backscatter centroid altitude are consistent with that of PSCs. A centroid altitude temperature threshold of -70 ∘C is implemented based on the occurrence frequency distribution of aerosol mid-layer temperatures during the Arctic and Antarctic PSC season in 2008 (Fig. 5). The -70 ∘C temperature threshold captures the increased aerosol occurrence frequency at colder temperatures, corresponding to the possible formation of STS PSCs (e.g., temperatures of less than -75 ∘C correspond to temperatures consistent with PSC formation in this region; Rosen et al., 1997). Latitude and seasonal constraints are applied to ensure the PSA type is assigned where and when PSC formation occurs. PSA classification is only allowed for latitudes poleward of 50∘ S or N, and PSC seasons for the Arctic and Antarctic regions are assumed to be December–February and May–October, respectively (Poole and Pitts, 1994).

Next, in order to discriminate between volcanic ash, sulfate, and elevated smoke, the stratospheric aerosol typing algorithm evaluates layer-averaged δpest and χ′ against empirically derived thresholds. These thresholds were derived from frequency distribution analysis of δpest and χ′ measurements obtained from a manually identified subset of volcanic ash, sulfate, and high-altitude smoke layers (Fig. 6). The number of unique layers detected by CALIOP, geophysical events, and dominant aerosol types contributing to this subset are summarized in Table 1. Note that, as previously mentioned, weakly scattering stratospheric aerosol layers (layers with γ′<0.001 sr-1) are directly classified as sulfate/other due to their low SNR. As shown in Fig. 6a–c, volcanic ash and volcanic sulfate are fairly well separated with respect to δpest, with ash typically having δpest>0.15 and sulfate with 0.075<δpest<0.15. Smoke layers are less depolarizing than volcanic ash (Fig. 6d), with δpest<0.15. However, smoke layers can be either non-depolarizing or moderately depolarizing (Fig. 6e–f). An example of a moderately depolarizing smoke event is the February 2009 “Black Saturday” Australian bush fire for which δpest frequently exceeded 0.10. Non-depolarizing smoke layers (δpest<0.075) typically have χ′>0.5, whereas χ′ is more frequently lower for moderately depolarizing smoke layers (0.075<δpest<0.15). Based on this analysis, the stratospheric aerosol typing algorithm depicted in Fig. 4 was constructed using the thresholds indicated by the red lines in Fig. 6.

The following examples demonstrate the strengths and limitations of the stratospheric aerosol typing algorithm. Volcanic ash is well separated from the other types in terms of δpest and χ′, which often leads to robust subtype classifications. Figure 7 shows a scene in which the algorithm correctly classifies the bulk of a volcanic ash plume from the Puyehue-Cordón Caulle eruption in June 2011. Though the northernmost layers near 42–45∘ S are automatically classified as sulfate/other due to their low γ′, these layers are optically thin and the more optically thick layers of the ash plume are classified correctly. Also note that portions of the ash plume having backscatter centroids below the tropopause are misclassified as dust. This occurs because aerosol layers below the tropopause are assigned tropospheric aerosol subtypes that do not include ash. Because the noise-broadened distributions of δpest for ash and dust measured by CALIOP have very similar characteristics, we know of no robust way to discriminate the two within the troposphere (Winker et al., 2012). However, the lidar ratio assigned for ash is identical to that for dust (Table 2) so potential misclassifications will have minimal impact on the extinction products.

Figure 8 presents a scene in which the bulk of the Nabro volcano plume from June 2011 (Theys et al., 2013) is correctly classified as sulfate. In this example, most of the layer has γ′>0.001 and low depolarization, yielding a sulfate classification for the more optically thick segments. However, a small number of layers within the plume are misclassified as smoke. This is expected because of the overlap in the frequency distributions of δpest and χ′ for sulfate (Fig. 6b) and smoke (Fig. 6d). The optical properties used for these two types are not as well separated from each other as they are from volcanic ash, so some misclassifications can occur. Additionally, volcanic sulfate within the troposphere will be assigned a tropospheric aerosol type, usually elevated smoke or clean continental if weakly scattering. The last example in Fig. 9 shows an observation of a depolarizing smoke plume from the Black Saturday Australian bush fire in February 2009 (Pumphrey et al., 2011; de Laat et al., 2012). The majority of the plume above the tropopause is correctly classified as smoke, with the minority misclassified as ash due to δpest exceeding 0.15. The remainder of the plume below the tropopause is misclassified as dust and polluted dust, again due to elevated depolarization. In all of these examples, the V3 data products classified the layers detected above the tropopause as a generic stratospheric layer without applying any further subtyping. In V4, these same layers are most often correctly classified as aerosols by the new CAD algorithm (Liu et al., 2018). Similarly, the new stratospheric aerosol subtyping algorithm is largely successful in identifying the correct aerosol subtype.

In previous data releases, “fringes” at the bases of dense aerosol plumes were at times misclassified as an aerosol subtype inconsistent with the parent plume. These fringes typically lie below rapidly attenuating aerosol layers and are detected at the coarser horizontal resolutions (20 and 80 km) employed by CALIOP's iterated, multi-resolution layer detection scheme (Vaughan et al., 2009). An example is shown in Fig. 10a–b, in which fringes at the base of an elevated smoke plume are misclassified as clean marine aerosol. In this case, the fringes are misclassified because the layers are non-depolarizing and have top altitudes just below the 2.5 km altitude threshold that would have otherwise caused them to be correctly classified as elevated smoke according to the revised definition of elevated in V4 (Sect. 2.1.3). Given that this is an elevated plume not in contact with any aerosol beneath, it is reasonable to expect that the misclassified fringes at the base of the plume have the same aerosol subtype as the adjacent smoke layers. This same argument can be made for other aerosol types that contiguously span large horizontal distances (e.g., dust plumes, marine aerosol, volcanic ash, and volcanic sulfate).

In general, aerosol layers detected by CALIOP that are in contact with other aerosol layers are assumed to be of the same aerosol type. For this reason, V4 implements the Subtype Coalescence Algorithm for AeRosol Fringes (SCAARF), which reclassifies the aerosol subtype of these lower fringes to match the dominant subtype of the adjacent overlying layers. According to SCAARF, fringes are defined as aerosol layers detected at 20 or 80 km horizontal resolutions that are vertically adjacent to the base(s) of aerosol layers detected at finer spatial resolution (i.e., they are adjacent to more strongly scattering features). At least 50 % of the horizontal extent of the fringe candidate must be in contact with aerosol overhead. SCAARF is applied to all tropospheric and stratospheric aerosol layers meeting this fringe criteria. The dominant adjacent aerosol subtype is determined from the number of 5 km resolution samples vertically adjacent to the fringe. When two adjacent aerosol subtypes exist with equal frequency (i.e., neither is dominant in terms of number), the fringe is changed to match the subtype of the adjacent layers that are most similar to the fringe in terms of δpest and χ′. This is the subtype with the minimum Euclidian distance ri between δ¯pest and χ¯′ of the parent and δpest and χ′ of the fringe; i.e., ri=δp,fringeest-δ¯p,subtypeiest2+χ′fringe-χ¯′subtypei2,fori∈[1,2]. Here, δ¯p,subtypeiest and χ¯′subtypei are the average values of δpest and χ′ for all adjacent layers having unique subtypes i∈[1,2]. The fringe is changed to match subtype i where ri=minimumr1,r2. If three or more unique subtypes are adjacent to the fringe with equal frequency, SCAARF is not applied.

In effect, SCAARF aids in vertically homogenizing aerosol subtype classification along plume bases. Figure 10c demonstrates that the fringes misclassified as clean marine have been correctly classified as elevated smoke after SCAARF is implemented. Similar improvement occurs for volcanic ash layers straddling the tropopause. Lower fringes of these plumes below the tropopause would otherwise be misclassified as dust (Sect. 2.2), yet SCAARF helps retain the volcanic ash classification.

Table 2 shows the lidar ratios that characterized the V3 aerosol types and the revised values used in V4 for tropospheric aerosols and the newly introduced stratospheric aerosols. Except for polluted continental and polluted dust, V4 aerosol lidar ratios have been updated to reflect the improved knowledge from measurements reported in recent literature. In addition, lidar ratios have been defined for the new aerosol types: dusty marine and the stratospheric aerosol types. The modifications and new lidar ratios are based on the latest available measurements, from both CALIPSO and other researchers and field measurement campaigns.

The lidar ratio and uncertainties for clean marine aerosol are modified from 20±6 sr in V3 to 23±5 sr at 532 nm in V4. This change is consistent with results reported from numerous field campaigns since the launch of CALIPSO. Papagiannopoulos et al. (2016) report a lidar ratio of 23±3 sr for marine aerosol from the European Aerosol Research Lidar Network (EARLINET), based mainly on multiwavelength Raman lidar systems. Müller et al. (2007) also show that the mean lidar ratio for marine aerosols in the PBL is 23±3 sr from the Second Aerosol Characterization Experiment (ACE 2) and 23±5 sr from the Indian Ocean Experiment (INDOEX). In an analysis of collocated HSRL measurements acquired during dedicated CALIPSO underflights, Rogers et al. (2014) find that the median lidar ratio for layers identified as marine aerosol is 23 sr. Haarig et al. (2017b) observed lidar ratios for marine aerosols as a function of relative humidity with a multiwavelength polarization Raman lidar and found that the 532 nm lidar ratios increased from 23 sr for spherical sea salt particles to 25 sr for cubic-like particle ensembles.

With respect to marine lidar ratios at 1064 nm, Josset et al. (2012) applied the Synergized Optical Depth of Aerosols (SODA) technique (Josset et al., 2011) to CALIOP measurements at both 532 and 1064 nm and found no spectral dependence in the retrieved lidar ratios. Similarly, Sayer et al. (2012) calculate lidar ratios for marine aerosols from AERONET island sites, spread throughout the world's oceans, and find little spectral dependence. Based on these studies, the CALIOP lidar ratio for clean marine at 1064 nm is changed from 45±23 sr in previous versions to 23±5 sr, which is same value used at 532 nm in V4.

Feature type changes between V3 and V4 are predominantly due to extensive changes in the calibration coefficients reported in the CALIOP level 1 product, which in turn required major revisions of the PDFs that drive the CAD algorithm (Liu et al., 2018). Changes to the surface detection algorithm (Vaughan et al., 2018b) also contribute, but to a significantly lesser extent. In order to quantify how the occurrence frequency of aerosol types has changed, Table 4 reports the percent changes in “feature type” and “aerosol subtype” between V3 and V4 for all 60 m range bins in the CALIOP level 2 profile product from 2007 to 2009. Salient statistics drawn from Table 4 are given here and in subsequent sections. Within the 3-year analysis period, the classification of 13.7 % of the layers identified as tropospheric aerosols in V3 has changed to totally attenuated (0.1 %), clear air (6.3 %), cloud (7.0 %), or surface (0.4 %) in V4. In spite of this reduction, however, tropospheric aerosols in V4 increase by 18 % due to newly identified aerosols from regions that were identified in V3 as totally attenuated layer, clear air, cloud, or surface. The V3 CAD algorithm did not separate aerosols from clouds for layers detected in the stratosphere; instead, it identified these layers generically as stratospheric features. In V4, however, the CAD algorithm is applied in both the troposphere and stratosphere, and thus aerosol layers are identified and classified in the stratosphere. When including newly introduced stratospheric aerosol layers (13 %), aerosols increase by 31 % in V4 compared to V3. The CALIOP V4 algorithm detects more aerosol, resulting in an increase in AOD. Additionally, the improved surface detection scheme of V4 results in fewer opaque layers than in V3 (Vaughan et al., 2018b). Thus, regions below layers previously classified as opaque are now scanned for the presence of atmospheric features. This leads to an increase in the number of aerosol layers reported near the surface.

The spatial distribution and frequency of occurrence of aerosols has changed from V3 to V4 for reasons described in Sect. 3.1. Similarly, enhancements to the aerosol subtyping algorithm described in Sect. 2 are responsible for changes in the spatial distributions and occurrence frequencies of the different aerosol subtypes. The net effect of these changes is demonstrated by Fig. 11, which shows the difference in aerosol subtype detection frequencies for JJA 2007, day and night combined. For context, Fig. 12 shows the number of aerosol samples detected during the same time period. The frequency of clean marine aerosol is slightly reduced in V4 except for in the oceans around Antarctica (Fig. 11a), with most changed layers becoming dusty marine. Table 4 shows that 3.9 % of V3 clean marine aerosol is reclassified as dusty marine in V4. The increase in clean marine aerosol in V4 over the Antarctic Ocean mainly comes from clean continental and polluted continental aerosols due to the changes in aerosol subtyping algorithm over the polar regions (Fig. 1). Approximately 4.1 % of clean marine aerosol off the southwest African coast became elevated smoke in 2007–2009 (Table 4).

The revised definition for elevated smoke (Sect. 2.1.3) and the implementation of SCAARF (Sect. 2.3) are responsible for correcting the frequency of elevated smoke classifications in this region in V4 (Fig. 11f). Additionally, the revised elevated smoke definition is responsible for the changes in polluted continental/smoke and elevated smoke classifications over southern Africa in Fig. 11c and f, respectively. During JJA, smoke from biomass burning is ubiquitous in this region, so a smoke aerosol subtype classification is expected most often. Because the top altitudes of smoke layers within this region are often below 2.5 km above the ground level, many layers do not meet the V4 elevated definition, causing an increase in the frequency of polluted continental/smoke classifications and a reduction in the frequency of elevated smoke classifications compared to V3.

As expected and intended, the introduction of the new dusty marine aerosol type (Sect. 2.1.2) has improved the aerosol subtyping over oceans in regions where mixtures of dust and urban pollution are not expected (e.g., the mid-Atlantic and mid-Pacific oceans). This is shown by a decreased frequency of polluted dust aerosol layers in V4 (Fig. 11e). Over the north African–Arabian dust region, the frequency of polluted dust has increased while the frequency of dust has decreased (Fig. 11b), in part due to correcting the overestimate of δpest that existed in V3. This correction in δpest also caused about 4.9 % of V3 dust to become dusty marine in V4 (Table 4). Though the fraction of aerosol classified as dust has changed by a small amount, the number of dust layers at high altitudes has increased due to changes in CAD, which shows an improved ability to correctly classify lofted dust layers as aerosols rather than cirrus clouds (Liu et al., 2018).

The frequency of clean continental aerosol has decreased over regions characterized by snow, ice, and tundra (Fig. 11d) because all aerosol type classifications are allowed in V4 over these surface types (Sect. 2.1.1). Clean continental aerosols have mainly changed to clean marine (11.6 %), polluted dust (11.3 %), elevated smoke (10.2 %), and polluted continental/smoke (9.7 %). The increase in dust and polluted dust classifications over the Antarctic reflect CAD misclassifications of tenuous ice clouds and blowing snow. Only 8.8 % of clean continental aerosol layers are unchanged in V4 (Table 4).

As a global summary, Fig. 13 shows frequency distributions of aerosol subtypes for daytime and nighttime in V3 and V4, normalized by the total number of bins (day and night together for each version) that were classified as aerosol according to the AVD data from the level 2 aerosol profile products. More aerosol layers are detected at night for both V3 and V4 (Liu et al., 2018). This is expected since a higher SNR at night means the CALIOP layer detection algorithm detects more weakly scattering features during nighttime (Vaughan et al., 2009). Clean continental is only rarely identified in V4. Clean continental was common in the polar regions, especially over the Antarctic in V3. Because V4 allows all aerosol types in the poles, the dominance of clean continental is significantly reduced, as shown in Fig. 11d. The frequency of polluted dust is reduced for both day and night. While part of this reduction is due to the layer attenuation corrections mentioned in Sect. 2.1.1, the predominant reason is because layers previously classified as polluted dust are now more realistically classified as dusty marine in V4. Since the frequency of occurrence of polluted dust aerosols is larger for daytime compared to nighttime over ocean in V3, as shown in Fig. 3b, the change from polluted dust to dusty marine is relatively more frequent for daytime than nighttime. In fact, 59 % of the daytime dusty marine in V4 is polluted dust in V3, but only 42 % of the nighttime dusty marine is polluted dust in V3. The generic stratospheric features previously identified in V3 are now classified as clouds or aerosol in V4. During the daytime, these V3 stratospheric features are more frequently identified as clouds, rather than aerosols. At night the situation is reversed: nighttime V3 stratospheric features are most often classified as aerosols.

In order to compute the change in AOD from V3 to V4, CALIOP level 2 aerosol extinction profiles are vertically integrated and compared profile to profile between the two versions. Only profiles in which either V3 or V4 contained aerosol are included in the average. In cases in which only one version reports aerosol, the other version is assumed to have AOD = 0. Furthermore, only aerosol bins with the extinction quality control (QC) flags of 0, 1, 16, and 18 are used (excluding 6 % of aerosol samples), which represent unconstrained retrievals that do not change the lidar ratio (0), constrained retrievals that derive an optimized lidar ratio (1), opaque layers for which the lidar ratio was not changed (16), and opaque layers in which the lidar ratio was reduced to prevent the retrieval from diverging (18), respectively (Tackett et al., 2018). For 2007 to 2009, the total change in mean nighttime (daytime) CALIOP level 2 column AOD has increased from 0.084 (0.090) in V3 to 0.128 (0.126) in V4 (Table 5). Day and night AOD become more comparable in V4, whereas daytime AOD is larger than nighttime AOD in V3. Note that the mean AOD computed here is not meant to represent global conditions but instead examines AOD changes only where AOD is detected by CALIOP.

The AOD increase from V3 to V4 is due to various factors. Using the feature types and aerosol subtypes reported in the level 2 AVD, AOD changes attributed to layer detection, CAD, totally attenuated layers, surface detection, stratospheric aerosol classification, aerosol type, and lidar ratio are identified using the procedure diagrammed in Fig. 14. This strategy isolates changes in AOD due to each of these factors using CALIOP level 2 products from 2007 to 2009. All range bins whose feature type is determined as aerosol by either V3 or V4 are selected for the analyses. If a bin is identified as aerosol in one of V3 or V4 and the other is clear, the corresponding AOD change is regarded as changing due to the difference of layer detection in the two versions (pathway 1 in Fig. 14). Similarly, an aerosol bin that changed from/to cloud, totally attenuated layer, surface, and stratospheric feature is counted in the AOD changes due to the updates of CAD, totally attenuated signals, surface detection, and stratospheric aerosol in V4, respectively (pathways 2–5). When feature types in both V3 and V4 are aerosol, AOD differences can be due to aerosol subtype changes (pathway 6) or lidar ratio adjustments without changing subtype (pathway 7). If aerosol subtype is identified as polluted continental, polluted dust, or smoke in both V3 and V4, there are no changes in the aerosol subtyping (pathway 8). However, the AOD can be different between V3 and V4 even when there are “no changes” in their subtype and lidar ratio. The most likely source of these differences is changes in the magnitude of β′, either due to level 1 calibration improvements (Kar et al., 2018; Getzewich et al., 2018) or to changes in the two-way transmittances estimated for overlying cloud and/or aerosol layers (Young et al., 2018).

Table 6 quantifies the AOD changes from V3 to V4 for all-sky conditions for the different factors categorized in Fig. 14, i.e., layer detection, CAD, surface detection, stratospheric aerosol, aerosol subtype, lidar ratio, and no change. Here, the all-sky analysis includes all profiles that contain identified aerosol regardless of the presence of clouds. All of the factors listed above contribute to the increase in AOD in V4, and the magnitudes of the AOD changes are strongly related to their occurrence frequencies (Table 6).

Global maps of AOD changes for each factor are shown in Fig. 15. CALIOP AOD has increased by 0.007 and 0.005 for nighttime and daytime, respectively, because of changes in layer detection (Table 6 and Fig. 15b). This implies that the CALIOP V4 layer detection algorithm finds tenuous layers that were not found in V3. Note that no significant changes were made to the CALIOP layer detection algorithm in V4. The increased detection of faint layers is attributed primarily to changes in the 532 nm calibration coefficients that generally increase β′. The CAD algorithm classifies most of these new layers as aerosols.

Figure 15c shows that AOD changes due to CAD have a day and night difference. The mean daytime AOD increase is twice the nighttime AOD increase (Table 6). The daytime AOD increase is due primarily to a net increase in the number of V4 aerosols. The number of new aerosols in V4 (i.e., layers that were classified as clouds in V3) is much larger (∼3.4 times) than the converse (i.e., the aerosols in V3 that were classified as clouds in V4) at daytime. Among these new aerosols, the subtypes dust and polluted dust account for more than 50 %. The V4 CAD PDFs were deliberately tuned to be more sensitive to aerosol presence in the upper troposphere and lower stratosphere, resulting in improved performance in distinguishing high-altitude Asian dust plumes from cirrus (Liu et al., 2018). However, as a side effect, some fraction of the cirrus fringes detected along the edges and lower boundaries of cirrus clouds that were classified as clouds in V3 are now classified as aerosols in V4. Most of these misclassified fringes are subsequently identified as dust or polluted dust by the aerosol subtyping algorithm. These increases in misclassified dust and polluted dust at high altitudes contribute the most to the daytime AOD increase due to CAD. Although the misclassification of cirrus fringes as aerosols also occurs in the V4 nighttime aerosol products, the gain and loss in the total aerosol number due to CAD are about the same and hence the change in nighttime aerosols cannot fully explain the increase in the nighttime mean AOD increase in Table 6. It appears that the change in V4 level 1 data calibration also plays an important role. As discussed below for Fig. 15i, changes in the V4 level 1 data calibration alone can cause a nighttime AOD increase of 0.003, as shown in Table 6, about half the nighttime AOD change of 0.007 due to CAD. However, the net change for each aerosol subtype varies largely and may play an important role in the regional AOD changes as seen in the left panel of Fig. 15c.

If a bin is previously identified as totally attenuated in V3 and becomes aerosol in V4, the most likely cause is difference in surface detection. The V4 surface detection algorithm (Vaughan et al., 2018b) is considerably more effective than the V3 algorithm in detecting the Earth's surface after penetrating atmospheric layers of substantial optical depth (e.g., cirrus clouds with optical depths of 2.5 or larger). As a result, many regions where the signal was considered totally attenuated in V3 are now searched for the presence of features, and the aerosol layers detected in these regions contribute to an increase in AOD in V4 (Fig. 15d).

Changes in the surface detection and the newly introduced stratospheric aerosol types also contribute to AOD increases in V4, but not significantly. Figure 15e shows that some surface signals misclassified as aerosols in V3 are now classified as surface in V4. This leads to a decrease in AOD especially in the southern Asian continent. Since the surface returns are much stronger than backscatter signals for aerosols, AOD changes appear relatively large. AOD increases due to newly introduced stratospheric aerosols in V4 are found mainly in the Arctic and Antarctic regions where the STS PSCs are ubiquitous in the polar winter and are most often classified as stratospheric aerosol in V4 (Fig. 15f).

There is a decrease in the mean AOD due to aerosol subtype changes for daytime, but an increase for nighttime (Table 6). Figure 15g shows that AOD changes due to the aerosol type changes generally have opposite signs at day and night over oceans. The dominant aerosol type over oceans is clean marine, which has the smallest lidar ratio among the CALIOP aerosol models. Therefore, any changes from clean marine to other types of aerosol can lead to an AOD increase in V4. This is the dominant type change over ocean for the nighttime. For the daytime, however, a type change from polluted dust to dusty marine occurs more frequently, as explained earlier (Sect. 3.2). The reduction in lidar ratio from polluted dust (55 sr) to dusty marine (37 sr) leads to a decrease in the mean daytime AOD. AODs decrease for both day and night over the mid-Atlantic and Indian oceans as well as over the Arctic and Antarctic oceans, as shown in Fig. 15g. Dust is frequently transported to the Atlantic and Indian oceans from the Saharan and Arabian deserts. Aerosol type changes from dust or polluted dust to dusty marine dominate in these regions and lead to the AOD decreases in these regions. The AOD decreases over the Arctic and Antarctic oceans are because the V4 aerosol subtyping algorithm now allows all aerosol types over these regions rather than solely the clean continental and polluted continental subtypes (Sect. 2.1.1).

Updates in lidar ratio led to an AOD increase of 0.013 and 0.012 for nighttime and daytime, respectively. The 532 nm lidar ratios for three aerosol subtypes were changed in V4: clean marine, dust, and clean continental. Since the lidar ratios for these subtypes were all increased, AOD increased correspondingly. Figure 15h shows the AOD increases due to the lidar ratio updates. The AOD increase over oceans was due to the lidar ratio change for clean marine. The most significant AOD increase is seen over North Africa and the Arabian Peninsula where pure dust is a dominant aerosol type. The change was caused by the lidar ratio update for dust from 40 to 44 sr. Due to the nonlinear behavior of the AOD retrieval, this 10 % increase in lidar ratio yields an increase of up to 20 % in AOD for dust layers as shown by AOD retrieval above opaque water clouds (Liu et al., 2015). Some of the AOD increases are even larger over North Africa and the Arabian Peninsula, where the dust is generally denser than that over the East Atlantic, thus further amplifying the nonlinear response of AOD to lidar ratio.

Figure 15i shows AOD changes even if there are no changes in aerosol subtype and lidar ratio. AOD is slightly increased, especially over land. These changes are due to differences in calibration and two-way transmittance estimates for upper layers between V3 and V4.

AOD changes between V3 and V4 for cloud-free conditions are typically smaller than in the all-sky cases discussed above. Cloud-free cases are restricted to those profiles in which no clouds were detected for both V3 and V4 from the level 2 5 km profile product. Table 7 shows occurrence frequencies and AOD changes for cloud-free conditions. The frequencies for layer detection are substantially reduced compared to all-sky conditions. This implies that, when compared to V3, the V4 aerosol detection frequency is higher in cloudy skies but relatively unchanged for clear skies. This behavior is not unexpected and can be at least partially explained by the improved surface detection in V4 that identifies more transparent air columns compared with V3. This behavior can also be attributed partially to the presence of more misclassified cirrus fringes in V4 (Liu et al., 2018). Comparing Tables 6 and 7, a larger fraction of aerosols maintain their subtype in the cloud-free conditions than in the all-sky conditions (i.e., the lidar ratio category in Table 7 is 45.9 % night and 50.7 % day vs. 30.2 % night and 35.9 % day in Table 6). Due to the largely increased frequencies, AOD increases by the lidar ratio updates in V4 overwhelm all the other factors for cloud-free skies compared to cloudy skies.