Comparison of scattering ratio profiles retrieved from ALADIN/Aeolus and CALIOP/CALIPSO observations and preliminary estimates of cloud fraction profiles

The space-borne active sounders have been contributing invaluable vertically resolved information of atmospheric optical properties since the launch of Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) in 2006. To build long-term records from space-borne lidars useful for climate studies, one has to understand the differences between successive space lidars operating at different wavelengths, flying on different orbits, and using different viewing geometries, receiving paths, and detectors. In this article, we compare the results of Atmospheric Laser Doppler INstrument (ALADIN) and Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) lidars for the period from 28 June to 31 December 2019. First, we build a dataset of ALADIN–CALIOP collocated profiles ($\mathrm{\Delta }\text{dist}<\mathrm{1}{}^{\circ }$; Δtime<6 h). Then we convert ALADIN's 355 nm particulate backscatter and extinction profiles into the scattering ratio vertical profiles SR(z) at 532 nm using molecular density profiles from Goddard Earth Observing System Data Assimilation System, version 5 (GEOS-5 DAS). And finally, we build the CALIOP and ALADIN globally gridded cloud fraction profiles CF(z) by applying the same cloud detection threshold to the SR(z) profiles of both lidars at the same spatial resolution.

Before comparing the SR(z) and CF(z) profiles retrieved from the two analyzed lidar missions, we performed a numerical experiment to estimate the best achievable cloud detection agreement CDA_norm(z) considering the differences between the instruments. We define CDA_norm(z) in each latitude–altitude bin as the occurrence frequency of cloud layers detected by both lidars, divided by a cloud fraction value for the same latitude–altitude bin. We simulated the SR(z) and CF(z) profiles that would be observed by these two lidars if they were flying over the same atmosphere predicted by a global model. By analyzing these simulations, we show that the theoretical limit for ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{theor}}\left(z\right)$ for a combination of ALADIN and CALIOP instruments is equal to 0.81±0.07 at all altitudes. In other words, 19 % of the clouds cannot be detected simultaneously by two instruments due to said differences.

The analyses of the actual observed CALIOP–ALADIN collocated dataset containing ∼78 000 pairs of nighttime SR(z) profiles revealed the following points: (a) the values of SR(z) agree well up to ∼3 km height. (b) The CF(z) profiles show agreement below ∼3 km, where ∼80 % of the clouds detected by CALIOP are detected by ALADIN as expected from the numerical experiment. (c) Above this height, the ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{obs}}\left(z\right)$ reduces to ∼50 %. (d) On average, better sensitivity to lower clouds skews ALADIN's cloud peak height in pairs of ALADIN–CALIOP profiles by $\sim \mathrm{0.5}\pm \mathrm{0.6}$ km downwards, but this effect does not alter the heights of polar stratospheric clouds and high tropical clouds thanks to their strong backscatter signals. (e) The temporal evolution of the observed ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{obs}}\left(z\right)$ does not reveal any statistically significant change during the considered period. This indicates that the instrument-related issues in ALADIN L0/L1 have been mitigated, at least down to the uncertainties of the following ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{obs}}\left(z\right)$ values: 68±12 %, 55±14 %, 34±14 %, 39±13 %, and 42±14 % estimated at 0.75, 2.25, 6.75, 8.75, and 10.25 km, respectively.

1 Introduction

Clouds play an important role in the energy budget of our planet: optically thick clouds reflect the incoming solar radiation, leading to cooling of the Earth, while thinner clouds act as “greenhouse films”, preventing escape of the Earth's long-wave radiation to space. Climate feedback analyses show that clouds are a large source of uncertainty for the climate sensitivity of climate models and, so, for the future climate evolution (e.g., Nam et al., 2012; Chepfer et al., 2014; Guzman et al., 2017; Vaillant de Guélis et al., 2018; Zelinka et al., 2020). Understanding the Earth's energy budget requires knowledge of cloud's coverage, geographical and vertical distribution, temperature, and optical properties.

Satellite observations have been providing a continuous survey of clouds over the whole globe. Infrared sounders have been observing our planet since 1979: from the TIROS Operational Vertical Sounder (TOVS) instruments (Smith, 1979) on board the National Oceanic and Atmospheric Administration (NOAA) polar satellites to the Atmospheric InfraRed Sounder (AIRS) spectrometer (Chahine et al., 2006) on board Aqua (since 2002) and to the Infrared Atmospheric Sounder Interferometer (IASI) instrument (Chalon et al., 2001; Hilton et al., 2012) on board MetOp (since 2006), with increasing spectral resolution. Despite an excellent daily coverage and daytime/nighttime observation capability (Menzel et al., 2016; Stubenrauch et al., 2017), the height uncertainty of the cloud products retrieved from the observations performed by these space-borne instruments is large (e.g., Feofilov and Stubenrauch, 2017). This precludes the retrieval of the cloud's vertical profile with the accuracy needed for climate relevant processes and feedback analysis. This drawback does not exist for active sounders, which measure the altitude-resolved profiles of backscattered radiation with accuracy on the order of 10–100 m. Among them, one can name the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) lidar (Winker, 2003; Winker et al., 2004, 2007, 2009) and CloudSat radar (Stephens et al., 2002, 2009), which have been providing vertically resolved cloud and aerosol properties since 2006. The CATS (Cloud-Aerosol Transport System) lidar on board ISS provided measurements for over 33 months starting from the beginning of 2015 (McGill et al., 2015). The Atmospheric Laser Doppler INstrument (ALADIN) lidar on board Aeolus (Krawczyk et al., 1995; Stoffelen et al., 2005; Andersson et al., 2008) has been measuring horizontal winds and aerosols/clouds since September 2018. More lidars are planned – in 2023, the ATmosperic LIDar (ATLID)/EarthCare instrument (Héliere et al., 2012) will be launched, and other space-borne lidars are in the development phase. All active instruments share the same measuring principle – the emitter sends a brief pulse of laser or radar electromagnetic radiation to the atmosphere, and the receiver registers a time-resolved backscatter signal collected through its telescope. However, the wavelength, pulse energy, pulse repetition frequency (PRF), telescope diameter, orbit, detector, and many other parameters are not the same for any pair of instruments. These differences define the active instruments' capability of detecting atmospheric aerosols and/or clouds for a given atmospheric scenario and observation conditions (day, night, averaging distance). At the same time, there is an obvious need to ensure the continuity of global space-borne measurements and to get a smooth transition between the satellite missions (Winker et al., 2017; Chepfer et al., 2014, 2018).

This work seeks to address this issue using ALADIN/Aeolus space-borne wind lidar operating at 355 nm and CALIOP/CALIPSO atmospheric lidar operating at 532 nm. Even though the primary goal of ALADIN is wind detection (Reitebuch et al., 2020; Straume et al., 2020), its products include profiles of atmospheric optical properties. In addition, the methods developed in this study and its conclusions will set the stage for the future comparison of the ATLID/EarthCare observations with other space-borne lidars.

The structure of the article is as follows. In Sect. 2, we describe the datasets used in this study and explain the collocation criteria. Section 3 provides the definitions and the basic formulae needed for comparison of two lidars operating at different wavelengths. In Sect. 4, we describe the numerical experiment aimed at the estimation of the best possible theoretically achievable cloud detection agreement for the cloud fraction profiles retrieved from CALIOP and ALADIN observations. Section 5 is dedicated to the analysis of the results and to the discussion of similarities and differences between the collocated SR profiles and cloud fraction distributions. Section 6 concludes the article.

2 Data

We start this section with the description of ALADIN/Aeolus optical properties dataset, followed by the description of CALIOP/CALIPSO product. In the next steps, we define the procedures and criteria for the comparison of these two products.

2.1 AEOLUS

In this work, we provide only a brief description of the lidar and the details necessary for understanding the key differences between the compared instruments. For a detailed description of the Aeolus mission and its instrument, we refer the reader to Krawczyk et al. (1995), Stoffelen et al. (2005), Andersson et al. (2008), and Flamant et al. (2017). The Aeolus satellite carries a Doppler wind lidar called ALADIN, which operates at 355 nm wavelength and is composed of a transmitter, a Cassegrain telescope, and a receiver capable of separating the molecular (Rayleigh) and particular (Mie) backscattered photons (high-spectral-resolution lidar, HSRL). The lidar observes the atmosphere at 35^∘ from nadir and perpendicular to the satellite track, its orbit is inclined at 96.97^∘, and the instrument overpasses the Equator at 06:00 and 18:00 local solar time (LST); see also Fig. 1 and Table 1 to compare with CALIOP.

Figure 1 Observation geometry and orbits of ALADIN/Aeolus and CALIOP/CALIPSO space-borne lidars. ALADIN observes the atmosphere at dawn–dusk, whereas CALIOP passes the Equator at 01:30 and 13:30 local solar time. The difference between (a) and (b) is in the position of Earth and the time: in (b), AEOLUS overflies the same area (centered over Africa) as was observed by CALIOP ∼4.5 h earlier (in a). Figure source: own visualization in Blender 2.93 software using NASA's Earth Observatory “Blue Marble: Next Generation” collection textures.

Table 1 Comparison of orbital parameters, viewing geometries, and resolutions of ALADIN and CALIOP instruments.

The laser emitter of the lidar sends 15 ns long pulses of 355 nm radiation down to the atmosphere 50 times per second. The lidar optical system collects the backscattered photons, which are then registered in the instrument's Rayleigh and Mie channels. Wind is detected with the help of an interferometric technique from the image formed on the Accumulation Charge Coupled Device (ACCD) detectors of the lidar (Chanin et al., 1989). Besides the winds, the Aeolus processing algorithms retrieve the optical properties of the observed atmospheric layers (Ansmann et al., 2007; Flamant et al., 2017). The vertical resolution of the instrument is adjustable, but the total number of points in a vertical profile is equal to 24, which corresponds to the number of rows in the ACCD. The observation priorities changed throughout the period of the mission (ESA, 2021), and for most of the period considered in this work (see below), the vertical sampling of both Mie and Rayleigh channels between 2 and 22 km was equal to 1 km, whereas the sampling below 2 km varied from 0.25 to 1 km. The native horizontal resolution of 140 m of the instrument is sacrificed to achieve higher signal-to-noise ratio (SNR) both on board by accumulating the detected profiles and on the ground by averaging the downloaded profiles at different steps of the processing chain (Flamant et al., 2017). In Sect. 4.2, we address the effects of averaging on the accuracy of cloud detection for this instrument and give general recommendations for the future missions.

ALADIN is a relatively new instrument, and its calibration/validation activity is still on the way (Baars et al., 2020; Donovan et al., 2020; Kanitz et al., 2020; Reitebuch et al., 2020; Straume et al., 2020). This includes but is not limited to internal calibration and comparisons with other observations. The Aeolus mission faced several technical issues which hindered getting the planned specifications. These issues are related to several factors: (a) laser power degradation (60 mJ per pulse instead of 80 mJ per pulse) and signal losses in the emission and reception paths (33 %) that result in lower SNR than planned; (b) telescope mirror temperature effects, biasing the wind detection and calibration of Mie and Rayleigh channels of ALADIN; and (c) a constantly increasing number of hot pixels of both ACCD detectors (Weiler et al., 2021), leading to errors both in wind speed and in retrieved optical parameters of the atmosphere. The Aeolus teams mitigated at least some of these adverse effects (e.g., Baars et al., 2020; Weiler et al., 2021), and it would be interesting to see whether the pilot L2A dataset, Prototype_v3.10, is free of cloud detection quality trends.

We have performed the present study using the pilot L2A dataset from Aeolus, Prototype_v3.10, which is available for participating Cal/Val teams for a limited period of ALADIN's observations, from 28 June through 31 December 2019. According to Flamant et al. (2008, 2017), the L2A data are retrieved from the L1B product of this instrument, and they contain height profiles of Mie and Rayleigh co-polarized backscatter and extinction coefficients, scattering ratios (SR), and lidar ratios along the lidar line of sight (Flamant et al., 2017; Lolli et al., 2013). For each vertical profile corresponding to a slant path in Fig. 1, we extracted the SR, backscatter, and extinction profiles calculated by standard correct algorithm (Flamant et al., 2017). As for the SR, we draw the reader's attention to the definitions and conversion formulae given below in Sect. 3.2. The horizontal resolution of this product is 87 km.

The important companions of these profiles are quality flag columns. For our analysis, we kept only the layers which are marked either by a high Mie SNR flag or by a high Rayleigh SNR flag and by a flag indicating an absence of signal attenuation. These flags are necessary and sufficient for valid extinction, backscatter, and SR(z) profiles, which we use in the analysis.

2.2 CALIPSO-GOCCP

CALIOP, a two-wavelength polarization-sensitive near-nadir-viewing lidar, provides high-resolution vertical profiles of aerosols and clouds (Winker et al., 2004, 2007, 2009). Its orbital altitude is 705 km, and the orbit is inclined at 98.05^∘. The lidar overpasses the Equator at 01:30 and 13:30 LST; see also Table 1 and the left-hand side of Fig. 1a and of Fig 1b. It uses three receiver channels: one measuring the 1064 nm backscatter intensity and two channels measuring orthogonally polarized components of the 532 nm backscattered signal. Cloud and aerosol layers are detected by comparing the measured 532 nm signal return with the return expected from a molecular atmosphere (see the definitions in Sect. 3.2).

The General Circulation Model (GCM) Oriented Cloud Calipso Product (CALIPSO-GOCCP) was initially designed to evaluate GCM cloudiness. It is derived from CALIPSO L1/NASA products at the Laboratory of Dynamic Meteorology (LMD) and Institute of Pierre-Simon Laplace (IPSL) with the support of NASA/CNES, ICARE Thematic Center (Lille, France), and ClimServ data service (IPSL), and it contains observational cloud diagnostics, including the instantaneous scattering ratio (profiles) at the native horizontal resolution of CALIOP (333 m) and at 480 m vertical resolution (Chepfer et al., 2008, 2010, 2013). This makes it a good reference dataset for ALADIN retrievals because one can easily recalculate it to ALADIN's horizontal and vertical grids through averaging along the track and in vertical bins, respectively.

2.3 Collocation of AEOLUS and CALIPSO profiles

Figure 1 illustrates the orbit and overpass time differences between the two lidars. In Fig. 1b, AEOLUS overflies the same area as was measured by CALIOP ∼4.5 h earlier (Fig. 1a). We recall here that ALADIN's line of sight is pointed at 35^∘ to nadir and perpendicular to the flight direction (purple slant paths on the right-hand side of Fig. 1a and of Fig. 1b), whereas CALIOP probes the atmosphere in near-nadir mode (3^∘ off-nadir). As for any collocation, there is a trade-off between the quality of collocation and the number of collocated pairs of profiles. As we show below, for AEOLUS and CALIPSO, one has to supplement this tradeoff with a requirement of a representative geographical coverage because imposing a stricter temporal overlap criterion adversely affects the latitudinal distribution of the collocated points. Since the horizontal averaging and resolution of the Aeolus Prototype_v3.10 product is 87 km, there is not much sense in collocating the data with the accuracy better than this value. On the other hand, a fractional standard deviation f_c of cloud water content at 1^∘ (∼111 km) distance is about 0.5 for a cloud cover of 1 (Boutle et al., 2014), and there is a risk of comparing incoherent quantities, so we took $\mathrm{\Delta }\text{dist}=\mathrm{1}{}^{\circ }$ as a limit for the collocations and created several subsets based on the Δtime, the absolute value of the difference between the two collocated measurements. In Fig. 2, we show six such subsets, and Table 2 provides the total number of collocations for each of them. On the one hand, one can see that a strict collocation criterion of Δtime<1 h (black curve in Fig. 2) provides the information only about two narrow zones in the southern and northern polar regions. On the other hand, an excellent latitudinal coverage corresponding to Δtime<24 h (dashed magenta curve in Fig. 2) comes at the cost of mixing up the cases, which differ by almost 1 d, which is unacceptable from the point of view of temporal variation. Finally, we have chosen a subset corresponding to Δtime<6 h for the analysis. Over the oceans, the diurnal effects in cloud distribution associated with this delay are small (e.g., Noel et al., 2018; Chepfer et al., 2019; Feofilov and Stubenrauch, 2019), and the land represents just one-third of the analyzed cases. ALADIN observes the atmosphere in dusk–dawn mode, whereas CALIPSO has a clear separation between the daytime and the nighttime observations (Fig. 1). To avoid the risks associated with the solar contamination, we only picked up the collocations which correspond to nighttime CALIPSO observations. This yielded about 7.7×10⁴ pairs of SR profiles. In the Supplement (Feofilov et al., 2021), we provide the complete collocated database, which corresponds to the last line and fourth column of Table 2 (3.2×10⁵ collocations), for further analysis by interested teams.

Figure 2 Latitudinal coverage of collocated points for $\mathrm{\Delta }\text{dist}<\mathrm{1}{}^{\circ }$ and different limits for Δtime.

Table 2 Number of collocated cases for $\mathrm{\Delta }\text{dist}<\mathrm{1}{}^{\circ }$ and different Δtime values.

3 Method

3.1 Lidar equation

An atmospheric lidar sends a brief pulse of laser radiation directed towards the atmosphere. The lidar optics collects the backscattered photons and drives them to a detector. The detected signal is time-resolved, and each time bin corresponds to a fixed distance from the lidar to a certain atmospheric layer. The AEOLUS wavelength of emission is 355 nm, while CALIPSO emits at 532 and 1064 nm. In the atmosphere, the photons coming from the lidar can be backscattered by the molecules, which are much smaller than the wavelength of our two lasers (Rayleigh scattering), by the aerosol particles, which are comparable to or larger than the wavelengths of our two lasers, and by the coarse aerosol and cloud particles, which are much larger than the wavelengths of our two lasers (Mie scattering). These processes are characterized by the corresponding wavelength-dependent backscatter coefficients β_mol(λ, z) and β_part(λ, z) measured per meter per steradian (m⁻¹ sr⁻¹). The attenuation of the laser beam along its path within each layer is characterized by extinction coefficients α_mol(λ, z) and α_part(λ, z) per meter (m⁻¹). On their pathway in the atmosphere, the photons are also scattered in other directions than backscatter and then collected in the telescope after multiple scatterings. The total lidar attenuated backscattered signal (ATB) corrected for geometrical effects and normalized to molecular signal is usually written as

$$\begin{array}{}\text{(1)}& \begin{array}{rl}\text{ATB}(\mathit{\lambda },z)=& \left({\mathit{\beta }}_{\mathrm{mol}}(\mathit{\lambda },z)+{\mathit{\beta }}_{\mathrm{part}}(\mathit{\lambda },z)\right)\\ & \times e^{-\mathrm{2}{\int }_{Z_{\mathrm{sat}}}^z\left({\mathit{\alpha }}_{\mathrm{mol}}\right(\mathit{\lambda },z^{\prime })+\mathit{\eta }{\mathit{\alpha }}_{\mathrm{part}}(\mathit{\lambda },z^{\prime }\left)\right)\mathrm{d}{z}^{\prime }},\end{array}\end{array}$$

where Z_sat is the altitude of the satellite, λ is the wavelength, and η is a multiple scattering coefficient, which depends on the lidar configuration and is set to 0.7 for CALIOP (see Winker, 2003; Chiriaco et al., 2006; Chepfer et al., 2008; Garnier et al., 2015; Reverdy et al., 2015, for the discussion of this value).

3.2 Two definitions of scattering ratio profile

To highlight particles in an atmospheric layer versus molecular background, one often uses the “scattering ratio” or SR. But, two different definitions of SR exist in the literature, and in particular, the ALADIN documents and CALIPSO documents do not use the same definition. So, we provide both definitions and explain our choice below. The first one relates only to scattering properties of the medium and is used in the ALADIN product (Flamant et al., 2017):

$$\begin{array}{}\text{(2)}& {\text{SR}}^{\mathrm{A}}(\mathit{\lambda },z)={\displaystyle \frac{{\mathit{\beta }}_{\mathrm{mol}}(\mathit{\lambda },z)+{\mathit{\beta }}_{\mathrm{part}}(\mathit{\lambda },z)}{{\mathit{\beta }}_{\mathrm{mol}}(\mathit{\lambda },z)}}.\end{array}$$

According to this definition, SR^A(λ, z) is strictly equal to or greater than unity, and its interpretation is straightforward: the larger the number, the stronger the contribution of particles to backscattered signal is. But, this definition requires knowledge of both β_mol(λ, z) and β_part(λ, z), which are available from ALADIN observations thanks to its HSRL capability (see Sect. 2.1) but not from non-HSRL lidars such as CALIPSO (see Sect. 2.2).

The second definition is closer to the profiles observed by classic non-HSRL lidars and is used in CALIPSO products (e.g., Chepfer et al., 2008, 2013):

$$\begin{array}{}\text{(3)}& {\text{SR}}^{\mathrm{C}}(\mathit{\lambda },z)={\displaystyle \frac{\text{ATB}(\mathit{\lambda },z)}{\text{AMB}(\mathit{\lambda },z)}},\end{array}$$

where ATB(λ, z) is the total attenuated backscatter given by Eq. (1), and AMB(λ, z) is the attenuated molecular backscatter estimated in the absence of particles:

$$\begin{array}{}\text{(4)}& \text{AMB}(\mathit{\lambda },z)={\mathit{\beta }}_{\mathrm{mol}}(\mathit{\lambda },z)\times e^{-\mathrm{2}{\int }_{Z_{\mathrm{sat}}}^z{\mathit{\alpha }}_{\mathrm{mol}}(\mathit{\lambda },z{}^{\prime })\mathrm{d}{z}^{\prime }}.\end{array}$$

The ATB(λ, z) values in Eq. (3) are measured by a lidar, and the profile of AMB(λ, z) can be estimated from Eq. (4) if the molecular density profile is known. Since the exponential part in the enumerator of Eq. (1) leads to a significant attenuation in the presence of particles, the value of SR(λ, z) can be less than unity below a thick cloud layer. The definition (Eq. 4) is convenient for the lidars, which cannot distinguish the molecular and particulate components of the backscattered signals. Since CALIOP is such a lidar, we will use this very definition in the present work as was done before (Chepfer et al., 2008, 2013). Correspondingly, here and below $\text{SR}(\mathit{\lambda },z)={\text{SR}}^{\mathrm{C}}(\mathit{\lambda },z)$. In the rest of the paper, we use the definition of $\text{SR}(\mathit{\lambda },z)={\text{SR}}^{\mathrm{C}}(\mathit{\lambda },z)$ given in Eq. (3), and we do not use the SR^A(λ, z) given in Eq. (2).

3.3 Estimating SR(z) profiles at 532 nm from ALADIN data

After having stated the SR definition (Eq. 3) that we will use to compare the observations collected by the two instruments, we now need to consider their wavelength differences. Indeed, the SR(z) profile is wavelength-dependent as α_mol(λ, z), α_part(λ, z), β_mol(λ, z), and β_part(λ, z), and, therefore, ATB(λ, z) and AMB(λ, z) depend on the wavelengths. Therefore, one needs to convert the first instrument's SR values to those of the second one, or vice versa. Leaping ahead, we say that since ALADIN can distinguish the molecular backscatter from the particulate one, it provides more information, so it is better suited for the conversion than CALIOP. Below, we provide the formalism used for the conversion (Collis and Russell, 1976; Bucholtz, 1995) as well as the corresponding variable values calculated at two wavelengths, 355 and 532 nm. For the molecular backscatter,

$$\begin{array}{}\text{(5)}& {\displaystyle }\begin{array}{l}{\mathit{\beta }}_{\mathrm{mol}}(\mathit{\lambda },z)={\left(\mathrm{d}\mathit{\sigma }/\mathrm{d}\mathrm{\Omega }\right)}_{\mathit{\lambda }}\times N\left(z\right);\\ {\mathit{\alpha }}_{\mathrm{mol}}(\mathit{\lambda },z)={\displaystyle \frac{\mathrm{4}\mathit{\pi }}{\mathrm{1.5}}}{\mathit{\beta }}_{\mathrm{mol}}(\mathit{\lambda },z)\end{array}\text{(6)}& {\displaystyle }{\left(\mathrm{d}\mathit{\sigma }/\mathrm{d}\mathrm{\Omega }\right)}_{\mathit{\lambda }}={\displaystyle \frac{\mathit{\sigma }(\mathit{\lambda },z)}{\mathrm{4}\mathit{\pi }}}\times {\displaystyle \frac{\mathrm{3}}{\mathrm{4}}}\left(\mathrm{1}+{\cos }^{\mathrm{2}}\left(\mathit{\pi }\right)\right)\text{(7)}& {\displaystyle }\mathit{\sigma }(\mathit{\lambda },z)={\displaystyle \frac{\mathrm{24}{\mathit{\pi }}^{\mathrm{3}}{\left(n_{\mathrm{s}}^{\mathrm{2}}\left(\mathit{\lambda }\right)-\mathrm{1}\right)}^{\mathrm{2}}\left(\mathrm{6}+\mathrm{3}\mathit{\rho }\left(\mathit{\lambda }\right)\right)}{{\mathit{\lambda }}^{\mathrm{4}}{N}_{\mathrm{s}}^{\mathrm{2}}{\left(n_{\mathrm{s}}^{\mathrm{2}}\left(\mathit{\lambda }\right)+\mathrm{2}\right)}^{\mathrm{2}}\left(\mathrm{6}-\mathrm{7}\mathit{\rho }\left(\mathit{\lambda }\right)\right)}},\end{array}$$

where (dσ/dΩ)_λ is a differential cross section (m² sr⁻¹), N(z) is the number density (m⁻³), σ(λ, z) is the Rayleigh cross section (m²), n_s(λ) is the refractive index for standard air, ρ(λ) is the depolarization factor, and N_s is the number density of standard air (2.54743×10²⁵ m⁻³). We estimated the n_s(λ) values according to Ciddor (1996) and obtained n_s(355 nm)=1.00028571 and n_s(532 nm)=1.00027821. We took the ρ(λ) values from Table 1 of Bucholtz (1995) according to which $\mathit{\rho }\left(\mathrm{355}\mathrm{nm}\right)=\mathrm{3.01}\times {\mathrm{10}}^{-\mathrm{2}}$ and $\mathit{\rho }\left(\mathrm{532}\mathrm{nm}\right)=\mathrm{2.84}\times {\mathrm{10}}^{-\mathrm{2}}$. The corresponding values of (dσ/dΩ)_λ at 355 and 532 nm are then $\mathrm{3.2897988}\times {\mathrm{10}}^{-\mathrm{31}}$ m² sr⁻¹ and $\mathrm{6.1668318}\times {\mathrm{10}}^{-\mathrm{32}}$ m² sr⁻¹, respectively. For the particulate backscatter, we took advantage of the fact that the extinction and backscatter coefficients α_part(λ, z) and β_part(λ, z) barely change at these wavelengths for large particles. Using a known molecular density profile from GEOS-5 DAS (Goddard Earth Observing System Data Assimilation System, version 5; see Rienecker, 2008), and estimating the AMB(532 nm, z) and ATB(532 nm, z) values from Eqs. (1), (2), and (4)–(7), we finally get the $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ profile for ALADIN, which is comparable with the SR(532 nm, z) of CALIOP:

$$\begin{array}{}\text{(8)}& \begin{array}{rl}& \text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)={\displaystyle \frac{{\mathit{\beta }}_{\mathrm{mol}}(\mathrm{532},z)+{\mathit{\beta }}_{\mathrm{part}}(\mathrm{355},z)}{{\mathit{\beta }}_{\mathrm{mol}}(\mathrm{532},z)}}\\ & \times {\displaystyle \frac{e^{-\mathrm{2}{\int }_{Z_{\mathrm{sat}}}^z({\mathit{\alpha }}_{\mathrm{mol}}\left(\mathrm{532},z{}^{\prime }\right)+{\mathit{\eta }}_{\mathrm{355}}{\mathit{\alpha }}_{\mathrm{part}}\left(\mathrm{355},z{}^{\prime }\right))\mathrm{d}{z}^{\prime }}}{e^{-\mathrm{2}{\int }_{Z_{\mathrm{sat}}}^z{\mathit{\alpha }}_{\mathrm{mol}}\left(\mathrm{532},z{}^{\prime }\right)\mathrm{d}z{}^{\prime }}}}.\end{array}\end{array}$$

In Eq. (8), we deliberately kept 355 nm for particle backscatter and extinction coefficients to show their provenance. As for the multiple scattering coefficient η₃₅₅, it is usually accepted that the multiple scattering effects can be neglected for ALADIN with its field of view (FOV) of just ∼0.02 mrad. But, there are indications (e.g., Donovan et al., 2020) that these effects still can affect the observations, so we took it equal to 0.9. We note that the value of η₃₅₅ in our conversion (Eq. 8) mostly affects the low-level clouds, where, as we will see below, the instruments compare well.

Since the vertical resolution of ALADIN is changing with geographical region, and the special range-bin adjustment was performed for certain periods throughout the mission, we recalculated all SR profiles to a regular 1 km vertical grid.

3.4 Calculating averaged SR(z) profiles from CALIOP data

Since we used high-resolution CALIOP data on a 333 m horizontal grid, a direct comparison with the ALADIN L2 product with its 87 km horizontal averaged data was not possible. To calculate the averaged CALIOP SR(z), we took the original ATB(z) and AMB(z) profiles, averaged them in the ±40 km along the CALIOP orbit track in the vicinity of the point defined as the closest one to ALADIN's track, and got the SR(z) (Eq. 3). The CALIOP vertical binning was also adjusted to imitate the coarse vertical resolution of ALADIN (1 km). This averaging, together with the application of a SR cloud detection threshold, may lead to an overestimation of the cloud fraction in the boundary layer, for a field of optically thick geometrically small liquid clouds, e.g., shallow cumulus (see Sect. 4.2), as discussed in Chepfer et al. (2010, 2013), but this overestimation should be similar for both CALIOP and ALADIN.

In the rest of the article, we discuss the collocated and recalculated SR profiles at 87 km horizontal and 1 km vertical resolution.

3.5 Cloud detection, cloud fraction, and normalized cloud detection agreement

In this work, we define an atmospheric layer as cloudy when the following condition is fulfilled (Chepfer et al., 2013):

$$\begin{array}{}\text{(9)}& \text{SR}(\mathrm{532}\mathrm{nm},z)>\mathrm{5}.\end{array}$$

For cloud detection, we deliberately do not apply the second criterion of Chepfer et al. (2013),

$$\begin{array}{}\text{(10)}& \mathrm{\text{ATB}}(\mathrm{532}\mathrm{nm},z)-\text{AMB}(\mathrm{532}\mathrm{nm},z)>\mathrm{2.5}\times {\mathrm{10}}^{-\mathrm{6}}{\mathrm{m}}^{-\mathrm{1}}{\mathrm{sr}}^{-\mathrm{1}},\end{array}$$

because of two reasons: (a) this criterion was introduced in Chepfer et al. (2013) to filter noise in individual profiles at native CALIOP resolution ($\mathrm{1}/\mathrm{3}$ km along track), whereas in this work we use the SR(532 nm, z) averages recalculated from ATB and AMB over ∼80 km distance along the track, and (b) this would have adversely affected the high cloud amount of ALADIN. Even though this definition makes the CALIOP clouds inconsistent with their definition in current CALIPSO products, this allows the potential capabilities of ALADIN for cloud detection to be estimated. If a given atmospheric layer was observed multiple times, we define the cloud fraction (CF) in the usual way:

$$\begin{array}{}\text{(11)}& \text{CF}\left(z\right)={\displaystyle \frac{N_{\mathrm{cld}}\left(z\right)}{N_{\mathrm{tot}}\left(z\right)}},\end{array}$$

where N_cld(z) is a number of times the condition of Eq. (9) is fulfilled and N_tot(z) is a total number of measurements in this layer. As for cloud detection agreement and disagreement, we distinguish four cases: when both CALIOP and ALADIN detect a cloud; when neither of them detects a cloud; when CALIOP detects a cloud, whereas ALADIN misses it; and when ALADIN detects a cloud, whereas CALIOP misses it. We will name these cases YES_YES, NO_NO, YES_NO, and NO_YES and will define their occurrence frequencies as

$$\begin{array}{}\text{(12)}& R_{\mathrm{YES}\mathrm{\_}\mathrm{YES}}\left(z\right)={\displaystyle \frac{N_{\mathrm{YES}\mathrm{\_}\mathrm{YES}}\left(z\right)}{N_{\mathrm{tot}}\left(z\right)}};R_{\mathrm{NO}\mathrm{\_}\mathrm{NO}}\left(z\right)={\displaystyle \frac{N_{\mathrm{NO}\mathrm{\_}\mathrm{NO}}\left(z\right)}{N_{\mathrm{tot}}\left(z\right)}};\end{array}$$

$$R_{\mathrm{YES}\mathrm{\_}\mathrm{NO}}\left(z\right)={\displaystyle \frac{N_{\mathrm{YES}\mathrm{\_}\mathrm{NO}}\left(z\right)}{N_{\mathrm{tot}}\left(z\right)}};R_{\mathrm{NO}\mathrm{\_}\mathrm{YES}}\left(z\right)={\displaystyle \frac{N_{\mathrm{NO}\mathrm{\_}\mathrm{YES}}\left(z\right)}{N_{\mathrm{tot}}\left(z\right)}}.$$

The first term in Eq. (12) corresponds to cloud detection agreement (CDA(z)), which we will also use in its normalized form, CDA_norm(z):

$$\begin{array}{}\text{(13)}& {\text{CDA}}_{\mathrm{norm}}\left(z\right)={\displaystyle \frac{\text{CDA}\left(z\right)}{\text{CF}\left(z\right)}}={\displaystyle \frac{N_{\mathrm{YES}\mathrm{\_}\mathrm{YES}}\left(z\right)}{N_{\mathrm{cld}}\left(z\right)}}.\end{array}$$

As follows from these definitions, if CF(z) is greater than zero, and CDA_norm(z) is equal to 1, then there is a perfect agreement between the clouds retrieved from both instruments. In the same way, if CF(z) is greater than zero, and CDA_norm(z) is equal to 0, then there is no agreement.

4 Theoretical estimate of the best achievable cloud detection agreement between ALADIN and CALIOP

The aforementioned differences between the missions prevent the two lidars from observing the same clouds at the same time, except for the polar zones. Knowing the differences in the orbits, wavelengths, and spatial resolution, one can carry out a numerical experiment aimed at the estimation of the best achievable agreement CDA^theor(z) and ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{theor}}\left(z\right)$ that one can expect for a combination of these two missions.

4.1 Setup of the numerical experiment

To estimate the theoretically possible cloud detection agreement for a considered combination of two lidars and for the chosen collocation criteria, we performed the following numerical experiment outlined in a flowchart in Fig. 3. First, we created a gridded atmosphere from the output of the U.S. Department of Energy's Energy Exascale Earth System Model (E3SM) atmosphere model (EAM) version 1 (EAMv1; Rasch et al., 2019) for the conditions of autumn equinox in the Northern Hemisphere. This subset does not contain the winter atmosphere possible for the period covered by the Aeolus Prototype_v3.10 dataset, but it is representative enough from the point of view of the cloud fraction profiles and their variability since it presents a snapshot of both hemispheres, pole to pole. From these data, we created a set of daily orbits or “lidar curtains” at the horizontal resolution of CALIOP (333 m). Since the resolution of EAMv1 data is coarser than that of CALIOP, we estimated the subgrid cloud variability along the satellite's track using the parameterization of Boutle et al. (2014) and added it to the data.

Figure 3 A flowchart explaining the numerical experiment on estimating the best possible cloud detection agreement for a combination of ALADIN and CALIOP observations. Green boxes list the input and output data. Black text between boxes describes actions performed on each dataset. Blue text in the boxes marks the datasets used in the estimation. White text in square brackets in the boxes indicates horizontal (H) and vertical (V) resolutions of the datasets.

Then we fed this high-resolution atmospheric input to the Cloud Feedback Model Intercomparison Project Observational Simulator Package, v2 (COSP2) simulator, which calculates the atmospheric observables for space-borne instruments (Swales et al., 2018). The CALIOP simulator is built into COSP2 (Chepfer et al., 2008), whereas the ALADIN simulator is not yet a part of this package, so we used the 355 nm calculations by COSP2 (initially developed for ATLID; Reverdy et al., 2015) at fine grid scale, corresponding to ALADIN's original laser pulse frequency rate (50 Hz). To imitate the diurnal variation, we modulated the SRs using the 6 h diurnal cycle amplitudes for land and ocean retrieved from active and passive observations (Noel et al., 2018; Chepfer et al., 2019; Feofilov and Stubenrauch, 2019). With these two high-resolution simulations in hand, we created simulated pairs of “collocated” data with the Δdist distribution modulated by that of a real collocated dataset. Then we averaged the high-resolution profiles over ∼80 km distance along the track and over 1 km vertically. Besides testing noise-free simulations, we also checked the effects introduced by instrumental noise, which we estimated from the uppermost parts of measured profiles. For both instruments, these measurements are cloud-free, and the molecular return is supposed to be smooth. Correspondingly, we estimated it by a least-squares fit to measured molecular return and subtracted it from the profile. The root-mean-square of the remaining difference gave us a noise level, which we used in the simulations. For CALIOP, the noise level obtained for instantaneous measurements was scaled in accordance with the averaging distance (see Sect. 3.3). Overall, we considered about 1×10⁵ pairs of pseudo-collocated averaged profiles of SR(532 nm, z) and $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$. Using these pairs and applying the same cloud detection threshold (Eq. 9), we estimated the cloud fraction profiles (Eq. 11) and the occurrence frequency profiles for simultaneous cloud detection by both instruments (Eq. 12). Finally, we estimated the normalized cloud detection agreement (Eq. 13).

4.2 Horizontal and vertical averaging and its effects on ALADIN's capability to retrieve clouds

A common way of reducing noise in the observations is accumulation of signal and averaging N realizations of the same measurement, which will reduce noise level by a factor of $\sqrt{N}$. The reverse side of this improved signal-to-noise ratio (SNR) is a loss of information if the signal varies. The Nyquist–Shannon–Kotelnikov sampling theorem says that “if a function x(t) contains no frequencies higher than B Hz, it is completely determined by giving its ordinates at a series of points spaced $\mathrm{1}/$ (2B) seconds apart”. For satellite observations of clouds, we can reformulate this as follows: for a scene composed of a mixture of clear sky and clouds (e.g., Fig. 4a), the averaging length should be comparable to the size of the smallest element one wants to resolve. One can show that neglecting this rule will lead to overestimating cloud fraction. Consider an inhomogeneous scene, which is measured 250 times (line 1 in Fig. 4a), 80 of which give a strong backscatter signal from clouds and 170 are clear sky cases. Here, the cloud fraction calculated in accordance with Eq. (11) will be equal to 0.32. At the same time, averaging the same scene through one long measurement (line 2 in Fig. 4a) will return CF=1.0 because N_cld=1 will be triggered by a strong signal coming from a mixture of thick clouds and clear sky cases. This problem has been addressed before (e.g., Chepfer et al., 2010, 2013), and here we will illustrate it in application to CALIOP–ALADIN comparisons at their native scales to see what are the effects of averaging in ALADIN. In this exercise, we use two resolutions of ALADIN. Even though the PRF of ALADIN laser is 50 Hz, its ACCD accumulates the backscattered photons over the periods of 0.4 s (∼2.9 km) and transmits them to the ground where the L2A processor further averages the signal, leading to 87 km resolution of the L2A optical products. Recently, the onboard accumulation time (or distance) was doubled because of optical losses in the emission path of the instrument. In the exercise below, we show the estimates for 2.9 km averaging (line 3 in Fig. 4a), which corresponds to the period analyzed in this work.

Figure 4 Effects of horizontal and vertical averaging on cloud detection. (a) Visible satellite image from the NASA Moderate Resolution Imaging Spectroradiometer (MODIS) showing horizontal cloud structure of the order of 5–7 km; image from the NASA MODIS Rapidfire archive. Image scale is approximately 330 km×660 km, and the artificial CALIPSO and Aeolus tracks are superimposed to illustrate the problem of averaging over long distances. 1 – CALIOP at 333 m. 2 – ALADIN at 87 km. 3 – ALADIN at 2.9 km. (b) Simulated scattering ratio for CALIOP at its native resolution of 333 m (H) and 60 m (V). (c) CALIOP clouds. (d) Simulated scattering ratio for ALADIN converted to SR_532 at 1 km (V) and 87 km (H). (e) ALADIN clouds estimated for (d). (f) Same as (d) but for 2.9 km (H). (e) ALADIN clouds estimated for (f).

For this exercise, we used the setup described in Sect. 4.1 and picked up a piece of orbit which contains both thick and thin and single- and multilayer clouds. The first simulation (Fig. 4b) shows the scattering ratio for this scene calculated at the original vertical and horizontal resolutions of CALIOP (60 and 333 m, respectively). In Fig. 4c, the red color marks the areas for which the scattering ratio is greater than cloud detection threshold (Eq. 9). Dashed circles mark the areas that require specific attention (see below). In Fig. 4d and e, we show the results of the same calculations performed at the resolution of ALADIN L2A product (1 km V, 87 km H). As one can see, the clouds are reproduced well, except for the areas marked by dashed circles. In the first and second circles from the left, the cloud is not detected because it is thin, and the signal is not strong enough to trigger the detection. However, in the third circle, the cloud fraction is overestimated. In Fig. 4f and g, we estimate the ALADIN cloud detection at the same vertical resolution of 1 km and horizontal resolution of 2.9 km. As one can see, Fig. 4g resembles Fig. 4c in encircled areas much more than Fig. 4e does. The thin cloud in the second dashed circle could be further improved if the vertical resolution was different. Technically, one could improve it for certain heights at the cost of the other ones, but practically the ALADIN vertical resolution is adjusted mostly for surface and boundary areas, and the total number of vertical bins is limited by 24, so we did not attempt to model this setup. Summing up, if the L2A processor of ALADIN manages to process real measurements at ∼3 km horizontal resolution, we would recommend this resolution as a tradeoff between information content and noise for this instrument. For future lidar missions, it is highly advisable to register data at CALIOP vertical and horizontal resolutions and average them only if needed to detect long, thin cloud and/or aerosol layers.

4.3 Theoretically achievable cloud detection agreement between ALADIN and CALIOP

In Fig. 5, we show the profiles of CF^theor(z), CDA^theor(z), and ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{theor}}\left(z\right)$ estimated in the approach outlined above. To address the contribution of different processes to the cloud detection agreement, we show both the simulations performed with the instrumental noise and diurnal variation and the simulations performed without these perturbations. As one can see, both the CDA^theor(z) and ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{theor}}\left(z\right)$ are mostly defined by a horizontal variability of aerosols/clouds combined with differences in viewing geometries of two instruments. Observation noise and diurnal variation play a secondary role (compare the curves with and without variations or “noise” in Fig. 5). Overall, we estimate the mean value of the theoretically achievable normalized cloud detection agreement ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{theor}}\left(z\right)$ for the collocated data in the outlined setup to be equal to 0.81±0.07. As one can see, the vertical profile of ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{theor}}\left(z\right)$ does not change much with altitude, indicating that the primary sources of discrepancy are the observation geometry and the spatial variability of clouds combined with the chosen collocation criterion. If the noise were the primary source of discrepancy, we would observe a decrease of ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{theor}}\left(z\right)$ profile with height. In Sect. 5.3 and 5.5, we will use the theoretical limit of 0.81±0.07 obtained in this section as a benchmark.

Figure 5 Estimating theoretical cloud detection agreement (CDA) using pseudo-collocated SR(532 nm,z) and $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ profiles calculated using the COSP2 lidar simulator coupled with the output of the EAMv1 atmospheric model (see Fig. 3). The definitions of cloud fraction (CF), CDA, and CDA_norm variables are given in Sect. 3.5. “Noise” stands for calculations considering experimental noise of CALIOP and ALADIN and diurnal variation of the clouds for the collocation Δtime up to 6 h (see Fig. 2). The “${\text{CDA}}_{\mathrm{norm}}^{\mathrm{obs}}\left(z\right)$” cyan curve comes from the analysis of real collocated data and is mentioned in Sect. 5.3.

5 Analysis of the ALADIN and CALIPSO observations

5.1 Comparison of SR–height histograms in each latitude band

To give a general overview of the agreement between the SR(532 nm, z) and $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$, we have split the collocated data to latitudinal zones: 90–60^∘ S, 60–30^∘ S, 30^∘ S–30^∘ N, 30–60^∘ N, and 60–90^∘ N (Fig. 6). If the detection efficiency of different cloud types were the same for two instruments, the pairs of Fig. 6 panels (a–f), (b–g), (c–h), (d–i), and (e–j) would have been close to each other because of two reasons. First, the horizontal variability of clouds would have canceled out due to averaging over many profiles within the zone. Second, the diurnal variation is minor over oceans, which make up two-thirds of the data used for Fig. 6 (Noel et al., 2018; Chepfer et al., 2019; Feofilov and Stubenrauch, 2019). Analyzing Fig. 6 one can note that (1) the SR–height histograms of CALIOP (Fig. 6c–e) show two distinct peaks corresponding to low-level and high-level clouds; this feature is coherent with other observations, e.g., with GEWEX (Global Energy and Water cycle Experiment) cloud assessment (Stubenrauch et al., 2013). (2) The SR–height histograms built for $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ retrieved from ALADIN's observations (Fig. 6f–j) are characterized by a smoother occurrence frequency plot, where the two-peak structure is less pronounced than in CALIOP. (3) Even though ALADIN detects polar stratospheric clouds (PSCs), its overall sensitivity to clouds above ∼3 km altitude is lower than that of CALIOP. (4) In each latitude band, the SR–height histograms agree reasonably well up to ∼3 km altitude. (5) Both datasets show a layer of enhanced backscatter closer to the tropopause, which is not strong enough to trigger the cloud detection defined in this work. (6) ALADIN's PSCs retrieved from $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ appear “brighter” than those estimated from CALIOP's SR(532 nm, z). This is likely an artifact due to our conversion of the ALADIN signal into $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$, which assumes the particulate backscatter is about the same at 532 and 355 nm (Eq. 8), while about 50 % of PSCs contain droplets composed of super-cooled ternary solution (STS), the backscatter of which at 355 nm is roughly twice as large as that of 532 nm (Jumelet et al., 2009). The latter point explains ALADIN's higher sensitivity to PSCs compared to cirrus clouds. In the next step, we compare the “instantaneous” profiles provided by CALIOP and ALADIN having in mind the cloud detection sensitivity issues observed in Fig. 6.

Figure 6 Scattering ratio–height (SR–height) distributions in different latitude zones for the Δtime<6 h, Δdist<1^∘ collocated nighttime data subset (see Table 2): (a–e) CALIOP SR(532 nm,z) averages and (f–j) $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ estimated from ALADIN extinction and backscatter coefficients. (a, f) 90–60^∘ S, (b, g) 60–30^∘ S, (c, h) 30^∘ S–30^∘ N, (d, i) 30–60^∘ N, and (e, j) 60–90^∘ N.

5.2 Comparing individual SR profiles

Since Fig. 6 revealed certain differences between the two datasets, we inspected collocated data, looking for the specific cases which would explain the differences shown in Fig. 6. First, we wanted to test ALADIN's capabilities of high cloud detection. The subset we used for this task had to satisfy the following criteria: (1) both instruments should have at least one strong SR peak, (2) the height of this peak detected by one instrument should match the height of the peak detected by a second instrument within 1 km, and (3) the CALIOP SR profile should have a peak at or above 9 km (Fig. 7a–j). For comparison purposes, the panels in Fig. 7 represent the individual profiles belonging to the same five zones as the panels of Fig. 6. As for the potential capability of ALADIN to detect high clouds, the subset shown in Fig. 6a–e represents the cases for which this instrument retrieved the peak of about the same magnitude and height as the peak detected by CALIOP. Even though these cases exist, they are less frequent than those shown in Fig. 7f–j when ALADIN misses a high cloud but detects a lower cloud reported by CALIOP.

Figure 7 Pseudo-instantaneous comparisons of collocated ALADIN L2A SR profiles and CALIOP SR profiles averaged over 67 km along the track: (a, f, k) 90–60^∘ S, (b, g, l) 60–30^∘ S, (c, h, m) 30–30^∘ N, (d, i, n) 30–60^∘ N, and (e, j, o) 60–90^∘ N. (a–e) Cases confirming ALADIN's capability to detect high-level clouds, (f–j) cases showing the cases when ALADIN misses a high cloud detected by CALIOP, and (k–o) cases showing a low-level cloud detected by ALADIN and not detected by CALIOP in the presence of a higher thick cloud detected by both instruments.

To test whether the said mismatch is linked with the diurnal variation, we varied Δtime in 3–12 h limits, but this did not change the frequency of occurrence of high and low cloud detection. This gives a hint that the instrumental part itself provides the backscatter information sufficient for cloud detection up to 20 km, but the detection algorithm suppresses noisy solutions. The reasons for this presumable “noise” might be linked with instrumental issues discussed below, but they might also be related to the ratio of particulate and molecular backscatter at 355 nm. Let us have a closer look: the molecular signal is stronger at 355 nm, and the particulate signal is comparable to that at 532 nm. At the same time, ALADIN is an HSRL instrument, and the separation to molecular and particulate component requires disentangling of the signals measured in Mie and Rayleigh channels (crosstalk correction). Correspondingly, the error propagation in this procedure might adversely affect the SNR in Mie channel and, therefore, the SNRs of the extinction, backscatter, and recalculated $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$. Characterizing these differences and their impact on retrieved clouds is beyond this study, and it requires further investigation, but we believe that the high cloud detection agreement might be improved by studying the collocated cases provided in the Supplement and by applying different noise filtering techniques in the $\text{L0}\to \text{L1}\to \text{L2}$ elements of the ALADIN retrieval chain. As for Fig. 7k–o, we will discuss them below in the context of low-level cloud observations.

5.3 Cloud detection agreement

In Fig. 8, we show zonal cloud fraction profiles built from the collocated dataset of SR(532 nm, z) and $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ using the same threshold (Eq. 9) for both datasets. Despite the differences in SR absolute values, the CF(z) profiles estimated from CALIOP and ALADIN demonstrate reasonable agreement. Pearson's correlation coefficient for the panels in Fig. 8a and b varies between 0.7 and 0.9 for most heights (Fig. 8c), and the relative difference between the panels changes from 50 % in the lower layers through − 50 % at 11 km and to 25 % near the tropopause (Fig. 8c).

Figure 8 Latitudinal–altitudinal distributions of cloud amount defined from (a) CALIOP and (b) ALADIN and altitudinal profiles of Pearson's correlation coefficient and relative difference between ALADIN and CALIOP (c).

To illustrate the zonal CF(lat, z) profiles' behavior, we split the collocated data into four groups defined in Sect. 3.5 (Eq. 12 and text preceding it). In Fig. 9, we show the distributions of R_{YES_YES}(lat, z), R_{NO_NO}(lat, z), R_{YES_NO}(lat, z), and R_{NO_YES}(lat, z). From the definition (Eq. 12), it follows that for an ideal agreement, the R_{YES_NO}(lat, z) and R_{NO_YES}(lat, z) values should be equal to zero. However, Fig. 9c and d show occurrence frequencies comparable to those of Fig. 9a. From the study presented in Sect. 4.2, we expect that the ratio of R_{YES_YES}(lat, z) to CF(lat, z) should be about 0.81±0.07 if we take CALIOP as a reference for cloud detection sensitivity. If we build the CDA_norm(z) estimated from Figs. 9a and 8 (Eq. 13), we see that it fits the prescribed value up to ∼3 km (cyan curve in Fig. 5). Above this altitude, the normalized cloud detection agreement oscillates around 0.5.

Figure 9 Occurrence frequency for collocated observations: (a) both CALIOP and ALADIN detected a cloud (R_{YES_YES}(lat,z)); (b) neither CALIOP nor ALADIN detected a cloud (R_{NO_NO}(lat,z)); (c) CALIOP detected a cloud, whereas ALADIN missed a cloud (R_{YES_NO}(lat,z); and (d) CALIOP missed a cloud, whereas ALADIN detected a cloud (R_{NO_YES}(lat,z)).

The distribution of R_{YES_YES}(lat, z) (or CDA(lat, z)) shown in Fig. 9a resembles a typical cloud fraction profile plot (compare with Fig. 8). This is not surprising because R_{YES_YES}(z) must turn to CF(z) if the agreement is perfect (see Eqs. 11, 13). Even though the distribution in Fig. 9a looks physical, the ratios for the heights above 3 km are ∼40 % lower than expected from the theoretical estimates (see Fig. 5 or compare Fig. 9a with Fig. 9c). As one can see from Fig. 9c, the missing cases also form a structure which resembles the CF(lat, z) distribution. This shows that 40 % of ALADIN's $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ values are below the threshold (Eq. 9). Technically, we could fix this by lowering the detection threshold, but this would increase the R_{NO_YES}(lat, z) occurrence frequency (Fig. 9d), which is not desired.

As for R_{NO_NO}(lat, z), shown in Fig. 9b, it is close to 100 % in the high-altitude area where there are no clouds. This indicates that the false cloud detection induced by a small SNR in cloud-free area is rare for both instruments. We consider this to be a good sign as it shows the stability of the ALADIN retrieval algorithm for weak signals.

We draw the readers' attention to the fact that we did not expect the NO_YES mismatches for the considered combination of lidars at low altitude (Fig. 9d). Let us explain. The molecular extinction at 355 nm is larger than at 532 nm, and the observation geometry of ALADIN makes the optical paths $\mathrm{1}/\cos \left(\mathrm{35}{}^{\circ }\right)=\mathrm{1.22}$ times longer than those for CALIOP, where 35^∘ is a satellite viewing angle. The particulate backscatter coefficients at these wavelengths are almost the same. Therefore, for the same low-level cloud, all other factors being equal, cloud detection should be more probable for CALIOP and not for ALADIN. The typical individual profiles corresponding to NO_YES mismatches are shown in Fig. 6k–o. As one can see, despite the unfavorable observation conditions (e.g., a cloud with a peak SR(532 nm) value of ∼20 at 7 km in Fig. 7l), ALADIN retrieves one or two valid points beneath a cloud detected by both instruments. Let us consider plausible reasons for the observed behavior:

1. Since many cases of NO_YES type are over the ocean, one can rule out the continent surface echo contamination of the backscattered signal at 2 km height.

2. The horizontal cloud inhomogeneity could explain the individual cases shown in Fig. 7k–o, but it cannot explain the general behavior observed in Fig. 9d.

3. The higher detection rate in the lower layers cannot be fixed by increasing the SR threshold (Eq. 9) because it will adversely affect the agreement at other altitudes.

4. Since the $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ values in this work were recalculated from the source ALADIN data at 355 nm, the uncertainties and biases of the parameters used for recalculation (Sect. 3.4) could have biased the results. These effects would accumulate along the line of sight, so one can expect the errors to be larger near the ground.

Let us verify the last hypothesis and consider the elements of Eq. (1):

• 4a.

  α_part(λ, z) and β_part(λ, z) are retrieved from ALADIN, and their uncertainty or bias will propagate through the calculations. Moreover, a small bias in α_part(λ, z) will accumulate with distance (Eq. 1). Therefore, one cannot rule out this source of discrepancy. To explain the observed behavior, α_part(λ, z) should be biased towards smaller values, or β_part(λ, z) should be biased towards larger values in the considered layers.

• 4b.

  α_mol(λ, z) and β_mol(λ, z) are calculated with high accuracy given that the molecular density profile in CALIOP comes from the GEOS-5 DAS database; see Rienecker (2008). The uncertainties of the parameters used for their estimate are small (Bucholtz, 1995; Ciddor, 1996). Therefore, it is unlikely that they can explain the observed NO_YES cases.

• 4c.

  The physical meaning of the multiple scattering coefficient η is an increase in number of photons remaining in the lidar receiver field of view (Garnier et al., 2015). Its value depends on the type of scattering media and FOV of the lidar and varies between 0.5 and 0.8 for commonly used lidars (Chiriaco et al., 2006; Chepfer et al., 2008, 2013; Garnier et al., 2015). For ALADIN, with its narrow FOV of ∼0.02 mrad, it is usually considered to be equal to 1, but, as we explained, we used η=0.9 for the conversion. Decreasing the η value is not justified; in addition, it will increase the number of low-level clouds and NO_YES cases. On the other hand, its increase will reduce the fraction of NO_YES cases in the lower layers, but it will worsen the YES_YES agreement at the same time. Still, this parameter remains on the list of the variables which may affect the quality of the $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ conversion, and adapting the most recent model of η for ALADIN should be the next step in merging the cloud record from these two lidars.

Summarizing this section, we conclude that (a) a cloud layer detected by CALIOP is detected by ALADIN in ∼80 % of cases for cloud layers below ∼3 km and in ∼50 % of cases for higher cloud layers; (b) in the cloud-free area, the agreement between the datasets is good, indicating the stability of ALADIN L2 retrieval algorithm for weak signals; and (c) half of the cases when ALADIN detects a low-level cloud missed by CALIOP cannot be explained by sampling and geometrical differences, diurnal variation, or uncertainties in the $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ profile recalculation.

5.4 Cloud altitude detection sensitivity

We now analyze whether clouds detected by the two lidars peak at the same altitude. We note that we are not looking for an altitude offset here. The altitude detection of both instruments is beyond question. Instead, we would like to check whether the higher detection rate of lower clouds leads to slight systematic differences in the cloud altitudes derived from the two lidars. To do so, we have carried out the following analysis. For each pair of collocated profiles selected for YES_YES plot (Fig. 9a), we scanned vertically through ALADIN profile step by step, looking for a local maximum, satisfying the following conditions:

$$\begin{array}{}\text{(14)}& \text{SR}\left(i\right)>\mathrm{5};\text{SR}\left(i\right)>\text{SR}(i-\mathrm{1});\text{SR}\left(i\right)>\text{SR}(i+\mathrm{1}).\end{array}$$

For each local peak found, we have searched for a peak or for a maximal value of CALIOP's SR(z) profile in the vertical vicinity of ±3 km from the peak height determined from ALADIN. We have chosen these search limits by inspecting the collocated profiles, considering the natural variability of cloud heights at distances similar to those used in collocations. According to our analysis of CALIOP data, at these distances, ∼75 % of clouds move vertically by less than 1 km, ∼8 % by 1–2 km, ∼5 % by 2–3 km, ∼4 % by 3–4 km, ∼3 % by 4–5 km, and ∼5 % by more than 5 km. We note that by imposing the ±3 km search criteria, we filter out about 12 % of the cases linked to natural variability, which slightly reduces the number of cases selected for the analysis. At the same time, we lower the rate of picking up the peak from a different cloud layer.

We stored the differences between ALADIN's and CALIOP's cloud peak heights and then averaged them in the corresponding latitude–altitude bins (Fig. 10). As one can see, the agreement is good for the tropical high clouds. This is probably linked with thick Ci clouds, which should be reliably detected by both instruments. For the southern polar zone, this figure reveals the PSCs which are barely visible in Fig. 9a but which can be seen in Fig. 6f for ALADIN. These clouds form at very low temperatures and are partially composed of large ice particles, yielding a reflection detected at both wavelengths if the layer is thick enough (e.g., Adriani et al., 2004; Noel et al., 2008; Snels et al., 2021). However, the $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ in PSC is likely positively biased due to our conversion of ALADIN data to $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$, as discussed in Sect. 5.1.

Figure 10 Cloud altitude detection sensitivity represented as a height difference between the CALIOP local peak height and corresponding ALADIN's cloud peak height or maximal SR height found in the ±3 km vertical vicinity of CALIPSO peak. The subset corresponding to YES_YES selection (Fig. 9a) was used. Dashed white isoline corresponds to colored area in Fig. 9a (occurrence frequency of about 5 % or higher).

As one can see in Fig. 10, the higher sensitivity to low-level clouds shifts the average ALADIN cloud height downwards compared to CALIOP. At the heights of 3–5 km, the shift is as large as 0.8–1.2 km. One can attribute a part of this effect to the reasons discussed for the existence of NO_YES cases (e.g., if one assumes larger values of η, the average downward shift will be smaller, but this kind of “tweaking” would need to be justified). Summarizing, the assumption of skewing the average cloud height through higher sensitivity to lower clouds proves to be valid, and we estimate a mean downward shift to be equal to 0.5±0.6 km.

5.5 Temporal evolution of cloud detection agreement

As mentioned in Sect. 2.1, the ALADIN lidar faced several technical issues which hindered getting the planned specifications. Among them, we named the “hot pixel” issue, which requires some explanation. First, the information from the pixels is not completely lost, and there is a way of recalibrating these pixels (Weiler et al., 2021). Second, if we compare the hot pixel distribution for Mie and Rayleigh channel ACCD detectors for the period considered in this work (see Table 2 of Weiler et al., 2021), we will see 3 and 5 new hot pixels for Mie and Rayleigh matrices, respectively. For the Mie detector matrix, the lowermost hot pixel, which appeared during the considered period, corresponds to ∼15 km height. Even though these pixels do not overlap with the maxima of cloud height distributions, they still might affect the retrieval results below because of the optical path passing through the corresponding layers (see Eq. 1). As for new Rayleigh hot pixels, the lowermost two correspond to 1 km height, the next two to 5 km, and the last one to 18 km. The Rayleigh matrix pixels are not directly linked to cloud detection, but their crosstalks are used in ALADIN's α_part(λ, z) and β_part(λ, z) calculations, so they might also affect the results.

In Fig. 11a–d, we show the temporal evolution of R_{YES_YES}(time, z), R_{NO_NO}(time, z), R_{YES_NO}(time, z), and R_{NO_YES}(time, z) over the whole period of collocated dataset (28 June–31 December 2019). Figure 11e and f show the temporal evolution of CDA_norm(time, z) in two forms: as a color plot and as 2D linear fitting at the heights characterized by high occurrence frequency (0.75, 2.25, 6.25, 8.75, and 10.0 km). Unfortunately, the period available for analysis does not cover the entire year, so the plots Fig. 11a–d can be affected by seasonal variation of cloud distributions. Still, the latitudinal and longitudinal coverage of collocated data does not change throughout the year and a mixture of the Northern and Southern Hemisphere should partially compensate for seasonal anomalies. As for the ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{obs}}(\text{time},z)$ panels (Fig. 11e, f), the normalizing by CF(time, z) should compensate for the seasonal variation in these plots. Possible artifacts linked to laser power degradation, hot pixels, and bias correction would likely show up as a decrease in R_{YES_YES}(time, z) and R_{NO_NO}(time, z) occurrence frequencies (Fig. 11a, b) and as an increase in R_{YES_NO}(time, z) and R_{NO_YES}(time, z) occurrence frequencies (Fig. 11c, d).

Figure 11 Temporal evolution of occurrence frequencies for (a) R_{YES_YES}(z,time), (b) R_{NO_NO}(z,time), (c) R_{YES_NO}(z,time), and (d) R_{NO_YES}(z,time) for the period of 28 June–31 December 2019. The legend is consistent with that of Fig. 9. Dashed white vertical lines correspond to the Air Motion Vector (AMV) campaign (28 October–10 November 2019), which is characterized by smaller bin sizes and, therefore, larger SNRs for Mie and Rayleigh channels up to the height of 2250 m. (e) Normalized cloud detection agreement CDA_norm(z,time). (f) Same as (e) presented for five heights as linear fits in 2D with error bars. The error bars were estimated as root-mean-square values for 1-week chunks of altitude subsets.

However, this is not the case: visually, panels Fig. 11a–d do not show any anomaly, which would go beyond their noise levels. We note that there is a special region corresponding to a forced vertical bin size reduction in the period of 28 October–10 November 2019, which is marked by dashed white lines in Fig. 11 and which should not be considered at heights below 2250 m. To quantify the tendencies and to compare them with noise levels, we analyzed the ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{obs}}(\text{time},z)$ distributions (Fig. 11e, f). The results presented in these panels confirm the previous conclusions regarding the ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{obs}}\left(z\right)$ profile: for the clouds below 3 km, it is better than for higher ones (68±12 % at 0.75 km and 55±14 % at 2.25 km versus 34±14 %, 39±13 %, and 42±14 % at 6.75, 8.75, and 10.25 km, respectively). The uncertainty limits in these estimates are relatively large. Nevertheless, the absence of statistically significant trends indicates that the compensation for hot pixel effects (Weiler et al., 2021) and for signal losses in the emission and reception paths removes the signatures of the experimental issues from the ALADIN L2 optical products, at least down to these uncertainty limits.

6 Conclusions

The active sounders are advantageous for atmospheric and climate studies because they provide precise vertically resolved information. Building a long-term cloud profile climate record with the help of these instruments requires understanding of the differences between space-borne lidars operating at different wavelengths, flying on different orbits, and using different observation geometries, receiving paths, and detectors. In this article, we compared the ALADIN and CALIOP lidars using their scattering ratio products (CALIPSO-GOCCP and Aeolus L2A, Prototype_v3.10) for the period from 28 June to 31 December 2019. We defined the spatial collocation criterion of 1^∘ based on the averaging distance of Aeolus L2A Prototype_v3.10 data. The temporal collocation criterion of Δtime<6 h used in this work is a tradeoff between the geographical coverage of the collocated profiles, their number, and uniformity of Δtime distribution throughout the globe. With the named criteria, we found $\sim \mathrm{7.8}\times {\mathrm{10}}^{\mathrm{4}}$ collocated nighttime profiles, which underwent a series of analysis summarized here.

For an adequate comparison with CALIOP's SR(532 nm, z), we converted ALADIN's α_part(355 nm, z), β_part(355 nm, z), and SR^A(355 nm, z) to $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$, and we discussed the uncertainties of this conversion.

Before analyzing the actual observations, we performed a numerical experiment to estimate the best achievable cloud detection agreement between the two missions. We found that the agreement between ALADIN and CALIOP clouds should be about 0.81±0.07, regardless of the altitude. The numerical experiment used the outputs from a global atmospheric model coupled with a lidar simulator and a horizontal cloud variability parameterization and considered the lidar orbit, sampling, averaging, noise, and observation geometry differences in the two lidars.

Analyzing the actual observations, namely, the ALADIN dataset converted to $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ profiles and compared with the SR(532 nm, z) profiles of CALIOP, both at a vertical resolution of 1 km and horizontal resolution of 80 km, we report a good agreement in the lower atmospheric layers. Above 3 km, the agreement is worse. We explain this by lower SNR for ALADIN at these heights that is due both to physical reasons (ratio of particulate to molecular backscatter is smaller at 355 nm than at 532 nm) and technical reasons (lower emission and lower transmissivity of receiving path than planned). The PSC detection by ALADIN confirms this hypothesis: at 355 nm, the PSC backscatter is stronger than at 532 nm, which leads to high SNR and reliable cloud retrieval.

Switching from the absolute SR(532 nm, z) and $\text{SR}{}^{\prime }(\mathrm{532}\mathrm{nm},z)$ values to cloud fraction profiles obtained by applying a fixed cloud detection threshold of SR>5, the zonal mean cloud profiles of the two compared instruments show relatively good agreement, with Pearson's correlation coefficient varying from 0.7 to 0.9 and relative difference varying within ±50 % on the altitude. In the lower 3 km, the estimated ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{obs}}\left(z\right)$ profile almost reaches its theoretically estimated value ${\text{CDA}}_{\mathrm{norm}}^{\mathrm{theor}}$ of 0.81±0.07, whereas in the upper layers, its value is about 40 % less. Better detection of lower clouds skews the mean ALADIN cloud peak height in pairs of ALADIN–CALIOP profiles by $\sim \mathrm{0.5}\pm \mathrm{0.6}$ km downwards. For the reasons explained above, the agreement of PSC peak heights and of tropical high clouds does not suffer from these effects. In the cloud-free area, the agreement between two instruments is good, indicating a low rate of noise-induced false detection for both instruments.

Last but not least, the temporal evolution of cloud agreement does not reveal any statistically significant change during the considered period. This shows that hot pixels and laser energy and receiving path degradation effects in ALADIN have been mitigated, at least down to the uncertainties of the following normalized cloud detection agreement values: 68±12 %, 55±14 %, 34±14 %, 39±13 %, and 42±14 %, estimated at 0.75, 2.25, 6.75, 8.75, and 10.25 km, respectively. We believe that the provided collocated dataset will facilitate the further analysis and improvement of ALADIN L2A data. From our point of view, the outlook for a cloud product retrieved from ALADIN observations to be part of cloud lidar long records is promising: its L1 to L2 algorithms and the thresholds can be adapted to retrieve some of the same clouds as from CALIOP. This will help to better understand the instrumental and observational differences and build a long-term cloud profile climate record.