Global measurements of the optical properties of atmospheric particles, encompassing aerosols and clouds, play a pivotal role in evaluating their climatic effects through atmospheric radiation processes. Zhang et al. (2022) analyzed the simulation results of the Coupled Model Intercomparison Project Phase 6 (CMIP6) global models and indicated that 20 % of radiative effect uncertainty for anthropogenic aerosols was associated with aerosol vertical distribution under clear-sky condition. In addition, the presence of highly light-absorbing aerosols such as black carbon above or below the cloud layer can prodoundly the radiative transfer process of the upper atmosphere (Takemura et al., 2002; Oikawa et al., 2013, 2018). The Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite, equipped with the two-wavelength (1064, 532 nm) polarization Mie-scattering lidar CALIOP (Cloud-Aerosol LIdar with Orthogonal Polarization), has globally measured aerosols and cloud vertical distributions over the long term from 2006–2023 (Winker et al., 2010). This role will transition to ATLID (Atmospheric Lidar) (Heìliere et al., 2017; do Carmo et al., 2021), the 355 nm high-spectral resolution lidar (HSRL) with polarization measurement function onboard the EarthCARE satellite (Earth Cloud Aerosol and Radiation Explorer). EarthCARE is a joint Japan-Europe satellite observation mission by JAXA (Japan Aerospace Exploration Agency), NICT (National Institute of Information and Communications Technology), and ESA (European Space Agency), which aims to conduct comprehensive global observations of clouds, aerosols, and atmospheric radiation (Wehr et al., 2023). The EarthCARE satellite features a 94 GHz cloud radar (CPR) with doppler measurement capability (Nakatsuka et al., 2023), a multiwavelength imager (MSI), comprising seven channels in the visible to infrared wavelength range (Wallace et al., 2016), and a broadband radiometer (BBR) designed for measuring shortwave and longwave radiation (Wallace et al., 2016). The scientific objectives emcompass (1) observing the global vertical distributions of natural and anthropogenic aerosols and their interaction with clouds; (2) observing global cloud distributions, cloud-precipitation interactions, and vertical motion characteristics within clouds; (3) evaluating the vertical profiles of radiative heating and cooling of the atmosphere (Illingworth et al., 2015; Okamoto et al., 2024a).

ATLID, a HSRL, can independently measure backscattered light from atmospheric particles (Mie signal) and backscattered light from atmospheric molecules (Rayleigh signal) separately, distinguishing it from CALIOP, a Mie-scattering lidar. This unique feature allows for the independent extraction of the vertical distributions of the extinction coefficient and backscattering coefficients of atmospheric particles by analyzing these two signals. Moreover, ATLID conducts polarization measurements, enabling the extraction of the co-polar component, parallel to the laser polarization (co-polar component), the cross-polar component (perpendicular to the laser polarization) of the Mie signal at a wavelength of 355 nm, and the Rayleigh signal. Signal calibrations are performed on these measured signals, resulting in calibrated attenuated backscatter coefficients.

Multichannel lidar observations using a combination of polarization measurements, Rayleigh/Mie separation measurements, and/or various wavelength measurements, including those of ATLID, CALIOP, and ground-based lidar measurements, facilitate simultaneous understanding of various optical and microphysical properties of atmospheric particles. This includes the identification of major atmospheric layers such as aerosol or cloud layers (Okamoto et al., 2008; Vaughan et al., 2009; Liu et al., 2009; Hagihara et al., 2010) and the identification of aerosol and cloud types such as marine, dust, warm water, or 2-D ice (Omar et al., 2009; Yoshida et al., 2010; Illingworth et al., 2015; Okamoto et al., 2019). The size distribution and refractive index of total aerosols can be estimated using extinction coefficients and backscatter coefficients at two wavelengths (355 and 532 nm) observed by the Raman lidar and HSRL (e.g., Müller et al., 2014). The extinction coefficients of some major aerosol components (such as mineral dust and black carbon) have been estimated (Nishizawa et al., 2017; Kudo et al., 2023). Understanding of the climate impact of aerosols, necessitates not only comprehending the optical and microphysical properties of total aerosols but also those of individual aerosol components. For example, aerosols with strong light absorption properties (e.g., black carbon) have been reported to influence cloud formation and global atmospheric and water circulations in large fields (Menon et al., 2002; Koren et al., 2004). In addition, the top height of the planetary boundary layer (PBL) has been detected by various methods such as the gradient method (Lammert and Bösenberg, 2006), the wavelet covariance transform (WCT) method (Brooks, 2003), or other methods (e.g., the standard deviation method by Menut et al., 1999). Since aerosols emitted from the surface are trapped within the PBL because of the temperature inversion between the PBL and free troposphere, its top height, by which the transition point of the aerosol concentration gap is characterized, is an important parameter for understanding climate and air quality.

The EarthCARE mission generates Level 1 (L1) products, which are physical quantities derived after calibrating the measurements of each instrument, and Level 2 (L2) products, consisting of geophysical variables related to clouds, aerosols, etc, utilising L1 data either independenally or in combination (Eisinger et al., 2024). Each sensor development organization (JAXA and NICT for CPR, and ESA for others) generates the L1 product, and the L2 product is generated by individual agencies. For example, L2 products using ATLID, such as layer identifiers, particle (aerosol, cloud) type identifiers, and optical properties, such as extinction coefficient, backscatter coefficient, and depolarization ratio, and cloud top height, are estimated by ESA (Irbah et al., 2023; Donovan et al., 2023, 2024; Zadelhoff et al., 2023; Wandinger et al., 2023). Synergy products are also generated by combining L1 data from the other instruments. JAXA also provides various ATLID standalone and synergistic products, while similar to ESA products, are produced using different independent retrieval methods developed independently.

To determine the globally vertical distribution of optical properties of aerosols and clouds needed to assess their climatic effects, we have developed atmospheric particle retrieval algorithms to generate JAXA L2 products using ATLID L1 data. In this study, we focus on the retrieval algorithms and products related to aerosols, whereas the description of cloud property estimation is planned for separate papers. Section 2 describes the products retrieved using the algorithms and their flows. Section 3 describes the algorithms developed to retrieve the individual products. Section 4 presents the products estimated using the simulated input data (ATLID L1 data) and the performance of the various algorithms. In Sect. 5, we summarize and discuss the prospects.

Figure 1 illustrates the products and flow of the algorithms. First, to improve signal quality, the algorithm reduces the signal noise using a discrete wavelet transform (DWT) (Fang and Huang, 2004). In this study, two types of Daubechies wavelets (support numbers 2 (D2) and 4 (D4)) are employed. Noise reduction is performed iteratively with D2, followed by D4, and then the noise reduction using D2 is applied again, followed by D4 and so on (D2=>D4=>D2=>D4=>D2…). A simulation analysis with added random noise indicated that the applying the DWT method could improve the signal-to-noise ratio (SNR) by a factor of two or more. Therefore, this method was adopted in the present study.

To further enhance the SNR, we integrate the data horizontally after applying the DWT method to the minimum horizontal resolution (∼0.3km) data from ATLID L1. This integration yields data at a 1 km horizontal resolution (ATLID L1(1) in Fig. 1). Additionally, a moving average of this 1 km horizontal resolution data with a width of 10 km was produced (ATLID L1 (1*) in Fig. 1). Consequently, the algorithm uses the three L1 data created (ATLID L1 (0.3), (1), and (1*) in Fig. 1), with an altitude resolution of 0.1 km, akin to the original data, as input data when estimating each product. A quality check (QC) involving the SNR is performed on each of the generated data.

Six different products (FM (Feature Mask), AOP (Aerosol Optical Properties), COP (Cloud Optical Properties), ATM (Aerosol Target Mask), CTM (Cloud Target Mask), PBLH (Planetary Boundary Layer Height) in Fig. 1) are created from the ATLID L1 data. First, the algorithm identifies which component of atmospheric molecules, aerosols, or cloud particles is predominantly present at each altitude, and outputs its identifier (FM). Additionally, the surface, subsurface, and fully attenuated layers are identified here. Secondly, the particle optical properties (POP), that is, the extinction coefficient (αp), backscatter coefficient (βp), depolarization ratio (δp), and lidar ratio (Sp) at 355 nm, are retrieved from the ATLID L1 data. Whether the derived POP are of aerosol or cloud origin is determined according to the FM product (AOP or COP). Aerosol (ATM) and cloud (CTM) particle types are also identified at each altitude. The PBLH is then estimated. The product derived from original 0.3 km data is only the FM because of the low SNR. To account for cloud heterogeneity and the low SNR of the aerosol layers, the cloud products (COP and CTM) and PBLH are derived from the 1 km horizontal resolution data. The aerosol and cloud products (AOP, ATM, COP, and CTM) and PBLH are created from the 1* km horizontal resolution data. In the following section, products and algorithms related to aerosols (i.e., FM, AOP, ATM, PBLH) are mainly described. To estimate these products, the optical properties of atmospheric molecules, such as extinction and backscatter coefficients (MOP in Fig. 1) are required. Here, the EarthCARE auxiliary product, containing a subset of meteorological fields from an ECMWF forecast model (Eisinger et al., 2024) is used to compute the molecular optical propertes.

To demonstrate the products and the performance of the algorithms, the ATLID L1 data (i.e., Eq. 1a–c) simulated using the Joint Simulator for Satellite Sensors (Joint-Simulator) (Hashino et al., 2013, Satoh et al., 2016, Roh et al., 2023a) were used. Cloud (and precipitation) distributions simulated using the Nonhydorstatic Icosahedral Atmospheric Model (NICAM; Satoh et al., 2014) and aerosol distributions simulated using NICAM Spectral Radiation Transport Model for Aerosol Species (NICAM-SPRINTARS; Takemura et al., 2000) were used as the input data. Signal noises such as shot noise, dark noise, and CCD read-out noise expected from the ATLID system were evaluated and added as gaussian random noise.

The ATLID L1 data for the cloud-predominant case simulated along the satellite path using Joint-Simulator are shown in Fig. 3. The cloud and aerosol fields used for the signal simulation are also shown in Fig. 4. Clouds are present from the surface to the altitude of 15 km. The SNR of the Mie co-polar- and cross-polar attenuated backscatter coefficients for the cloud layers are generally greater than 5 for 0.3 km horizontal resolution data; the SNR for 1 km horizontal resolution data is approximately twice than that for 0.3 km horizontal resolution data. The FM algorithm (Sect. 3.1) was applied to the simulated L1 data (Fig. 5). To assess the effect of signal noise on the estimates, the algorithm was applied to L1 data with or without signal noise, and the agreement between them was evaluated. The results for the 0.3 and 1 km horizontal resolution data showed generally good agreement for each layer type, such as “cloud” and “clear-sky or aerosol.” For the 0.3 km horizontal resolution data, the total number of layers for the cloud type was 226 000, of which the number of misidentification was 24 000, corresponding to a 11 % relative error. For the 1 km horizontal resolution data, the misidentification of the cloud layers was 9 %. For the clear-sky or aerosol type, the misidentification reached to 41 % for the 0.3 km horizontal resolution data; however, it improved to 5 % for the 1 km horizontal resolution data. Many misidentification were noted at the edges of the cloud layers, highlighting the need for improvement is this aspect in future studies.

The results of the aerosol layer identification are shown in Fig. 7. Here, we used the the L1 data simulated for the aerosol-predominant case, where the dust layer is widely suspended over an altitude range of 3–20 km (Fig. 6), and the FM algorithm was applied to the L1 data with or without noise. The results for the data with signal noise are generally in good agreement with those for the data without signal noise. The total number of aerosol layers was approximately 280 000 and the number of misidentified layers was 30 000, resulting in 11 % misidentification. As in the cloud-predominant case (Fig. 5), much of the misidentification occurred at the edges of the aerosol layers, which is also an aspect that can be improved for aerosol identification.

The results of the AOP retrieval are shown in Fig. 8. The AOP retrieval algorithm was applied to the aerosol-predominant case, as shown in Fig. 6. The SNRs of the L1 data for the dust layer generally ranged from 5–20 for 1* km horizontal resolution data. The estimated backscatter coefficients and depolarization ratios generally agreed well with the actual values, which were the aerosol optical properties used in the L1 simulation. The mean of retrieved aerosol backscatter coefficient was 3.26×10-7m-1sr-1. The true value was 3.18×10-7m-1sr-1 and the mean error (ME: retrieval–truth) was 0.08×10-7m-1sr-1, corresponding to a relative error of 2 %. The root mean square error (RMSE) was 1.12×10-7m-1sr-1, corresponding to a relative error of 34 %. For the depolarization ratio, the means for the retrievals and true values were 0.27 and 0.26, respectively; ME=0.01 (4 %) and RMSE=0.07 (27 %). The figure also shows that the extinction coefficient and lidar ratio were more strongly affected by the signal noise than the backscatter coefficient and depolarization ratio. For the extinction coefficient, the means for the retrieval and true values were 1.32×10-5 and 1.35×10-5m-1, respectively; ME=-0.03m-1 (2 %) and RMSE=1.05×10-5m-1 (78 %). For the lidar ratio, the means of the retrieval and true values were 41 sr-1, respectively; ME=0sr-1 (0 %) and RMSE=25sr-1 (61 %). As with the backscatter coefficient and depolarization ratio, there was no significant bias error (ME) for the extinction coefficient and lidar ratio; however, the variation (RMSE) was relatively large owing to signal noise. Notably, the aerosol concentration used in this analysis was relatively low (i.e., 1.35×10-5m-1 on an average), and the RMSE value of the extinction coefficient is not large (i.e., 1.05×10-5m-1). For cases with higher aerosol concentrations, it can be expected that the relative error of the extinction coefficient and the RMSE of the lidar ratio decrease with an increase in the SNR of the Mie-attenuated backscatter coefficient (e.g., Nishizawa et al., 2017).

The results of the aerosol-type classification are shown in Fig. 9. The algorithm was applied to the L1 data with and without signal noise for the aerosol-predominant case, as shown in Fig. 6. For the dust layer, there appeared some misclassification (e.g., Dusty-mixture and Unknown); however, there was a fair amount of agreement. The total number of layers classified as Dust type was approximately 225 000 and the number of misidentified layers was 84 000, resulting in a 37 % misidentification. In addition, aerosol layers classified as Pristine type were also found below an altitude of 4 km; the total number of pristine layers was approximately 13 000 and the number of misidentified layers was 9700, resulting in 74 % misidentification. Since aerosol typing was classified using a two-dimensional diagram of the lidar and depolarization ratios (Fig. 2), aerosol type misclassification is directly reflected by the errors in the lidar ratio and depolarization ratio. The error in the lidar ratio is considerably larger than that in the depolarization ratio (about 25 sr in RMSE). Therefore, the retrieved lidar ratios for the dust layer fall outside the range of the two-dimensional diagram for “dust”, resulting in cases of misclassification as dusty-mixture or unknown. Similarly, pristine is misclassified as pollution or marine due to the errors in the lidar ratio. Thus, accurate estimation of the lidar ratio as well as depolarization ratio is required for aerosol type classification. For cases with higher aerosol concentrations, it can be expected that the misclassification of aerosol types decreases with an decrease of the retrieval error of the lidar ratio, as discussed in the AOP retrieval (Sect. 4).

Figure 10 shows an example of the PBLH detection for the L1 data simulated using Joint-Simulator, where the PBLH was approximately 2.1 km. The WCT was calculated using a Haar function with a dilation width of 1.0 km, and a WCT peak exceeding a threshold of 0.2, derived at 2.1 km. When signal noise was added to the L1 data (dotted line in Fig. 10), another WCT peak at ∼1km appeared because of signal fluctuation; however, the peak was below the threshold. In this case, the estimated PBLH with noise agreed well with that without noise. In actual observations, the PBLH detection may be difficult for cases of noisy data, low aerosol concentrations, and the existence of residual layers at night because the gap of BR′ for the PBL and free troposphere may be small.

We developed algorithms to produce JAXA ATLID L2 products for aerosols and clouds using ATLID L1 data. The algorithms retrieve the following products: (1) feature mask; (2) particle optical properties such as extinction coefficient, backscatter coefficient, depolarization ratio, and lidar ratio at 355 nm; (3) target mask; and (4) planetary boundary layer height. The performance of the algorithms was demonstrated using pseudo-ATLID L1 data with realistic signal noise, which were calculated according to the ATLID specification for the aerosol or cloud dominated cases by Joint-Simulator.

The main findings of this simulation study are as follows: (1) the misidentification of the aerosol and cloud layers by the feature mask algorithm was relatively low (approximately 10 %). (2) The retrieval errors of the AOP were 0.08×10-7±1.12×10-7m-1sr-1 (2±34% relative error) for the backscatter coefficient and 0.01±0.07 (4±27% relative error) for the depolarization ratio of aerosols; the relative errors of the extinction coefficient and lidar ratio were worse than those of the backscatter coefficient and depolarization ratio. (3) The aerosol-type classification generally performed well, with 37 % misclassifications for the dust type, although there were more misclassifications for the pristine type. These misidentifications were mainly due to errors in the estimation of the lidar ratios. Accurate estimation of the lidar ratio as well as depolarization ratio is essential for aerosol type classification. (4) PBLH retrieval using the WCT method with ATLID L1 data was feasible. These results indicate that the algorithm's capability to provide valuable insights into the global distribution of aerosols and clouds, facilitating assessments of their climate impact through atmospheric radiation processes.

The validation of these aerosol and cloud products is essential, and JAXA and ESA will conduct validation using various platforms, such as airborne, shipborne, and ground-based measurements, and various observation instruments, including lidars. The validation studies include assuarance of the product accuracy and the improvement of the L2 algorithms mentioned above, including the verification and improvement of various assumptions (e.g., optical models of aerosols and clouds) and the criteria used in the algorithms.

To understand the climatic effects of aerosols, it is essential to understand the properties of individual aerosol components as well as the properties of total aerosols such as the JAXA ATLID L2 products described above. Therefore, we are currently developing an algorithm to estimate the extinction coefficients of dust, sea salt, carbonaceous (light-absorbing particles), and water-soluble aerosols at 355 nm using the difference in the depolarization and light absorption properties of each aerosol component from the L2 products and ATLID L1 data. We are also currently developing an ATLID-MSI synergy algorithm to retrieve the vertical mean mode-radii of dust and fine-mode aerosols, and the extinction coefficients of the abovementioned four aerosol components. These algorithms are being developed based on the aerosol component retrieval algorithms that have been developed for the analysis of CALIOP and ground-based lidar data. (Kudo et al., 2023; Nishizawa et al., 2007, 2008, 2011, 2017). The aerosol component products derived by these algorithms using ATLID and MSI data are planned to be released as JAXA's L2 products and are expected to enhance the scientific value of the EarthCARE mission.