Long-term validation of Aeolus L2B wind products at Punta Arenas, Chile, and Leipzig, Germany

Ground-based observations of horizontal winds have been performed at Leipzig (51.35^∘ N, 12.43^∘ E), Germany, and at Punta Arenas (53.15^∘ S, 70.91^∘ W), Chile, in the framework of the German initiative EVAA (Experimental Validation and Assimilation of Aeolus observations) with respect to the validation of the Mie and Rayleigh wind products of Aeolus (L2B data). In Leipzig, at the Leibniz Institute for Tropospheric Research (TROPOS), radiosondes have been launched for the Aeolus overpasses on each Friday (ascending orbit) since the middle of May 2019. In Punta Arenas, scanning Doppler cloud radar observations have been performed in the framework of the DACAPO-PESO campaign (dacapo.tropos.de) for more than 3 years from the end of 2018 until the end of 2021 and could be used to validate Aeolus measurements on its ascending and descending orbits. We present two case studies and long‐term statistics of the horizontal winds derived with the ground-based reference instruments compared to Aeolus horizontal line-of-sight (HLOS) winds. The wind products of Aeolus considered are the Mie cloudy and Rayleigh clear products. It was found that the deviation of the Aeolus HLOS winds from the ground reference is usually of Gaussian shape, which allowed the use of the median bias and the scaled median absolute deviation (MAD) for the determination of the systematic and random errors of Aeolus wind products, respectively. The case study from August 2020 with impressive atmospheric conditions at Punta Arenas shows that Aeolus is able to capture strong wind speeds of up to more than 100 m s⁻¹.

The long-term validation was performed in Punta Arenas covering the period from December 2018 to November 2021 and in Leipzig from May 2019 until September 2022. This analysis showed that the systematic error of the Aeolus wind products could be significantly lowered during the mission lifetime with the changes introduced into the processing chain (different versions are called baselines). While in the early mission phase, systematic errors of more than 2 m s⁻¹ (absolute values) were observed for both wind types (Mie and Rayleigh), these biases could be reduced with the algorithm improvements, such as the introduction of the correction for temperature fluctuations at the main telescope of Aeolus (M1 temperature correction) with Baseline 09. Hence, since Baseline 10, a significant improvement in the Aeolus data was found, leading to a low systematic error (close to 0 m s⁻¹) and similar values for the midlatitudinal sites in both hemispheres. The random errors for both wind products were first decreasing with an increasing baseline but later increasing again due to performance losses of the Aeolus lidar instrument. Nevertheless, no significant increase in the systematic error in the Aeolus wind products was found. Thus, one can conclude that the uncertainty introduced by the reduced atmospheric return signal received by Aeolus mostly affects the random error.

Even when considering all the challenges during the mission, we can confirm the general validity of Aeolus observations during its lifetime. Therefore, this space explorer mission could demonstrate that it is possible to perform active wind observations from space with the applied technique.

1 Introduction

In 2018, the Aeolus satellite of the European Space Agency (ESA) was launched with the goal to improve weather forecast through global measurements of wind profiles (Stoffelen et al., 2005; Reitebuch, 2012). To obtain vertically resolved wind measurements around the globe, the high spectral-resolution (HSR) Doppler lidar ALADIN (Atmospheric Laser Doppler Instrument) was installed on board. It was the first time that a lidar with Doppler capabilities as well as with high-spectral-resolution capabilities was operated in space. Given this unique and novel space-borne technique, it is possible to actively measure vertical profiles of the line-of-sight (LOS) wind in clear sky by using air molecules as a tracer (Rayleigh methodology) and in cloudy atmospheric regions by using cloud particles as a tracer (Mie methodology; de Kloe et al., 2023; Tan et al., 2008; Baars et al., 2020b). The profiles of LOS wind velocity (measured at 35^∘ off nadir) are then projected onto a plane parallel to the Earth's surface to obtain the horizontal line-of-sight (HLOS) wind, i.e., one wind component of the horizontal wind vector (near-west–east direction). Besides this, the HSR lidar technology onboard Aeolus can be used to obtain profiles of aerosol and cloud optical properties as a spin-off product (e.g., Flament et al., 2021; Baars et al., 2021; Siomos et al., 2021; Abril-Gago et al., 2022; Gkikas et al., 2023).

The main goal of the mission has been the assimilation of the wind products into numerical weather prediction (NWP) models to demonstrate the benefit for weather forecasting (Stoffelen et al., 2006; ESA, 2008, 2023a). This has been done at several meteorological centers (Rennie et al., 2021; Rani et al., 2022; Martin et al., 2023), and a clear positive impact on forecast skills has been reported (ECMWF, 2019a, b).

Given the novelty, extensive validation efforts (calibration/validation – Cal/Val) have been needed to verify the observations, detect unforeseen challenges (instrument and processing wise), and develop respective correction or calibration updates in order to make such a data assimilation within near real time (less than 3 h) possible at all. For this reason, an intense feedback from Cal/Val teams was desired and obtained to work together with Aeolus DISC (Data, Innovation, and Science Cluster) and ESA itself on the improvement and stability of instrument and products.

Since the launch in 2018, several challenges were identified by DISC and ESA according to the feedback from the Cal/Val teams (e.g., Krisch and the Aeolus DISC, 2020). Some important issues are listed in the following:

• lower laser energy with a more rapid decline than expected (Simonelli et al., 2019; Reitebuch et al., 2020; Lux et al., 2020b),

• switch to second laser with different beam characteristics which were also changing over time (Straume et al., 2020),

• occurrence of increased background noise for some pixels (hot pixels) on the ACCD (accumulation charge-coupled devices) of ALADIN (Weiler et al., 2021a),

• changes in the wind accuracy according to differences in temperature at the main telescope mirror of ALADIN (Weiler et al., 2021b).

DISC and ESA have worked hard on these features to improve the stability of the instrument and its products which is a prerequisite for the direct assimilation. As the above-mentioned issues influence the use, e.g., the assimilation, of the Aeolus data, continuous and long-lasting validation becomes very important. Most of the operational validation of Aeolus products was performed with NWP models (using, of course, also assimilated measurements; Chen et al., 2021; Hagelin et al., 2021; Martin et al., 2021; Liu et al., 2022; Zuo et al., 2022), while a direct validation with dedicated measurements has been rare or covered only a short period and usually only a certain geographic region (Baars et al., 2020a; Witschas et al., 2020; Lux et al., 2020a; Baars et al., 2020b; Chen et al., 2021; Martin et al., 2021; Belova et al., 2021; Iwai et al., 2021; Zuo et al., 2022; Wu et al., 2022; Geiß et al., 2022; Lux et al., 2022b; Witschas et al., 2022).

The Leibniz Institute for Tropospheric Research (TROPOS) performed direct, long-term Aeolus-dedicated measurements of the wind vector at two distinct locations in the framework of the cooperation project EVAA (Experimental Validation and Assimilation of Aeolus observations; Baars et al., 2020a) between the Ludwig-Maximilians-Universität of Munich, the German Aerospace Center (DLR), the German Meteorological Service (DWD), and TROPOS. In Leipzig, Germany (51.35^∘ N, 12.43^∘ E), dedicated radiosondes have been launched for the weekly overpass on Aeolus' ascending orbit since May 2019. We analyzed this radiosonde data covering the period from June 2019 until September 2022, thus the mission period for which the second laser (FM-B) was in operation. In Punta Arenas, Chile (53.15^∘ S, 70.91^∘ W), continuous remote-sensing observations of LACROS (Radenz et al., 2021) served as one of the very rare southern hemispheric Aeolus validation sites (Ratynski et al., 2023; Zuo et al., 2022): scanning Doppler cloud radar data, which have been collected in the framework of the DACAPO-PESO campaign (Radenz et al., 2021) from November 2018 until November 2021, were used for validation activities covering the ascending and descending orbits. In addition, occasional radiosondes were launched at the local airport, but due to the low number, we have used these data only for case study validation of both Aeolus and the ground-based cloud radar observations but not for the statistical validation approach.

In the work presented here, we assessed the performance of Aeolus and its wind products (both Rayleigh clear and Mie cloudy) for a large part of its lifetime by using our long-term reference measurements. We also evaluated the potential improvements in the wind products by the introduction of new algorithm versions (baselines) into the operational retrieval chain. We thus aimed to analyze the overall performance of Aeolus and the improvements introduced by new processor versions and calibration strategies based on two single sites located in the northern and southern midlatitudes.

The paper is structured as follows: in Sect. 2, the experimental setup including the campaign locations and instrumentation is described. In Sect. 3, we explain the methodology applied to derive horizontal wind from the ground-based remote-sensing instrument and our validation strategy with respect to Aeolus. Selected case studies are discussed in Sect. 4 to explain the methodology and already show some of the potential and the limitations of Aeolus. Finally, long-term statistics are presented and discussed with respect to the different algorithm versions and the overall performance of Aeolus during its lifetime in Sects. 5 and 6. Last but not least, the conclusions drawn are presented.

2 Experimental setup

2.1 Campaign locations

Measurements at two midlatitudinal locations have been used for the validation activities described here: Leipzig, Germany, and Punta Arenas, Chile. Their locations are shown in Fig. 1 together with other ground-based stations which contribute to the validation of Aeolus.

Figure 1 Map of Aeolus Cal/Val stations. The ones considered in this work are highlighted as green stars.

2.1.1 Punta Arenas, Chile

The remote-sensing supersite LACROS (Leipzig Aerosol and Cloud Remote Observations System) was operated at Punta Arenas, Chile (53.15^∘ S, 70.91^∘ W) from November 2018 to November 2021 for the DACAPO-PESO campaign (Dynamics, Aerosol, Cloud And Precipitation Observations in the Pristine Environment of the Southern Ocean; Radenz et al., 2021). Thereby, one of the first multi-year ground-based remote-sensing data sets in the southern midlatitudes was obtained. The LACROS instrumentation comprises a PollyXT Raman polarization lidar (Engelmann et al., 2016; Baars et al., 2016), a CHM15kx ceilometer, a MIRA35 scanning cloud Doppler radar (Görsdorf et al., 2015), a HATPRO microwave radiometer, and a StreamLine Doppler lidar. Additionally, radiosondes (Lockheed Martin LMS6) could be launched at the airport of Punta Arenas for dedicated objectives.

Punta Arenas is an ideal location for the validation of Aeolus in terms of wind conditions. A strong circumpolar flow is a characteristic feature of the Southern Ocean with the southern tip of South America being the only barrier in the latitude band from 47 to 63^∘ S. Low-pressure systems embedded in this flow usually pass through the Drake Passage south of Punta Arenas causing prevailing wind directions of between southwest and northwest. A comprehensive description of the meteorological conditions is provided in Radenz et al. (2021).

Aeolus overpasses considered for the validation were on Wednesdays, at ca. 23:26 UTC on the ascending orbit, and on Thursdays, at around 09:56 UTC on the descending orbit. For the presented study, the scanning Doppler cloud radar has been used for the long-term validation and is thus explained in more detail in Sect. 2.2. The Doppler lidar performed scans for the horizontal wind as well, but due to the very low number of particles in Punta Arenas, the performance during the scans was not optimal for the Aeolus validation. Mostly, wind retrievals were restricted to the local boundary layer. But due to the relatively long distance to the Aeolus ground track (often more than 50 km) and the complex orography, the data were not useful for the Aeolus validation.

2.1.2 Leipzig, Germany

At the ACTRIS (Aerosol, Clouds and Trace Gases Research Infrastructure) site of Leipzig, Germany (51.35^∘ N, 12.43^∘ E), Aeolus Cal/Val activities were focused on dedicated radiosonde launches (see Sect. 2.2.2). These launches took place for the ascending orbit of Aeolus on Friday evening at around 16:50 UTC.

Leipzig is located in central Europe, being in the intermediate state between maritime and continental climate. Prevailing winds are usually westerlies, but due to wave activities, winds from all directions can be observed. Leipzig is located in the lowland area. There are no orographical obstacles around the city, making it a perfect location for the validation of Aeolus.

2.2 Instrumentation

2.2.1 Scanning Doppler cloud radar

Continuous measurements were conducted with a 35 GHz Doppler cloud radar of the type Metek MIRA35 (Görsdorf et al., 2015). Once per hour, the stare mode (vertical profiling) was interrupted for a range–height indicator (RHI) and plan position indicator (PPI, also called VAD – variable azimuth display) scan from minute 29 to 36 of each hour. The PPI scan started around minute 35 and lasted for 60 s and covered one full 360^∘ rotation made in 6^∘ steps. Only the PPI scans are considered for the horizontal wind retrieval, which were performed at an elevation angle of ε=85^∘. A pulse repetition frequency of 5000 Hz gives a maximum unambiguous radial velocity of 10.56 m s⁻¹, while the range resolution of 31.17 m is determined by the pulse length of 208 ns. Frequent cloud occurrence over Punta Arenas makes this instrument a perfect tool for retrievals of horizontal wind profiles, particularly during austral winter (Seifert et al., 2020; Radenz et al., 2021). The methodology for retrieving wind information from scanning Doppler remote-sensing instruments is described in more detail in Sect. 3.1.

2.2.2 Radiosonde

Radiosondes of the type Vaisala RS41 (Jauhiainen et al., 2014; Jensen et al., 2016) were launched at Leipzig each Friday for the regular Aeolus evening overpass (on its ascending orbit) since May 2019. The launch time of the radiosondes was at 16:00 UTC, thus ca. 50 min before the Aeolus overpass, to have a good coverage of the atmospheric conditions up to about 25 km. Usually, the complete ascent up to the burst height is about 2 h and thus perfectly centered around the overpass time to have the best temporal coverage possible. We therefore assume that the horizontal drift of the radiosonde does not introduce a systematic bias into our statistical validation analysis. The RS41 delivers profiles of temperature, humidity, pressure, and wind speed and direction. The uncertainty for the wind products is estimated to be between 0.4 and 1 m s⁻¹ for the wind velocity and 1^∘ for the wind direction based on calculations of the Global Climate Observing System Reference Upper-Air Network (GRUAN; Dirksen et al., 2014). Even though these estimations are based on Vaisala radiosonde type RS92, there is no significant difference in the uncertainty between both radiosonde types as they are based on the same technique to derive wind velocity and direction (Jensen et al., 2016). A bigger gap in coverage occurred during winter 2020–2021 with only sporadic radiosondes (the reason was local access restriction due to COVID-19), but in total more than 125 launches could be completed. These radiosonde profiles were not assimilated so that they can serve as an independent reference for Aeolus products.

For Aeolus overpasses at Punta Arenas, dedicated radiosondes were launched irregularly. The radiosonde type deployed in Punta Arenas was Lockheed Martin LMS6, which also delivered profiles of temperature, humidity, pressure, and wind speed and direction. In total, 41 radiosondes were launched during the 3-year campaign. Due to the irregularity of the launches, the radiosonde data were used only for case study analysis and thus also mainly for the validation of the cloud-radar-derived winds.

3 Methodology

The methodology with which Aeolus retrieves the HLOS is described in many other publications, e.g., in detail in Krisch et al. (2022) and Rennie et al. (2021) but also in Baars et al. (2020b), Weiler et al. (2021b), Chou et al. (2022), Witschas et al. (2022), Bley et al. (2022), and references therein. However, the methodology with which to retrieve horizontal wind vector from ground-based Doppler instruments, like the Doppler cloud radar at Punta Arenas, is not straightforward as several methodologies exists. Here, we give a short overview of which methods are used and, thus, how the HLOS is retrieved from the cloud radar (Sect. 3.1) and of the general validation strategy for Aeolus (Sect. 3.2).

3.1 Retrieving horizontal wind profiles from radar

While the remote-sensing instruments of TROPOS in Punta Arenas usually operated in stare mode (vertical profiling), regular PPI scans have been performed with the Doppler cloud radar to obtain the horizontal wind vector. For these scans, the radar is rotated around azimuth α with a fixed elevation angle ε (which is set to 85^∘). A sketch showing the different measurement modes is provided in Fig. 2 (left).

Figure 2 Left: sketch of the different operating modes of the Doppler cloud radar. Stare mode is used for the retrieval of vertical wind speed, while PPI scans are used to retrieve profiles of horizontal wind speed and direction. Right: scanning geometry and nomenclature for the PPI scans as used in this work. The sketch was presented in Päschke et al. (2015) under Creative Commons Attribution 3.0 License and is shown here with permission of the authors.

The measured line-of-sight (LOS) Doppler velocity v_LOS at the range R and azimuth angle α is retrieved as the mean of the measured Doppler velocities for a given range band Δr. The root mean square (rms) of this distribution is used to calculate the uncertainty in v_LOS. In the following, we use the notation that is given in Päschke et al. (2015), which is shown in Fig. 2 (right) and neglect that all variables are a function of the range R to allow better reading. The final result, of course, depends on R and thus gives a vertical profile.

In line with the standard approach of deriving horizontal wind from PPI scans (Browning and Wexler, 1968; Päschke et al., 2015), the mean horizontal wind speed v_advection can be approximated by fitting the measured v_LOS with a trigonometric function of the azimuthal coordinate of the scan corrected for positioning errors (α_corrected):

$$\begin{array}{}\text{(1)}& \begin{array}{rl}{v}_{\mathrm{projection}}\left({\mathit{\alpha }}_{\mathrm{corrected}}\right)& =v_{\mathrm{advection}}\cdot \cos \left({\mathit{\alpha }}_{\mathrm{corrected}}-{\mathit{\alpha }}_{\mathrm{wind}}\right)\\ & +B+\mathit{\sigma }.\end{array}\end{array}$$

This formula gives the horizontal wind direction α_wind and the horizontal wind speed v_advection, with v_projection being the horizontal component projected from v_LOS:

$$\begin{array}{}\text{(2)}& v_{\mathrm{projection}}={\displaystyle \frac{v_{\mathrm{LOS}}}{\cos \left(\mathit{\epsilon }\right)}}.\end{array}$$

The term σ in Eq. (1) reflects the remaining variation and is aimed to be minimized. For the fit procedure, v_advection, α_wind, and B are dependent variables, chosen to minimize σ, which is the remaining residual. The extra term B stands for the contribution of two factors to the measured Doppler velocity: the divergence in the wind field and the vertical component of the average wind velocity. Both effects are neglected within the following analysis, as it is also done for the Aeolus HLOS retrieval.

Three different fit methodologies are used to derive the horizontal wind vector. The first one is equivalent to the methodology described in Päschke et al. (2015):

• 1.

  A least square regression is applied to fit v_projection and α_corrected considering also their uncertainties. This method is subject to Doppler folding (Ray and Ziegler, 1977). This means that v_LOS that exceeds the Nyquist velocity will appear as a smaller measured velocity in the opposite direction. Usually, this effect will result in a poor fit quality with a high residual, as the measured v_LOS(α_corrected) and thus v_projection(α_corrected) will not approximate a trigonometric function and can thus be discarded.

The other two fit methods that are applied to retrieve wind velocities from the raw Doppler velocity data are based on the method by Tabary et al. (2001), which uses the approximation of an azimuthal derivative of the velocity distribution. This method is performed in two different ways:

• 2.

  The horizontal wind vector profile is retrieved from the gradient $\partial v_{\mathrm{LOS}}/\partial {\mathit{\alpha }}_{\mathrm{corrected}}$, which is approximated by overlapping piecewise linear fits centered around each point of the initial distribution as recommended by Tabary et al. (2001). This procedure is the second method and is usually consistent with the first method but may lead to higher standard deviations due to the removal of data points and the extra stages of calculation. Conversely, when Doppler folding occurs, this method is able to fit to a transformed version of the data with much higher accuracy.

• 3.

  The third method is applied because processing large numbers of linear fits as for the second method can sometimes be numerically unstable. This backup method applies the direct differences between consecutive values divided by the azimuthal distance. This approach is consistent with the former ones but leads to correspondingly higher errors because it excludes the averaging that occurs with the linear fit procedure. On the other hand, if the previous method fails to converge, this (third) Doppler-folding-safe methodology can be applied to derive the horizontal wind vector.

All three methods are performed for each range R to calculate the horizontal wind vector. In a final step, a best estimate is computed, which selects the method with the lowest error out of the three methods. In the data set, the retrieval results from all three methods plus the best estimate are stored. This best estimate is then used for the comparison with the Aeolus winds but not considering cloud-radar-derived HLOS winds with an error higher than 10 m s⁻¹.

3.2 Aeolus validation strategy

For the validation of Aeolus, we focus on the L2B wind products obtained by the Rayleigh methodology in clear air, called Rayleigh clear winds, and with the Mie methodology in clouds, called Mie cloudy winds. A more thorough description of the different products can be found, e.g., in the product description document (de Kloe et al., 2023) or in Baars et al. (2020b). If not otherwise stated, from now on we use the term “Rayleigh” for the Rayleigh clear-wind products and the term “Mie” for the Mie cloudy-wind products not considering the theoretically available Rayleigh cloudy and Mie clear winds. Rayleigh winds are delivered at 87 km horizontal resolution, while the Mie wind resolution was mainly at 15 km (setting flexible, see Table 1).

(Weiler et al., 2021a)(Weiler et al., 2021b)

Table 1 Overview of the different algorithm versions (called baseline) for the processing of the Aeolus data together with some important additional information.

We analyze all Aeolus-derived horizontal line-of-sight (HLOS) wind speeds (i.e., at different altitudes) from the Rayleigh and Mie products according to their mean coordinates (in the center of the horizontal width) that are within a radius of 100 km around the measurement site.¹ The radius used is a good compromise between the number of available comparison data and the representativeness between the two different measurements as shown in other studies (Geiß et al., 2019; Cossu et al., 2022). Accordingly, two overpasses per week for each station fulfill these conditions and have been suitable for validation:

• Punta Arenas – Wednesdays at 23:26 UTC and Thursday at 09:56 UTC;

• Leipzig – Fridays at 16:50 UTC and Sundays at 05:29 UTC.

The performed orbit shift has not changed the considered overpasses for the stations, but the mean distance from the ground site to the Aeolus wind curtains has changed. Before the orbit shift, this mean distance was 17 and 42 km on Fridays and Sundays at Leipzig and 27 and 75 km on Wednesdays and Thursdays at Punta Arenas, respectively.² After the orbit shift, it changed to 33 and 93 km at Leipzig and 75 and 85 km at Punta Arenas, respectively. As the orbit of Aeolus varies slightly and the distances given here are only mean values, some of the overpasses at Punta Arenas were outside the 100 km criterion so that we adjusted the radius to 120 km. For Leipzig this was not needed, as only Friday overpasses were considered.

For a better understanding of the general procedure, an example of the Aeolus Rayleigh wind profiles over Punta Arenas is shown in Fig. 3.

Figure 3 L2B wind curtain around Punta Arenas, Chile, on 30 September 2020 visualized with VIRES (Santillan et al., 2019). The Rayleigh wind product is shown for the whole curtain. The Mie wind product (red box) and the Rayleigh wind products (green box) 100 km around Punta Arenas are shown in the lower right.

On 30 September 2020, strong westerly winds (different shades of blue in Fig. 3) occurred over Punta Arenas at altitudes above 5 km. Closer to the South Pole, easterly winds (different shades of red in Fig. 3) prevailed. A patchy wind speed pattern was observed close to Punta Arenas near the ground, caused by cloud contamination of the Rayleigh winds. Given the example in Fig. 3, one sees that depending on the actual track of Aeolus, one to three wind profiles fulfill the criterion of being within 100 km radius of the observational site (see green box in Fig. 3). Considering a 15 km horizontal resolution for the Mie product since 5 March 2019 (before the resolution was 87 km), one can have up to 13–20 “Mie winds” for one altitude range within 100 km of the ground-based location (see red box in Fig. 3). For the validation of Aeolus products, the temporally closest ground-based cloud radar measurement has been used allowing for a maximum-time-difference threshold of 1 h. This threshold is intended to ensure similar atmospheric conditions for the validation. For the radiosonde data, such a temporal constraint is not needed as the radiosondes were launched directly for the Aeolus overpass.

Furthermore, we converted the wind speed v_ref and direction φ_ref obtained with the reference instruments (subscript ref, i.e., cloud radar and radiosonde) to Aeolus-like HLOS winds $v_{{\mathrm{ref}}_{\mathrm{HLOS}}}$ with the equation described in Baars et al. (2020b):

$$\begin{array}{}\text{(3)}& v_{{\mathrm{ref}}_{\mathrm{HLOS}}}=v_{\mathrm{ref}}\cdot \cos \left({\mathit{\phi }}_{\mathrm{Aeolus}}-{\mathit{\phi }}_{\mathrm{ref}}\right).\end{array}$$

φ_Aeolus is the azimuth angle of Aeolus, which is obtained from the Level 2B data and differs depending on global position. The uncertainties in the ground-based observations were propagated forward. The derived ground-based profiles of HLOS wind were then vertically averaged to the Aeolus-range bin thickness (500 to 2000 m, mostly higher resolution near the ground and coarse resolution at high altitudes: Stoffelen et al., 2005; Straume et al., 2020; ESA, 2020; Bley et al., 2022) to allow a one-to-one comparison. This means that we do not aim to discuss here the small-scale wind variability within the relatively large Aeolus range bins, but rather we concentrate on the performance of the space-borne instrument.

During the lifetime of Aeolus, several algorithm versions of the processing chain (so-called baselines) were released and applied: some of them in operational mode and some of them to reprocess parts of historical Aeolus data. Thus, for certain dates in the Aeolus data set, several versions exist (processed with different baselines), while for other periods only one baseline was applied. An overview of the different baselines of Aeolus covering the observational period of our ground-based reference measurements (i.e., up to autumn 2022) is given in Table 1. Two major steps for boosting the performance of Aeolus were made. With Baseline 04, the so-called hot-pixel correction (Weiler et al., 2021a) was introduced. Before that, single pixels on the ACCD of Aeolus had a higher dark current and thus biased the retrieved winds. A second important step was the introduction of a correction with respect to changes in the telescope temperature of Aeolus (M1 temperature correction; Weiler et al., 2021b). This correction was implemented with Baseline 09 and should have brought a significant improvement in the performance of the Aeolus winds.

The switch from laser FM-A to laser FM-B was performed from 12 until 28 June 2019 and led to Baseline 05. However, a new response calibration needed to be applied, which was obtained in August 2019 and led to Baseline 06. The FM-B data before that date were then reprocessed. In June 2021, the orbit of Aeolus was shifted to favor the ground-based observations in Cabo Verde during the Joint Aeolus Tropical Atlantic Campaign (JATAC; Fehr et al., 2021). Therefore, mean horizontal distances of the Aeolus wind products to the ground-based reference stations changed and as a result we increased the maximum radius for Punta Arenas to 120 km to still be able to obtain two overpasses per week (as already discussed above).

Accounting for changes in units for the uncertainties within the Aeolus products between different baselines, all values for Aeolus horizontal wind speed and errors were transformed into meter per second. Beside this unit correction, all baselines were treated equally. Furthermore, next to the provided validity flag within the Aeolus wind products, additional quality measures, i.e., error thresholds for Mie and Rayleigh winds (5 and 8 m s⁻¹, respectively), have been applied. This means that wind products flagged as valid but with an error higher than these thresholds were discarded. This approach is consistent with DISC and/or ESA recommendations (e.g., Witschas et al., 2020; Rennie and Isaksen, 2020) and studies by other Cal/Val teams (e.g., Guo et al., 2021; Chen et al., 2021; Iwai et al., 2021; Abril-Gago et al., 2023). However, it needs to be mentioned that for future validation studies of re-analyzed wind products, other quality control approaches should be regarded because the quality of the estimated error, and thus its applicability as an additional quality control parameter, will also change with a changing baseline as, e.g., discussed in Lux et al. (2022b).

For the statistical analysis presented in Sects. 5 and 6, the Rayleigh and Mie wind products were treated separately. To obtain statistical metrics, a straight-line fit between the ground-reference winds and the Aeolus winds using an orthogonal distance regression (ODR) to include the effects of errors has been computed. We also created histograms of the deviations (reference wind minus Aeolus wind) in the range of −15 to +15 m s⁻¹ (with higher velocities being assigned to the outside bins) to check for a Gaussian distribution shape. A walk-through example of the statistical analysis is given in Sect. 5 after the general validation strategy based on case studies is discussed next.

4 Case studies

To illustrate the validation strategy and discuss the potentials and drawbacks, we present two interesting case studies performed for Punta Arenas in the following.

4.1 Punta Arenas – 6 February 2020

A schematic overview of how winds are retrieved from the ground-based observations and then compared to Aeolus wind products is shown in Fig. 4 for the case of 6 February 2020, representative of the southern hemispheric summer. The atmospheric conditions above Punta Arenas on this day are presented in Fig. 4a by means of the Cloudnet target categorization (Illingworth et al., 2007; Tukiainen et al., 2020; Seifert and O'Connor, 2022) derived from the vertically staring active remote-sensing instrumentation (cloud radar and ceilometer) and the passive microwave radiometer. During this day, a nearly cloud-free aerosol layer from the ground up to 1.5 km altitude was observed with enough particles to be identified by Cloudnet target categorization (dark yellow). Partly, the cloud radar observed a return signal within this aerosol layer, which is attributed to insects (not shown). Between 2 and 8 km, clear-sky conditions (white color) were found, while ice clouds (dark gray) occurred sporadically above 8 km altitude. Horizontal wind vector observations retrieved with the Doppler cloud radar from the hourly scans at minute 35 of each hour were therefore available in atmospheric regions where clouds existed (see Fig. 4b).

Figure 4 Schematic overview of the methodology used in this study for the example of 6 February 2020 in Punta Arenas. (a) Cloudnet target categorization obtained from the combination of vertically measuring (in stare mode) cloud radar, ceilometer, and microwave radiometer. (b) Wind speed and direction indicated by wind barbs as retrieved with the Doppler cloud radar scans at minute 35 of each full hour in regions of cloud occurrence. For a better visibility, only every second wind barb is shown for measurements above 8 km. (c) Comparison of the HLOS winds from Doppler cloud radar to Aeolus products (Mie and Rayleigh, Baseline 11) for the closest overpass. GDAS model winds are shown for comparison as well.

The Aeolus overpass on this day was at 09:48 UTC indicated by the red rectangle in Fig. 4a and b. Thus, the temporally closest wind profile from the Doppler cloud radar (at 09:35 UTC) plus the HLOS profiles extracted from GDAS1³ data for 09:00 and 12:00 UTC were used for a comparison with the Aeolus products (Mie and Rayleigh) as can be seen in Fig. 4c. The closest distance between the Aeolus ground track and the ground site was 59 km on this day.

In this example comparison, the advantages and drawbacks of the reference instrument used become clear. The cloud radar is only able to retrieve winds in regions where clouds are existent (on this day between 8 and 12 km), and no information can be obtained in clear-sky regions. In regions of clouds, however, the winds can be obtained with high vertical resolution and high quality. The GDAS-derived vertical profile of HLOS is available for all atmospheric states (independent of cloud occurrence, clear sky etc.) but at a coarser resolution. As GDAS data are a result of data assimilation, the GDAS profile is no direct validation measure and therefore shown only for consistency checks and not for Aeolus validation itself.

According to Fig. 4c, the cloud-radar-derived HLOS wind taken at 09:35 UTC provides values of 27 to 31 m s⁻¹ in the cloudy region around 10 km. GDAS HLOS wind speeds in this altitude region change from about 33 to 28 m s⁻¹ from 09:00 to 12:00 UTC, respectively. Due to the good agreement of both data sources with respect to the HLOS, the validity of the cloud-radar-derived winds can be assumed. Aeolus-derived wind profiles within a radius of 100 km were available in clear air (Rayleigh winds) and at the top of clouds (Mie winds). The Mie winds available at 10 to 12 km thus indicate the presence of clouds at similar altitudes on the Aeolus track (more than 59 km away from the ground site). If the cloud deck were persistent and optically thick over the whole horizontal Aeolus track, no Aeolus winds would be available below the altitude of the clouds due to the strong light attenuation within the cloud. As this is not the case because Rayleigh winds are available down to the surface, a broken cloud deck and/or optically thin clouds between 10 and 12 km with clear sky below and above can be considered in the Aeolus observation - which is in excellent agreement with the atmospheric scene observed over Punta Arenas by LACROS (Fig. 4a).

On this specific day in austral summer 2020, a good agreement between the Mie winds and the cloud-radar-derived winds were obtained at an altitude of around 10 km. Also, the delivered Rayleigh winds in this altitude region agree well with the radar and also with GDAS. The coexistence of Rayleigh and Mie winds in one altitude range is possible; because of the broken cloud deck Rayleigh and Mie winds can coexist (Reitebuch et al., 2018; Rennie et al., 2020; de Kloe et al., 2023) and all single Aeolus products within the defined horizontal radius of 100 km are considered. In contrast, at 17 km height, GDAS and Aeolus disagree for the only Mie observation there. The reason for that is as yet unclear but might be related to atmospheric heterogeneity, uncertainties in GDAS (which are not provided), or a misclassification of Aeolus as 17 km is usually well above the local tropopause and thus no clouds are expected (and not seen in the ground observations). However, smoke occurrence from Australian bush fires at this altitude range cannot be ruled out (Ohneiser et al., 2020), but the ground-based PollyXT lidar observations do not show a significantly enhanced backscatter at this region later in the day when the clouds at around 10 km disappeared, which would explain a misclassification by Aeolus. Thus, the obviously misclassified Mie wind observation should be revisited in the context of the validation of future reprocessed data. Rayleigh winds also show deviations around 17 km towards a higher HLOS wind speed in accordance to the Mie observations.

Below the cloud deck at 10 km, Rayleigh winds partly match the model data (GDAS) but with a tendency of higher Aeolus wind speeds down to around 5 km altitude. Sporadically, lower Rayleigh HLOS wind speeds were also observed. Deviations within the lowest 3 km might be caused by horizontal inhomogeneity within the 100 km radius around the ground-based station. For the statistics presented below in Sects. 5 and 6, we use wind derived with the Doppler cloud radar and compare it to the equivalent Aeolus HLOS winds. For the example case presented here, this means that a comparison to Mie winds is possible for the height range around about 11 km, as this is the only region for which cloud-radar-derived winds and Aeolus Mie winds coexist. Rayleigh wind comparisons can be done at the same height range (reference measurements from the cloud radar). The regions between the ground and 9 km altitude and above 11 km cannot be covered for the comparison due to the missing ground-based measurement data. Radiosondes could cover this gap, but they were launched at Punta Arenas only irregularly for the Aeolus validation so that a meaningful long-term validation is not possible. We did not aim for validation with model data, as this is done regularly at ECMWF (Rennie and Isaksen, 2020; Rennie et al., 2021) and by other validation activities (e.g., Martin et al., 2021; Liu et al., 2022; Chen et al., 2021; Hagelin et al., 2021; Rani et al., 2022). Instead, we concentrate on the direct measurements made from the ground.

4.2 Punta Arenas – 18 August 2021

The second case study from Punta Arenas presents an observation from the southern hemispheric winter. On this day, besides the Doppler radar, a local radiosonde launch was also available.

The atmospheric conditions on this day were remarkable as presented in Fig. 5. While typical HLOS wind speeds between 5 and 20 m s⁻¹ were observed in the troposphere, a steady increase in wind speed was observed above the tropopause (at ca. 9.5 km) leading to a maximum wind speed measured by the radiosonde of more than 100 m s⁻¹. The Aeolus-derived wind speed profile (overpass at 23:27 UTC, with the closest distance of 47 km to Punta Arenas) is in agreement with the GDAS (21:00 and 00:00 UTC) and the radiosonde data (launched 23:00 UTC) and thus gives confidence of its reality. The horizontal extent of the strong winds around Punta Arenas can be seen in Fig. 5 (bottom, right), which shows the Aeolus Rayleigh HLOS wind speed. It demonstrates the potential of Aeolus to detect such features.

Figure 5 Comparison of radiosonde, Doppler cloud radar, and GDAS HLOS winds to Aeolus products on 18 August 2021 in southern hemispheric winter conditions. Radiosonde vertical error bars indicate the mean wind speed of the radiosonde averaged to the Aeolus range bins; Aeolus vertical error bars indicate the respective range-bin extent. The Aeolus Rayleigh wind curtain for the analyzed overpass as visualized with VIRES (Santillan et al., 2019) is shown in the lower right.

The comparison of Aeolus-derived HLOS winds with the radiosonde, cloud radar (23:35 UTC), and GDAS is shown in Fig. 5 and reveals that the Rayleigh product above 10 km follows the observed radiosonde wind profile and GDAS products. A considerable deviation was observed only at the topmost Aeolus range bin (around 24 km). There, the wind speed measured by Aeolus was considerably lower compared to the radiosonde wind. The reason is not yet clear but might be simply due to the strong drift of the radiosonde because of the high wind speed. Below the tropopause at ca. 9.5 km, the Aeolus Mie winds and Rayleigh winds agreed mostly with the radiosonde. The GDAS data, however, show a significant variation at 8 km between the two profiles at 21:00 and 00:00 UTC. This behavior implies that fast changes in HLOS, i.e., in wind speed and direction, have taken place in this atmospheric region. It furthermore gives confidence that the 1 h time window for validation is a reasonable time period for the wind validation.

Usually, two Aeolus Rayleigh wind observations (of 87 km horizontal length in the case of Rayleigh) per Aeolus height bin exist in the validation radius of 100 km around the ground station when using the center coordinates of the wind products and are thus considered for the validation. Having a look at the HLOS observations at around 4.7 km, one sees that one wind product of Aeolus fits the reference wind profiles very well, while the other one shows considerable deviations (around 10 m s⁻¹ lower HLOS). This finding implies regional variations in the wind pattern. As a consequence, the observed outliers in the Aeolus Rayleigh winds at 3 and 4.7 km can be attributed to horizontal (and thus also temporal) heterogeneity in the wind field. This behavior shows the difficulty in comparing Aeolus HLOS winds to the ground-based observations because a perfect co-location in space and time can usually never be achieved. However, we consider that these meteorological variations do not lead to additional biases in the statistics presented in Sects. 5 and 6 but are properly covered by the statistical methodologies in terms of random error.

5 Example of statistical validation: Baseline 11 at Leipzig and Punta Arenas

To obtain statistical measures for the performance of Aeolus and its algorithms, we analyzed the Aeolus HLOS data by Doppler cloud radar and radiosonde (from now on called reference instruments) as described above for the locations of Leipzig, Germany, and Punta Arenas, Chile. To illustrate that approach, the validation of Aeolus Baseline 11 products around Punta Arenas is shown in Fig. 6. The direct comparison (panels a and c) and the frequency distribution of the deviation from the Doppler cloud radar values (reference minus Aeolus, panels b and d) are shown for the Mie (panels a and b) and Rayleigh winds (panels c and d). A generally good agreement can be seen between Aeolus and the cloud radar, being most of the time close to the one-to-one line and thus justifying the use of the orthogonal distance regression (ODR) for fitting the data.

Figure 6 Long-term wind statistic for the Baseline 11 wind validation based on ground-based Doppler cloud radar observations at Punta Arenas (ascending and descending orbits). (a, b) Aeolus Mie wind; (c, d) Rayleigh wind; (a, c) 1 : 1 statistic with respective measures (median and mean bias, scaled median absolute deviation (SMAD), standard deviation (SD) provided in m s⁻¹, and N indicating the number of samples); (b, d) frequency distribution of differences between the two data sources.

More data points could be evaluated for the Mie winds (in total 1780) than for the Rayleigh winds (629), which is a logical consequence of the higher resolution of the Mie winds and the fact that the cloud radar derives winds only in regions with cloud occurrence. Thus, the validation is more meaningful with respect to Mie winds.

For the Mie wind, we obtained a slope of 1.1 with the ODR. When forcing the slope to be unity, the resulting offset is equal to the mean bias as expected for a Gaussian distribution. A median bias of −0.41 m s⁻¹ and a mean bias of −0.17 m s⁻¹ was derived (i.e., Aeolus measures less than the ground-based reference) together with a standard deviation of 6.6 m s⁻¹ and a scaled median absolute deviation (MAD) of 5.1 m s⁻¹. For the Rayleigh wind validation, we obtained respective values of 1.05 (slope), 0.1 m s⁻¹ (mean bias), 0 m s⁻¹ (median bias), 8.9 m s⁻¹ (standard deviation), and 6.8 m s⁻¹ (scaled MAD); see the statistics box in Fig. 6a and c.

In the following, we use the scaled MAD as an indicator of the random error, analogous to the median bias for the systematic error often just called bias. The median values are less sensitive to outliers than the mean values but are a valid measure for the uncertainties as long as the frequency distribution is of Gaussian shape. The philosophy of the use of the scaled MAD for Aeolus comparisons is explained in detail in Martin et al. (2021), Lux et al. (2022a), and Weiler et al. (2021a).

We also performed a Z-score analysis as described in Lux et al. (2022b) and found that, for example, with a Z value of 3 for Baseline 10, 1.5 % of the Aeolus values are identified as outliers. However, in this publication we do not want to discuss the outliers of Aeolus wind products but rather the performance of the publicly available wind data as a whole. Thus, we do not exclude outliers based on the Z-score analysis but make a validation of the complete Aeolus data set by applying the recommended error thresholds of 8 and 5 m s⁻¹ for Rayleigh and Mie winds, respectively.

The frequency distribution of the differences (Fig. 6b, d) shows a near-Gaussian shape, giving confidence that the statistical measures described above can be applied. The obtained systematic errors of −0.41 and 0 m s⁻¹ and the random errors of 5 and 7 m s⁻¹ for Mie and Rayleigh products, respectively, for Baseline 11 validation at Punta Arenas are in line with results from other validation activities for this baseline (Zuo et al., 2022; Geiß et al., 2022). An overview of the main key numbers from this statistic is given in Table 2. We performed the same statistical analysis with the radiosonde data from Leipzig for Baseline 11. The results are presented in Fig. 7. Given the fact that the radiosonde delivers wind data in both clear and cloudy skies, it becomes clear that this reference instrument is well suited for the Aeolus Rayleigh and Mie wind validation. As radiosondes are not limited to certain atmospheric targets, a coverage up to 25 km height could usually be achieved, allowing us to validate all HLOS winds during an Aeolus overpass. Therefore, results are not confined to the cloud-laden troposphere like in Punta Arenas. Thus, the statistical analysis is more representative in terms of data points for the Rayleigh and Mie wind as can be seen in Fig. 7a and c. In total, more than 1500 and 2000 data points could be used for validating the Aeolus Mie and Rayleigh winds, respectively. The frequency distributions of the difference between the reference and Aeolus HLOS winds are of Gaussian shape for both Mie and Rayleigh winds and thus give again evidence for the validity of the applied statistical validation approach. The direct comparison (Fig. 7a, c) shows a generally good agreement with only sporadic outliers (e.g., ≈ 50 m s⁻¹ in the Aeolus Rayleigh wind product, while the radiosonde delivered less than 10 m s⁻¹). The majority of data points is, however, near the 1 : 1 line and thus in good agreement. For the Mie winds, we obtained similar values as for Punta Arenas in the Southern Hemisphere, with a systematic error of −0.4 m s⁻¹ and a random error of 4.6 m s⁻¹. For the Rayleigh winds, we obtained a median bias of −0.5 m s⁻¹ and a random error similar to the one for Punta Arenas with 5.8 m s⁻¹. Given the fact that more data points are available, the retrieved Rayleigh systematic error for Leipzig is more meaningful, even though one has to consider that latitudinal and longitudinal dependencies of the systematic error have been discovered (Martin et al., 2021; Weiler et al., 2021b) and thus single locations like Leipzig and Punta Arenas are not completely representative of the overall global performance of Aeolus. The so-called orbital bias was partly resolved with the M1 temperature correction (Weiler et al., 2021b). However, a leftover effect in this orbital bias oscillation cannot be ruled out. Especially, if one considers that for Leipzig we could only evaluate the ascending orbit, while for Punta Arenas we evaluated both orbit types.

Figure 7 Long-term wind statistic for the Baseline 11 wind validation based on radiosonde launches at Leipzig. (a, b) Aeolus Mie wind; (c, d) Rayleigh wind; (a, c) 1 : 1 statistic with respective measures (median and mean bias, scaled median absolute deviation (SMAD), standard deviation (SD) provided in m s⁻¹, and N indicating the number of samples); (b, d) frequency distribution of differences between the two data sources. Only observations on the ascending orbit node of Aeolus were considered.

Table 2 Overview of the metrics obtained for the validation of Baseline 11 at Punta Arenas (a, all orbit types) and Leipzig (b, ascending orbit).

We also performed a radiosonde-based validation for Punta Arenas, but too few data points from the very few radiosonde launches matching the evaluation criteria have been available so that the results are not statistically significant. We thus do not further discuss this specific validation methodology for Punta Arenas with respect to the long-term analysis. A final overview of the obtained metrics for the validation of Baseline 11 is shown in Table 2. The same methodology has been applied to the other baselines and will be discussed below.

6 Aeolus validation

We performed the validation analysis for Punta Arenas (Doppler cloud radar) and Leipzig (radiosonde) for all available baselines and thus for the time periods listed in Table 1. In Sect. 6.1, we analyze different product versions covering the same time period to assess the changes in product quality with a changing baseline. Periods marked with red, light blue, magenta, and dark yellow rectangles in Fig. 8 are very appropriate for a baseline intercomparison. For the period marked in red from June–September 2019, products from four different algorithm versions are available covering already the FM-B era. The time period marked with dark yellow dashes comprises baselines 06, 10, and 11 from July to October 2019 and the period marked with magenta dashes represents the comparison period for B07 to B11, which covers the time from November 2019 to April 2020. The periods marked in light blue reach from June until December 2019 and from May to October 2020 and cover two different algorithm versions. In Sect. 6.2 we use the latest algorithm version (baseline) available for the analyzed time period to discuss the performance of the instrument during its lifetime.

Figure 8 Overview of the different baselines (algorithm versions) that were used to process Aeolus data. The rectangles indicate the periods which have been used for the baseline comparison. The red rectangle refers to Sect. 6.1.1, the dark yellow dashed one to Sect. 6.1.2, the magenta dashed one to Sect. 6.1.3, and the light-blue ones to Sect. 6.1.4.

6.1 Comparison of baselines

Due to the reprocessing efforts of the Aeolus team, there are certain periods in which Aeolus data are available for different baselines, as shown in Fig. 8. This allows the validation of the improvements between the different baselines using the same reference data. However, a quantitative measure is not straightforward as due to quality control (QC) procedures, etc., it is not the same number of Aeolus wind data that are available. Nevertheless, such a comparison gives a first inside into the improvement made by introducing new baselines.

6.1.1 B05, B06, B10, and B11 comparison

To start with the analysis of the different baselines, we focus on the period from July to September 2019 for which data from four different baselines are available: B05, B06, B10, and B11 – see red rectangle in Fig. 8. The switch to laser FM-B had been already performed at this time. We analyzed this period with the reference data from Punta Arenas and Leipzig. The results for Punta Arenas are shown in Fig. 9 for the Mie (top) and Rayleigh (bottom) wind products.

Figure 9 Aeolus performance as obtained by comparison to ground-based Doppler cloud radar at Punta Arenas for baselines 05, 06, 10, and 11 in the period from July to September 2019 (ascending and descending orbits). Plots analogous to Figs. 6a and c and 7a and c.

As this period lasted 2 months, 16 overpasses in the southern hemispheric winter could be covered. The greatest differences can be seen for the Rayleigh winds. For Baseline 05, many outliers (data not close to the 1 : 1 line) have been observed, mainly at negative HLOS speeds, which led to a bias of −8 m s⁻¹ for B05. The random error is as high as 20 m s⁻¹ for the Rayleigh winds for this baseline (B05). The poorly estimated error product of the Aeolus Rayleigh winds at this baseline might have led to invalid winds observations being flagged as valid. For the Mie winds, systematic and random errors of −3 and 7 m s⁻¹ were found, respectively. With Baseline 06, the overall performance was improved at the cost of less valid data. This indicates an already introduced improvement by the new response calibration which was needed since the switch to laser FM-B but could only be obtained in August 2019 and thus 2 months after the switch. The fact that less valid Aeolus data were available might be caused by improved or stricter quality flags and error calculations (more wind observations flagged as invalid). Finally, fewer outliers are found for Rayleigh winds as seen in Fig. 9 (B06, Rayleigh). At this baseline (B06), the systematic and random errors were as low as 0.4 (1.9) and 5.1 (7.7) m s⁻¹ for Mie (Rayleigh) winds, respectively. Of course, these numbers have to be assessed with care due to the low number of compared overpasses. Nevertheless, when looking at the Rayleigh wind metrics of Baseline 10, a significant improvement is found while a slightly higher number of data are also available. The introduction of the M1 temperature correction with Baseline 09 seems to have significantly improved the Rayleigh winds. Biases of 0.9 and 1.4 m s⁻¹ have been detected with random errors of about 6 m s⁻¹ for both B10 and B11, respectively. Here, too, it holds that these numbers have to be treated with care due to the relatively low number of data, but it definitely shows that algorithms have significantly improved and systematic and random errors are at a good level to allow the use of the Rayleigh wind products.

For the Mie winds, the improvement in performance is less evident compared to the Rayleigh products, caused by the fact that Mie winds were already much more reliable for B06 (bias of 0.4 m s⁻¹ and random error of 5.1 m s⁻¹). The number of available measurements also stayed nearly constant. The reason for having already reliable Mie winds for B06 is that according to Weiler et al. (2021b), the Mie winds are 10 times less affected by the M1 temperature variations than the Rayleigh products due to the technical nature of the different detection schemes (e.g., also Reitebuch, 2012).

With the introduction of B10 and B11, the median bias stays below absolute values of 1 m s⁻¹ with random errors of about 5 m s⁻¹ for the Mie products. For both wind types, the difference between B10 and B11 itself is less significant, most probably because of the low number of data that could be used for the comparison. An intense discussion on the B10 to B11 comparison is done later in Sect. 6.1.4 for a longer time period.

For Leipzig, a similar but not identical behavior was observed, as shown in Fig. 10. The Mie systematic errors for all compared baselines had absolute values below 1.2 m s⁻¹. The magnitude of the random error for the Mie winds was improved but not as significantly as over Punta Arenas. In contrast, for the Rayleigh winds the bias was high at Baseline 05 (> 8 m s⁻¹ in absolute values) and could be significantly reduced until B11 (< 0.2 m s⁻¹). The major step forward concerning the random error for the Rayleigh winds has been found since B06 due to the new response calibration function, leading to a decrease from 15 to 4 m s⁻¹. Interestingly, for the B05 Rayleigh winds (Fig. 10, lower, left), a divided distribution was found with much stronger negative HLOS winds of Aeolus compared to the radiosonde reference. This behavior has completely vanished since Baseline 06.

Figure 10 Aeolus performance as obtained by comparison to radiosonde launches at Leipzig (ascending orbit only) for baselines 05, 06, 10, and 11 in the period from July to August 2019. Plots analogous to Figs. 6a and c and 7a and c.

6.1.2 B06 to B10 comparison

As shown in Fig. 11, the improvement for B10 Rayleigh winds compared to B06 products becomes even more evident if one compares the longer period available for B06 and B10 only (see a dark yellow dashed rectangle in Fig. 8). Here, the longer time period from July to the end of October 2019 could be considered, covering 34 overpasses (17 for each orbit type – ascending and descending). Furthermore, the comparison focuses on the data quality depending on the orbit types, i.e., whether Aeolus measured on an ascending or a descending orbit, to estimate the effect of the M1 temperature correction introduced since B09 (Weiler et al., 2021b). Therefore, we also did not include the Leipzig data (on ascending orbits only) here. There are obvious differences in the performance of B06 between ascending and descending orbit types for the Rayleigh and Mie winds, e.g., a systematic error for the Rayleigh wind products of −0.22 m s⁻¹ on the ascending orbit vs. 2.54 m s⁻¹ on the descending orbit. The majority of the outliers is seen for Mie winds on the descending orbit. These outliers in the Mie winds remain partly in Baseline 10, so that one can conclude that there must be other reasons for the discrepancy in the Mie wind measurement than the temperature deviation at the Aeolus telescope (Weiler et al., 2021b). For example, it might be atmospheric inhomogeneity which led to the result of Aeolus measuring about −45 m s⁻¹ but the reference instrument only about −30 m s⁻¹.

Figure 11 Comparison of Baseline 06 to Baseline 10 at Punta Arenas based on ground-based Doppler cloud radar observations separated for ascending and descending orbit types for the period of July to October 2019. Plots analogous to Figs. 6a and c and 7a and c.

For B10, a systematic (random) error of 3.4 (6.2) and −0.8 (7.9) m s⁻¹ was observed for the ascending and descending Rayleigh winds, respectively, still indicating a different behavior between the different orbit types. The same is valid for the Mie winds, with systematic errors of about 1.2 m s⁻¹ (ascending) and −0.75 m s⁻¹ (descending), while the random error is similar with around 6 m s⁻¹.

6.1.3 B07 to B11 comparison

We also analyzed the difference between B07 and B11 (see dashed magenta rectangle in Fig. 8). The data set used covers Punta Arenas and Leipzig data from November 2019 to April 2020, i.e., southern hemispheric summer and northern hemispheric winter conditions. It is therefore well suited to statistically analyze the influence of the implementation of the M1 telescope temperature correction on the data quality.

According to the results presented in Fig. 12, it becomes evident that between B07 and B11, like for B06 to B10, a significant improvement has to be attributed to the Rayleigh wind performance with many fewer outliers at both locations. Stated for Punta Arenas only, as an example (Fig. 12, left, bottom), a lower systematic error (−3 vs. −0.9 m s⁻¹) and almost identical random errors (about 7.8 m s⁻¹) were found for the Rayleigh winds. For the Mie wind performance at Punta Arenas (Fig. 12, left, top), the systematic error improved from −0.6 m s⁻¹ to near 0, while the random error stayed similar with less than 5 m s⁻¹ with a number of available observations that was even increasing (295 to 393). The number of available observations of the Mie winds also increased at Leipzig (Fig. 12, right, top) but with no significant changes in the errors for either Mie or Rayleigh (Fig. 12, right) wind products.

Figure 12 Aeolus performance at the reference stations of Punta Arenas (left; cloud radar, ascending and descending orbits) and Leipzig (right; radiosondes, ascending orbit) for baselines 07 and 11 in the period from November 2019 to April 2020. Plots analogous to Figs. 6a and c and 7a and c.

As the separation between the ascending and descending orbits is one key element for identifying the orbital bias effect, we separated the statistics according to that for Punta Arenas observations only and discarded the Leipzig data set (available for the ascending orbit type only). The results are shown in Fig. 13. Without going into too much detail, we could not identify a significant improvement in the performance with respect to the Mie wind product in this specific data set covering 5 months of observations. Based on around 150 data points, the Mie systematic error was +0.5 m s⁻¹ for B07 on the ascending orbit, while it was almost −3 m s⁻¹ on the descending orbit and thus remarkably different. With Baseline 11, however, the differences in the bias have even increased: +1.5 m s⁻¹ for the ascending orbit and −3.6 m s⁻¹ for the descending orbit, both for the Mie wind product. Thus, based on this limited data set, no improvement was found for the Mie winds. The Rayleigh winds, however, had a similar bias of around −3 m s⁻¹ for B07, which significantly decreased to 0.1 and −1.2 m s⁻¹ (ascending and descending, respectively) for B11.

Figure 13 Comparison of Baseline 07 to Baseline 11 at Punta Arenas based on cloud radar observations separated for ascending and descending orbit types for the period from December 2019 to April 2020. Plots analogous to Figs. 6a and c and 7a and c.

These comparisons discussed above are an indicator for the improvements made between different baselines – especially the importance of the implementation of the M1 temperature correction for the Rayleigh wind product. They are not meant as a general statement on the Aeolus performance as the analyzed time period is short. A significantly large data set is key to determine statistically significant measures for a single validation site, like Leipzig or Punta Arenas. Therefore, the product performance is analyzed between B10 and B11 on a longer time series in the following.

6.1.4 B10 to B11 comparison

As stated above, one major improvement step was reached by the introduction of the M1 telescope temperature correction with Baseline 09. Thus, it is also of interest to compare the algorithm versions beyond this baseline. This is possible for B10 and B11, for which a significant number of data are available in parallel as seen in Fig. 8 (light blue rectangles). The most important differences between B10 and B11 are the implementation of the satellite line-of-sight (Satellite-LOS) velocity correction, the reporting of the Rayleigh spot location and width values, and different signal-to-noise ratio (SNR) thresholds for the classification of Mie and Rayleigh (ESA, 2023b).

However, according to Fig. 14 (left), no significant difference can be found at Punta Arenas between B10 and B11 for either Mie or Rayleigh wind products, despite the fact that about 5 % more Mie winds are available, which is most probably due to the new SNR thresholds for the wind type classification. In fact, the performance of the Rayleigh and Mie winds is slightly worse (overall a small increase in systematic and random errors). We can only speculate about the reasons for this, but it might be due to the new wind type classification or the newly implemented Satellite-LOS velocity correction. Similar findings are made for Leipzig (Fig. 14, right), for which the radiosondes could cover a much larger height range compared to the cloud radar observations in Punta Arenas but only for the ascending orbit. Here, the absolute bias has slightly decreased from −0.44 to −0.29 m s⁻¹ for the Mie winds with a similar random error, but there are more measurements for B11 as also observed for Punta Arenas. For Rayleigh winds, no difference at all is seen, giving confidence that for this wind type Baseline 10 was already working well over the atmospheric range from 0 to 25 km – at least on Aeolus' ascending orbit over central Europe. If one separates the orbit types for the statistical analysis, which is possible for Punta Arenas (Fig. 15), it is interesting to note that a significant difference in the bias still occurs between the two orbit types for both baselines. With respect to the comparison of the Mie winds on the ascending orbit between B10 to B11, the bias decreased, while on the descending orbit it increased (in terms of magnitude) from −1.7 to −2.4 m s⁻¹. For Rayleigh winds, also like in Leipzig, no significant difference is seen in the geographic region of Punta Arenas between the two baselines but with significant differences between the orbit types (> 2 m s⁻¹ vs. ≈ −2 m s⁻¹). The random error remained identical between the baselines for both wind types.

Figure 14 Aeolus performance at the reference stations of Punta Arenas (left; cloud radar, ascending and descending orbits) and Leipzig (right; radiosondes, ascending orbit only) for baselines 10 and 11 in the period from July 2019 to October 2020. Plots analogous to Figs. 6a and c and 7a and c.

Figure 15 Comparison of Baseline 10 to Baseline 11 at Punta Arenas based on cloud radar observations separated for ascending and descending orbit types for the period from July 2019 to October 2020. Plots analogous to Figs. 6a and c and 7a and c.

6.2 Error evolution during lifetime

In the following, we assess the long-term performance of Aeolus. Thus, it is a mix of instrument performance and algorithm improvements. Figure 16 shows the temporal evolution of the systematic error (median bias, panels a and b) and random error (scaled MAD, panels c and d) for the Mie (panels a and c) and Rayleigh (panels b and d) products for the full 3-year data set at Punta Arenas. The temporal evolution was computed by using a 50 d moving average window, i.e., 7 full weeks with 14 Aeolus overpasses (seven for each orbit type). The newest baseline release was used for this analysis; i.e., for periods for which several baselines coexist, the one with the highest number was analyzed (i.e., B02: December 2018–May 2019; B03: May–June 2019; B11: June 2019–May 2021; B12: June–November 2021; B13: December 2021–March 2022; B14: April–September 2022). The results are shown for all validation measurements (blue line) and are split into ascending (gray–green) and descending orbits (orange). Note that we also present Aeolus data here which are not yet public, i.e., from the very early mission time, and thus this should not be regarded as final performance indicator of Aeolus. At the southern hemispheric midlatitude location of Punta Arenas, the systematic error in the Mie wind decreased from around 3 m s⁻¹ in the beginning to almost 0 m s⁻¹ since May/June 2019 (for the combined observations including both orbit types; blue color). However, a difference between the wind products of the separate two orbit types (orange: descending; gray–green: ascending) becomes obvious especially for the period between October 2019 and August 2020. Sporadic outliers like in September 2020 might be due to certain weather conditions in Punta Arenas. The increase at the end of the observational period in 2021 might be attributed to the orbit shift performed for Aeolus in June 2021 and the resulting larger distances to the validation site. We also had to increase the radius from 100 to 120 km to still be able to validate both orbit types. Thus, the significant increase in magnitude in the systematic error might be attributed to the increased distance (mean distance 75 and 85 km compared to 27 and 75 km before the orbit shift). The random error in the Mie winds at Punta Arenas varies between 2 and 10 m s⁻¹ but with most of the higher values after the orbit shift. The increase in random error since the beginning of 2019 might be attributed to the reduced return signal with laser FM-A and the calibration procedures after the laser switch (e.g., Parrinello et al., 2022). Additionally, the random error on the descending orbit shows a significant increase since the orbit shift.

Figure 16 Long-term evolution of the derived systematic error (a, b) and random error (c, d) for the Aeolus Mie (a, c) and Rayleigh (b, d) wind products obtained at the Cal/Val station of Punta Arenas with cloud radar for all Aeolus observations (blue) and separated by orbit type (ascending: gray–green; descending: orange). A 50 d moving average window was applied.

For the Rayleigh winds, a significant improvement in terms of bias can be seen shortly after the start of the observations. Afterwards, the systematic error in the Rayleigh wind product seems to fluctuate between −5 and +5 m s⁻¹ during the whole analyzed period, which might be an indicator for a reduced meaningfulness of the reference observations, which are available only in cloudy atmospheric regions, compared to the Rayleigh winds available only in clear-air regions for a 50 d averaging window. Nevertheless, generally the systematic error for the descending orbit is mostly negative, while the one for the ascending orbit is mainly positive. Therefore, the total retrieved bias fluctuates between positive and negative values. The random error in the Rayleigh winds has significantly improved over the course of the mission lifetime from more than 10 m s⁻¹ in the beginning of the observations to values of around 5 m s⁻¹ in the middle of the analyzed period. After the orbit shift in June 2021, the magnitude of the systematic and random errors increased for all orbit types. Detailed reasons are as yet unclear but might be simply attributed to the larger distances between Aeolus and the ground-reference instruments after the orbit change. The decrease in random error before the orbit shift is in contrast to expectations due to the parallel decrease in the atmospheric return signal of Aeolus and to other validation studies (e.g., Martin et al., 2021; Bley et al., 2022; Ratynski et al., 2023). However, published results of systematic and random error trends of Aeolus wind products are rare, and to our knowledge, the only trend series in the southern hemispheric midlatitudes is provided in this study. Thus, the opposite trend to what was expected might but not need to be attributed to atmospheric conditions.

A similar analysis was made for the Leipzig data set, which covers the ascending orbit only but therefore is available until the beginning of autumn 2022 (and thus includes baselines 13 and 14, completely). The results are shown in Fig. 17. Similarly to Punta Arenas, the temporal evolution of the systematic and random errors in the Mie and Rayleigh products for Leipzig has been analyzed by means of the median bias and the scaled MAD, respectively. A 50 d moving average window was applied; i.e., one smoothing window contained seven overpasses, as the ascending orbit is analyzed only for Leipzig.

Figure 17 Temporal evolution of the derived systematic error (blue – median bias) and random error (orange – scaled MAD) for the Aeolus Mie (a) and Rayleigh (b) wind products obtained at the Cal/Val station of Leipzig (ascending orbit) with radiosondes. A 50 d moving average window was applied.

In accordance with the analysis for Punta Arenas before the orbit shift, the analysis for this location reveals that the systematic error in the Mie wind product was close to 0 m s⁻¹ for the entire FM-B period (from June 2019 until September 2022). The random error for the Mie products was stable at values around 4 to 5 m s⁻¹ until the middle of 2021. Afterwards, it decreased significantly followed by an increase and stabilization at 3 to 4 m s⁻¹ until the end of the analysis period. Please note the observational gap which occurred during winter 2020–2021 and spring 2021 due to COVID-19 restrictions. Thus, no radiosonde (reference) data were available.

For the Rayleigh winds, a positive performance trend was also observed. The magnitude of the systematic error decreased significantly from values of around 4 m s⁻¹ to magnitude values below 2 m s⁻¹. The random error also decreased until the end of 2019 due to performance improvements obtained with the switch to laser FM-B but later mainly increased as a result of the decreasing return signal at Aeolus (e.g., Parrinello et al., 2022). The Rayleigh random errors at Leipzig, however, always stayed below 10 m s⁻¹. Partially, a decrease in random error and thus an increase in performance was found as, for example, around January 2022, mainly caused by short phases of increasing return signal due to laser energy and alignment improvements. Similar overall trends in terms of random error as for the northern hemispheric midlatitude station of Leipzig were reported by Martin et al. (2021) (Northern Hemisphere from 23.5 to 65^∘ N) and Bley et al. (2022) (tropical stratosphere).

6.3 Validation summary

We performed a validation analysis for both Aeolus wind products (Mie and Rayleigh winds) for the period for which our reference observations (Doppler cloud radar and radiosonde) were available. We thus considered several different baselines (see Table 1). The main results in terms of systematic and random errors for Punta Arenas (Doppler radar) and Leipzig (radiosonde) are summarized in Table 3.

Table 3 Overview of the systematic error (median bias) and the random error (scaled MAD) for the different baselines of Aeolus derived with the reference measurements at Punta Arenas (cloud radar, ascending and descending orbits) and Leipzig (radiosondes, ascending orbit). All values are in meters per second.

According to Table 3 (left half), the systematic error in the Aeolus wind products could be significantly lowered with the changes introduced into the processing chain (different baselines). While in the early mission phase, systematic errors of more than 2 m s⁻¹ (absolute values) were observed for both wind types, these biases could be reduced with the algorithm improvements, such as new calibration procedures or the M1 temperature correction with Baseline 09. Hence, since Baseline 10, a significant improvement in the Aeolus data was found leading to a low bias (close to 0 m s⁻¹) for the Rayleigh winds and nearly similar values for the midlatitudinal sites in both hemispheres. The systematic error in the Mie winds was already significantly reduced with Baseline 06. The random errors for the wind products, as shown in Table 3 (right part), first decrease with increasing baseline but later increase again as a result of the performance losses of the lidar instrument onboard Aeolus (Parrinello et al., 2022). But this mainly affects only the Rayleigh winds. The systematic error is only slightly affected by this issue, so one can conclude that the uncertainty introduced by the reduced atmospheric return signal received by Aeolus mostly affects the random error – of course at the cost of having less valid wind data, but at least no significant additional bias seems to be introduced.

7 Conclusions

To validate the novel wind lidar mission Aeolus, we have gathered long-term validation data at two midlatitudinal sites but in different hemispheres. More specifically, we have performed regular radiosonde launches for the weekly Aeolus overpasses at Leipzig, Germany (51.35^∘ N, 12.43^∘ E), since May 2019. We also operated a scanning Doppler cloud radar in Punta Arenas, Chile (53.15^∘ S, 70.91^∘ W), so that horizontal wind speed and direction could be retrieved in the vicinity of clouds. We used all these data sources to validate the overall Aeolus performance with respect to mission time, the algorithm (baseline) version applied to Aeolus data, and the orbit type (ascending, descending, both). It was found that the deviation of the Aeolus HLOS winds from the ground reference is of Gaussian shape. As a systematic error indicator we thus applied the median bias of this distribution, while the random error was attributed to the scaled median absolute deviation in accordance to previous validation work on Aeolus and in agreement within ESA and DISC (e.g., Lux et al., 2020a). It should be noted that while the radiosonde data collected at Leipzig on Aeolus' ascending orbit covers the whole atmosphere from the ground to ca. 22–25 km height, the Punta Arenas reference measurements with cloud radar are restricted to cloudy regions in the troposphere; i.e., the results from Punta Arenas do not represent stratospheric wind observations. Nevertheless, as clouds above Punta Arenas can be generally found at all altitudes and can be found frequently, we regard the troposphere as being well covered. The main findings, i.e., the systematic and random errors by baseline, have been summarized in Table 3. In general, we have found an improving performance of the HLOS wind products with respect to the baseline development. This effect was however partly masked by the effect of lower instrumental performance of Aeolus during its lifetime, especially for the random error. From the whole Aeolus lifetime, we mainly analyzed the period that was conducted with the spare laser called FM-B (starting with Baseline 05). Even when considering the issues with the emitted laser energy and the lower-than-expected received atmospheric return signal (e.g., Parrinello et al., 2022), which constantly decreased despite many efforts made, we can confirm the general validity of Aeolus observations during the lifetime. The systematic error in both wind products (Rayleigh and Mie) has significantly decreased as a result of newly introduced baselines with new calibrations and corrections. While at the beginning of the mission, absolute values as high as 5 m s⁻¹ were observed for the systematic error, it was continuously reduced to values close to 0 m s⁻¹ before the public release of the Aeolus data in April 2020. This proves the general concept of this Earth explorer mission to perform active wind observations from space. The random error was indeed higher than requested by the mission requirements, but compared to the loss in return signal, the performance of Aeolus has still been in a range bringing a significant benefit for the numerical weather forecast as demonstrated, e.g., at ECMWF (Rennie et al., 2021), DWD (Martin et al., 2023), and NCMRWF (Rani et al., 2022). The data set gathered at Punta Arenas, Chile, and Leipzig, Germany, in the course of the validation project EVAA will continue to be of high value for the Aeolus mission. It can, for example, further be used to validate new algorithm versions applied to historical Aeolus data or to test new methodological approaches. Such efforts will continue even after the satellite has stopped measuring and will help to foster potential follow-on activities for active wind measurements from space, as is currently planned.