On average, clouds cover about 67 % of the Earth's surface and therefore largely impact the global radiation and energy budgets determining our climate. With regard to the Earth's energy budget, clouds have both a cooling and a warming effect. On the one hand, the cooling effect originates from the reflection of the incoming shortwave radiation from the sun back to space, which is determined by the optical properties of the clouds. These optical properties depend on the phase of the cloud (pure liquid, ice or mixed phase) and the shape and size of its particles. On the other hand, clouds absorb longwave radiation originating from the Earth's surface while emitting at lower temperatures, which results in a greenhouse effect in the atmosphere. Since temperature decreases with height in the Earth's troposphere, the greenhouse effect increases with cloud top height. Therefore, exact knowledge of the cloud top height is important to determine the impact of clouds on the longwave radiation budget of the Earth.

As clouds form almost anywhere around the globe and appear in a large variety of cloud types (from optically thick cumulonimbus to thin cirrus clouds), their exact characterization is important to resolve their impact both on our daily weather and on the long-term climate. But resolving clouds in numerical weather prediction and climate models is limited due to their high spatial and temporal variability, requiring high computational costs. Thus, most models rely on cloud parametrizations, which are often based on measurement studies, making the observational characterization of clouds important e.g.,.

In recent decades, much effort has been made to better understand clouds and their feedback mechanisms to climate change, both from the modeling and the observational sides. In particular, a reduction in the uncertainty of the global cloud feedback has been accomplished, which is now likely expected to be positive with high confidence, as indicated by the most recent IPCC report . This was achieved by evaluating the regional feedbacks of clouds separately. For this, extensively studying the interaction between clouds, circulation and climate is indispensable . Airborne field campaigns, such as the Next-Generation Aircraft Remote Sensing for Validation NARVAL I and II; and the Elucidating the role of clouds–circulation coupling in climate (EUREC4A) field campaigns , took place in the vicinity of Barbados to classify the macro- and microphysical properties of trade-wind cumuli. Other campaigns, such as the Arctic Cloud Observations Using Airborne Measurements during Polar Day (ACLOUD); the Physical Feedbacks of Arctic Boundary Layer, Sea Ice, Cloud and Aerosol (PASCAL) both described in; and the recent Arctic Air Mass Transformations During Warm Air Intrusions and Marine Cold Air Outbreaks (HALO-(AC)3) campaign, were conducted for the characterization of clouds in the Arctic and their role in the Arctic Amplification. During the NARVAL expeditions, EUREC4A and HALO-(AC)3, the German High Altitude and LOng range research aircraft HALO; was operated as a cloud observatory . On board HALO, the spectrometer of the Munich Aerosol Cloud Scanner specMACS; provides wide-field and spatially highly resolved radiance measurements from which both cloud top heights and cloud optical properties can be obtained. The instrument originally consisted of two hyperspectral line cameras covering the wavelength range between 400 and 2500 nm but has been extended by two polarization-resolving RGB cameras (Phoenix 5.0 MP polarization model) prior to the EUREC4A campaign . The wide combined field of view of about 90°×120° of the two cameras allows for deriving cloud top heights and cloud droplet size distributions for a large area from spatially highly resolved intensity measurements at resolutions of 10–20 m at usual flight altitudes of 10 km . Moreover, the cameras provide simultaneous measurements at a frame rate of 8 Hz, resulting in a high temporal resolution. With the intensity measurements of the two RGB cameras, the cloud top heights are derived using a stereographic reconstruction method of the cloud geometry described by for data of the 2-D RGB camera installed prior to the EUREC4A campaign. To summarize, the algorithm relies on the identification of points on the cloud surface using contrast gradients. The reidentification of detected points in subsequent images and the associated observation from different perspectives enable localization in 3-D space. The polarization measurements allow us to determine cloud droplet size distributions in terms of the effective radii and effective variances of liquid water clouds derived from observations of the cloudbow . Hereby, the dependency of the polarized scattering phase function of water clouds in the scattering angle that ranges between 135 and 165°, the region of the cloudbow, on the size distribution of the cloud droplets is used to determine the effective radius and variance of the observed cloud targets. The cloud targets are defined as clusters of 10×10 cloudy pixels and thus have an approximate size of 100 m × 100 m depending on the actual distance to the cloud. The cloud targets are observed from multiple viewing angles in subsequent images while flying over the clouds. The retrieval is described in and consists of three steps which are shortly summarized in the following. First, possible cloud targets which are observed in the cloudbow region are identified. Then, the identified cloud targets are located in 3-D space using the corresponding cloud top heights from the stereographic reconstruction algorithm. Hence, an accurate determination of the cloud top heights is important as small errors in the cloud top height will lead to large localization errors in the cloud targets in subsequent images. In the case of inaccurate cloud top height data, the cloudbow signal will be wrongly aggregated, which in turn leads to errors in the derived cloud droplet size distribution. This further motivates the accuracy assessment of the cloud top height retrieval performed in this paper. Finally, pre-calculated polarized scattering phase functions from Mie theory are fitted against the polarized radiance measurements, and the best fit determines the effective radius and variance of the cloud target.

Similar techniques have been successfully applied to several space- and airborne instruments, such as the Polarization and Directionality of the Earth's Reflectances POLDER;, the Research Scanning Polarimeter RSP;, the Airborne Hyper-Angular Rainbow Polarimeter AirHARP;, and the Airborne Multiangle SpectroPolarimetric Imager AirMSPI;.

As both cloud top heights and cloud optical properties determine the radiative properties of clouds and their feedback with regard to climate change, it is important to accurately measure those properties. In this study, the benefits of realistic 3-D radiative transfer simulations generated with the Monte Carlo code for the physically correct tracing of photons in cloudy atmospheres MYSTIC; are exploited to evaluate the retrieval results and determine their accuracies. To do so, the usage of simulations is important as they rely on fully self-consistent cloud and radiation fields, while, for example, comparisons to other instruments always depend on the different sensitivities. Moreover, it is often hard to find suitable measurements, and even for large measurement campaigns such as EUREC4A with coordinated flights of remote sensing and in situ aircraft, simultaneous measurements of the same cloud and, in particular, its cloud top are rare. Furthermore, model simulations allow for separating the different error sources since one has control over all model variables. For example, the investigation of the influence of the cloud development during the aircraft overpass is possible by assuming either a realistically evolving or a temporally constant cloud field.

The benefits of radiative transfer simulations for the accuracy assessment of cloud droplet size retrievals have also been used by , who performed various tests on simplified 1-D and realistic 3-D radiative transfer simulations of polarized reflectance measurements of the RSP instrument. For example, it was studied how aerosol layers of different optical thicknesses above the cloud layer and the presence of multiple cloud layers affect the RSP retrieval. Although the retrieval algorithms for RSP and specMACS are based on the same principals, namely multi-angle observations of the cloudbow and the fit of the polarized phase functions to the observations, the retrievals differ from each other because of the different properties of the instruments: while RSP is an along-track scanning instrument with defined viewing angles and nine spectral channels , the polarization cameras of specMACS measure 2-D images , which allows for the retrieval of the cloud droplet size distribution for broader parts of the clouds. However, this comes at the cost of broader spectral response functions, which might influence the accuracy of the polarimetric cloud droplet size distribution retrieval, and will be tested with this work. In contrast to , who mainly used the 865 nm wavelength for the simulations, we applied the full spectral response functions of the cameras with central wavelengths (bandwidths) of approximately 620 nm (66 nm), 546 nm (117 nm) and 468 nm (82 nm) . Further, the whole retrieval procedure, including the identification of possible cloud targets and the geolocalization based on the stereographic cloud top heights, will be tested.

The polarimetric cloudbow retrieval was further studied in . In this work, large-eddy simulations (LESs) are used in combination with 1-D radiative transfer simulations for the comparison of cloud droplet size distributions derived from the bispectral Moderate Resolution Imaging Spectroradiometer (MODIS) retrieval and the polarimetric retrieval from the Polarization and Directionality of the Earth's Reflectances (POLDER) instrument. Further, simulated POLDER data were used by to investigate the influence of cloud sub-grid variability on the POLDER-derived cloud effective radius and variance.

In our study, the wide-field and highly resolved 2-D measurements of specMACS are simulated based on a realistic field of shallow cumulus clouds as observed during the EUREC4A campaign. The cloud data were obtained from LESs using the PALM model . This allows us to apply the stereographic reconstruction and the cloudbow algorithm to the simulated measurements and compare the results to the respective quantities determined by the model cloud field used for the simulations. Although it is well known that the signal of the cloudbow originates from single scattering and hence is weighted by exp⁡(-τ), with τ being the optical thickness , we show that it is not straightforward to obtain the corresponding true model quantities. Nevertheless, the accuracy of the retrievals can be assessed, and possible error sources can be quantified. This allows for a deeper understanding of (multi-angle polarized) observations and their importance for the characterization of the clouds' microphysics.

To evaluate the accuracy of the retrieval algorithms, a 1 min overflight of HALO over a LES-simulated shallow cumulus cloud field, as frequently observed during the EUREC4A campaign in the vicinity of Barbados in early 2020, was simulated. Highly resolved LESs were performed using the PALM model . Within the 60 s overflight, the field of view of a single specMACS camera covers an approximate area of 32×21 km2. Hence, a large cloud field of 25.6×12.8 km2 horizontal extent was simulated for a duration of 2 min with a second-by-second output at a horizontal grid size of 10×10 m2 to match the high spatial resolution of the two polarization cameras of specMACS. The vertical resolution was set to 5 m with up to 2 km height, which is approximately the height of the cloud tops. Above, the resolution is reduced until a resolution of approximately 15 m is reached at an altitude of 3 km. The LESs were initialized by dropsonde measurements from 28 January 2020 during the EUREC4A campaign. On that day, wide cloud patterns of shallow cumuli were observed . In Fig. a, the vertical wind profiles of the horizontal wind components u and v as well as the horizontal wind speed and its direction are shown for the first time step used for the simulation of the specMACS measurement. Within the minute of the simulated overflight, the wind profiles do not change significantly and hence are representative of the horizontal movement of the clouds.

Following , PALM uses a bulk two-moment liquid-phase cloud microphysics scheme of , providing the cloud droplet number concentration (N) and specific water content (LWC). In our setup, an extended scheme following , , , and was used. For the MYSTIC simulations, the cloud microphysics need to be described in terms of the liquid water content (LWC), the effective radius (reff) and the effective variance (veff). While the LWC is directly retrieved from the LES model output, the reff is derived from the model variables following : 1reff=3⋅LWC4π⋅k⋅N⋅ρ1/3×10-6. Here, N is the water droplet density in cubic meters and comes from the LES model output, and ρ is the water density (1000 kg m-3). k is the ratio between the volume mean radius rv=(∫n(r)r3dr/N)1/3 and the effective radius each to the third power: k=rv3/reff3. For maritime air masses, determined k=0.80±0.07, hence k=0.80 was chosen for the calculation of the effective radius from the LES data following Eq. (). The resulting distribution of effective radii for the first simulated time can be seen in Fig. b. For all other times, the distribution looks similar (not shown here). For the radiative transfer simulations, we assumed a constant effective variance of veff=0.1 to determine the optical properties of the clouds. Since k and veff are related by k=(1-veff)(1-2veff) for modified gamma distributions e.g.,, this corresponds to a value of k=0.72. However, as k is only used to derive an effective radius which is not provided by the LES model, the simulations are internally consistent, and the choice of k does not impact the analysis of this paper.

As shown by for marine stratocumulus clouds, the radiative effects of a cloud are sufficiently well represented in 3-D radiative transfer models if the spatial resolution of the model input resolves the mean free photon path l of the clouds, which is given by the inverse of the extinction coefficient l=kext-1. For the clouds obtained from the LES model, the mean free photon path was roughly estimated to be of the order of 20 m, which is comparable to the 20–30 m stated by for overcast marine stratocumulus clouds. Therefore, it was decided to reduce the horizontal resolution of the grid for the computationally expensive radiative transfer simulations by a factor of 2 such that the grid size is 20×20 m2, while the vertical resolution was reduced by a factor of 5 to about 25 m. In spite of the eventual resolution reduction for the radiative transfer simulations, the highly resolved LESs with a horizontal grid size of 10 m remain crucial due to the internal smoothing in the model.

The simulations of realistic measurements of the two polarization cameras of specMACS were performed using the 3-D radiative transfer model MYSTIC , which is part of the freely available libRadtran radiative transfer package . MYSTIC allows for the simulation of scalar radiances and also polarized radiation originating from scattering events of photons on cloud and aerosol particles or molecules . The number of photons was chosen so that the noise of the Monte-Carlo-simulated intensity had an average standard deviation of about 6 %. The high computational costs were reduced by simulating measurements at a frequency of 1 Hz instead of the operating acquisition frequency of 8 Hz of specMACS. This is in accordance with the actual frame rate at which the stereographic reconstruction algorithm is applied. While the operational cloudbow retrieval is carried out at an angular resolution of 0.3°, this is reduced to approximately 1.25° for the simulations. Such a reduced angular resolution is still sufficient for the cloudbow retrieval: as shown in Fig. 13b in , the minimum required Nyquist resolution to resolve the peaks of the supernumerary bows is about 1.2° for the largest effective radius simulated here of about 40 µm and a wavelength of 0.49 µm and hence just below the given angular resolution. For longer wavelengths and smaller effective radii, the minimum required resolution increases. However, as the effective radii in the model domain are mostly well below 25 µm (compare Fig. ) at which the minimum required Nyquist frequency is about 1.5° at a wavelength of 0.49 µm, the features of the supernumerary bows, which are needed for an accurate determination of cloud droplet size distributions, are still well resolved in the simulations.

As described by , the sensors of the two polarization cameras of specMACS are divided into 512×612 4×4 pixel blocks for the different color channels (red–green–green–blue) and the polarization directions. For the whole field of view, the measurements are interpolated to the full 2048×2448 grid. Hence, to simulate the specMACS measurements, the Stokes vectors measured at each 4×4 pixel block are simulated and then interpolated to the whole grid with regard to the polarization directions and the spectral response of the different color channels. Hereby, the simulations are performed in the wavelength range between 380 and 690 nm in 10 nm steps to represent the spectral response of the cameras as shown in Fig. 1b of . The central wavelengths (bandwidths) of the three color channels are approximately 620 nm (66 nm), 546 nm (117 nm) and 468 nm (82 nm) . To represent the specMACS measurements, the simulations are weighted with the corresponding spectral response functions of the different color channels and polarization directions.

It was chosen to simulate a scene at a solar zenith angle of 30° and a solar azimuth angle of 20° such that the cloudbow was covered by the field of view of the camera. The flight direction was chosen toward the north with a horizontal aircraft attitude (the three Euler angles, roll, pitch and yaw, are 0°) for the whole flight. This geometry assures that the cloudbow is well visible in the field of view of the camera such that multiple cloud targets are observed from all scattering angles between 135 and 165° as required for the cloudbow retrieval .

Finally, MYSTIC allows us to simulate complex fields of both liquid water and ice clouds . In this study, the simulated shallow LES clouds consist of liquid water droplets only. The optical properties are calculated by Mie theory .

To begin with, it is shown that the simulated cloud field is representative of shallow cumuli measured during the EUREC4A campaign. This is demonstrated considering a scene measured on 28 January 2020 at 16:29 UTC by the polarization camera looking to the lower right of the aircraft in the flight direction (polLR; ) of specMACS. The measured and simulated RGB images can be seen on the left of Fig. . Both scenes show shallow cumuli, and from visual comparisons of the two RGB images, it can already be stated that the simulations (middle panel) seem to be realistic. The simulations resolve optically thin and small clouds as well as the general cloud structure recognizable, e.g., by shadows on the cloud surfaces. Shadows can also be detected on the underlying ocean surface, which is simulated using the bidirectional reflectance distribution function (BRDF) after . In particular, shadows on the ocean surface are identifiable in the sunglint region, the specular reflection of the sun on the ocean surface. The position of the sunglint is determined by the relative position of the sun, which is different in both images because the simulations were not conducted for a specific flight geometry. Still, simulation and measurement can be compared in terms of the radiances as the solar zenith angle (SZA) is comparable for the measurement (SZA≈33°) and the simulation (SZA=30°).

For a more quantitative comparison, the measured and simulated radiances for the considered images are shown for the three color channels (red, green and blue) on the right of Fig. . Because of the similarity of the probability density functions of the different channels for the simulations and the measurements, the simulated cloud field can be stated to be representative of clouds as measured during the EUREC4A campaign.

In a similar manner, the components Q and U of the Stokes vector can be compared. For both the simulations and the measurements, the Stokes vector is defined with respect to the scattering plane such that U≈0, and the comparison can be reduced to the Q component. The respective measured and simulated results for the green channel are shown in Fig. . From the spatial distributions of Q it can be seen that the important features like the cloudbow and the sunglint show a linearly polarized signal of the same order for the measurement and the simulation. Since polarized radiances are much more sensitive to viewing geometry, the histogram on the right of Fig.  is restricted to the cloudbow region between 135 and 165° scattering angles. Except for some minor differences which are most likely due to the different geometries and cloud fields considered, the histograms for the different channels look very similar. Hence, it can be stated that the simulations are representative of the polarized measurements as taken during the EUREC4A campaign.

A main question for the evaluation of the retrieval results is how the derived data like cloud top heights and effective radius can be compared to the actual model input. This is related to the question of where the photons detected by the instrument originate from and, hence, which cloud top heights and cloud droplet size distributions we can expect to see from the model input. The polarized signal of the cloudbow is generated by both single and forward-directed multiple scattering, but since the latter does not affect the angular structure of the cloudbow, it also does not affect droplet size retrievals . Hence, the relevant contribution to the measured signal comes from singly scattered photons, which occur on average after the mean free photon path (roughly 20 m as stated above) at an optical depth of τ=1. To account for this, we performed a second simulation with MYSTIC using the same viewing geometry as 30 s after the simulation start and considered only singly scattered photons. For this simulation, the backward mode of MYSTIC was used, which implies that the photons are started at the detector and not at the sun . It should be noted that the choice of a particular time and the corresponding fixed viewing geometry means that photons of oblique-viewing pixels might be scattered at higher altitudes in the cloud than nadir-looking pixels. However, we compared the retrieval results to two other reference simulations after 15 and 45 s and verified that the comparisons are similar both from the individual simulations and from a combination of the three reference simulations such that we will limit ourselves to the 30 s reference simulation in this paper. Performing these reference simulations without any scattering from molecules or aerosols ensures that the first scatter events of the photons will occur either on cloud droplets or on the Earth's surface. From the scatter locations, the indices of the respective grid boxes of the model input can be directly determined. On average, the first scattering events will happen at an optical depth of τ=1. We further used only a single wavelength, namely 550 nm, corresponding approximately to the central wavelength of the green channel of the polarization cameras. Due to the similar penetration depth of the visible wavelengths, using a single wavelength is representative of the determination of the first scatter events. For each viewing direction, 1000 photons were simulated, and their scatter event locations determine a weighting function along the line of sight of the instrument. Then, the average of all scattering event locations gives the location of the cloud part which is expected to be seen by the instrument, and hence the expected cloud top height can be estimated. Although the cloud top height retrieval is based on intensity measurements and hence on the detection of both single- and multiple-scattered photons, it is particularly important for the correct geolocalization of the cloud targets and the aggregation of the signal in the polarimetric retrieval. Therefore, the retrieved height should be as close as possible to the height from which the polarized signal and thus the single-scattering signal originates. Moreover, the stereographic algorithm is based on the identification of contrast gradients which are not visible deeper in the cloud. If an optical depth of τ=1 is not reached along the viewing direction, a corresponding fraction of photons will be scattered at the ground such that the vertical coordinate of the scatter event is 0. These scatter events are not taken into account for the determination of the average scatter location. To compare the derived cloud droplet size distribution to the expected one from the model, all scatter event locations of photons that did not hit the ground are taken into account separately. This is explained in more detail at the beginning of Sect. .

In this study, measurements of the polarization-resolving polLR camera of specMACS flown on board the German research aircraft HALO were simulated using the 3-D radiative transfer model MYSTIC. The aim was the analysis of the accuracies of the retrieval algorithms applied to the measurements of the camera. The first algorithm is the stereographic reconstruction algorithm used to determine the cloud geometry and cloud top heights. The second algorithm is the polarimetric retrieval, which uses polarized measurements of the cloudbow to derive microphysical properties of the cloud droplet size distribution. Both algorithms rely on the observation of clouds from multiple viewing angles. Hence, a 1 min overflight over a realistic cloud field obtained from simulations with the LES model PALM was performed using the 3-D radiative transfer model MYSTIC. The LESs were initiated based on dropsonde measurements from the EUREC4A field campaign, which took place in early 2020 in the vicinity of Barbados, during which shallow cumulus convection was studied. In this way, it was ensured that the cloud field represents shallow cumuli as measured during the campaign, which could be shown by a qualitative comparison of the measured and simulated (total and polarized) radiance. In addition to the realistic simulation with advective and developing clouds, a theoretical simulation of non-evolving clouds was performed to test the sensitivity of the retrieval algorithms to cloud development and movement.

For the stereographic reconstruction of the clouds, it was shown that the derived cloud top heights differ on average by less than 70 m from the expected model heights with a standard deviation of about 130 m. In the case of non-evolving clouds, the deviation from the expected heights reduced to about 46 m with a standard deviation of about 137 m. A comparison of the pointwise differences revealed that the stereographic cloud top heights tend to be underestimated at the highest cloud tops, while they tend to be overestimated where the clouds are lower, i.e., at the cloud edges or in shadows.

Next to the cloud top heights from the stereographic reconstruction, the cloud droplet size distributions obtained in the form of the effective radius and effective variance from the polarized measurements of the cloudbow were tested in this study. Here, it was shown that the effective radius from the retrieval differs on average by about (-0.17±1.30) µm from the expected effective radius of the model input. In the case of non-evolving clouds the average difference is (-0.03±1.28) µm and thus slightly better than the result for the realistic simulation. The results for the effective variance were shown to be very similar for both simulated cases with evolving and non-evolving clouds. The mean difference was 0.02 with a standard deviation of 0.05. The distribution of effective variances derived from the cloudbow retrieval is broader than the expected distribution. The wider distribution might be explained by the fact that the effective variance highly depends on the signal of the supernumerary bows. If that signal is damped, larger effective variances are retrieved. The large sub-grid variability which has been observed for the shallow cumulus clouds considered in this study might cause such a damping of the signal. Comparing the results of the realistic simulation, including the cloud development and the idealistic case without any cloud evolution, indicates that the cloud development might also be one source of error of the overestimation of the effective variance by the retrieval. Furthermore, a smoothing of the signal due to contributions from multiple scattering and the stratification of the effective radius within the cloud can be reasons for this overestimation, as already observed by . With respect to some outliers with larger errors in the effective radius or variance, we studied the correlation between the differences in the effective radius and variance retrieved and those expected from the model and the optical thickness. However, no significant correlation which could point to an error source was found.

In future, further investigations of the observed sub-grid variability are planned. In particular, a reduction in the resolution of the LES model output could be used to exclude any sub-grid variability. Moreover, it is planned to include natural variations in the effective variance in future studies. Furthermore, the approach of 3-D radiative transfer simulations will be used for the accuracy assessment of the two retrieval algorithms for other cloud types, such as deeper convective or mixed-phase clouds.