Error sources in the retrieval of aerosol information over bright surfaces from satellite measurements in the oxygen A band
Retrieving aerosol optical thickness and aerosol layer height over a bright surface from measured top-of-atmosphere reflectance spectrum in the oxygen A band is known to be challenging, often resulting in large errors. In certain atmospheric conditions and viewing geometries, a loss of sensitivity to aerosol optical thickness has been reported in the literature. This loss of sensitivity has been attributed to a phenomenon known as critical surface albedo regime, which is a range of surface albedos for which the top-of-atmosphere reflectance has minimal sensitivity to aerosol optical thickness. This paper extends the concept of critical surface albedo for aerosol layer height retrievals in the oxygen A band, and discusses its implications. The underlying physics are introduced by analysing the top-of-atmosphere reflectance spectrum as a sum of atmospheric path contribution and surface contribution, obtained using a radiative transfer model. Furthermore, error analysis of an aerosol layer height retrieval algorithm is conducted over dark and bright surfaces to show the dependence on surface reflectance. The analysis shows that the derivative with respect to aerosol layer height of the atmospheric path contribution to the top-of-atmosphere reflectance is opposite in sign to that of the surface contribution – an increase in surface brightness results in a decrease in information content. In the case of aerosol optical thickness, these derivatives are anti-correlated, leading to large retrieval errors in high surface albedo regimes. The consequence of this anti-correlation is demonstrated with measured spectra in the oxygen A band from the GOME-2 instrument on board the Metop-A satellite over the 2010 Russian wildfires incident.
Aerosols are one of the largest sources of uncertainties in our understanding of the Earth's current climate and its future projection, because of the role they play in complex atmospheric processes that influence the Earth's radiation budget (IPCC, 2014). More generally, aerosols influence the climate either directly through absorption and scattering of solar radiation, or indirectly through cloud formation and aerosol–cloud interaction.
In climate studies, the direct radiative effect of aerosols is calculated to understand its net contribution to the Earth's total radiation budget. This depends on aerosol macrophysics (such as vertical distribution) and microphysics (such as size distribution and single-scattering albedo), which determine whether aerosols in a particular scenario are more efficient in absorbing or scattering the incoming solar radiation and the thermal radiation from within the Earth's atmosphere. The ability of aerosols to absorb radiation can influence thermal stability of the atmosphere, which in turn influences cloud formation and atmospheric chemistry (IPCC, 2014; Chung and Zhang, 2004). Knowledge of the vertical distribution on aerosols is, hence, an important piece of the puzzle to reduce uncertainties in our understanding of Earth's climate. Because of the high degree of variability of aerosols in both time and space, this knowledge is required at a high spatio-temporal resolution.
To observe (among other atmospheric parameters) aerosols, many space-borne Earth observation initiatives have been proposed to monitor the Earth's atmosphere with either active or passive remote sensing techniques. An example of such an initiative is the Cloud-Aerosol LIdar with Orthogonal Polarization (CALIOP) instrument on board NASA's Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) mission, which provides information on the vertical distribution of aerosols. However, because of the limited swath of a space-borne lidar instrument, the mission coverage area is also limited. This gap in the data can be filled with satellite missions carrying passive remote sensing instruments, which have a larger coverage area with good temporal resolution. One such initiative is the Copernicus programme by the European Commission (EC) partnered with ESA, which aims to provide accurate information of atmospheric composition from space. Of its missions, the Sentinel-5 Precursor, Sentinel-5 and Sentinel-4 are examples of polar-orbiting and geostationary satellites equipped with hyperspectral sensors (Veefkind et al., 2012; Ingmann et al., 2012).
Hyperspectral instruments on board the Sentinel-4/5/5P missions measure Earth radiance and solar irradiance in the top of atmosphere, spectrally resolved over a wide wavelength range. Of the wavelength bands measured, these instruments also measure in the oxygen A band between 758 and 770 nm, where absorption of solar radiation is dominated by molecular oxygen and its isotopologues. The presence of aerosols in the atmosphere significantly impacts absorption of solar radiation by molecular oxygen (Fig. 1a). In the absence of clouds and aerosols, the oxygen A band can either be almost transparent or opaque to solar radiation, owing to the large variation in the absorption cross section within the spectra. In the presence of an aerosol layer in the atmosphere, the absorption intensity of the spectra can provide useful vertical information (as observed in Fig. 1b) – deeper absorption lines correspond to a lower aerosol layer, and shallow absorption lines to a higher aerosol layer. This is the basis of retrieving aerosol layer height from the oxygen A band. Currently, the Copernicus Sentinel-4/5/5-P aerosol layer height algorithms are designed to exploit oxygen absorption spectra in the A band to retrieve the height of an aerosol layer.
The retrieval of aerosol properties from the oxygen A band presents a few challenges, one of them being that aerosol layers in the atmosphere are usually optically thin, and are quite difficult to observe in the presence of clouds. This is because clouds have an optical depth which is typically orders of magnitude larger than that of aerosols, and are more efficient in scattering incoming radiation. Consequently, aerosol retrieval algorithms generally refrain from retrieving over cloudy scenes; our algorithm is no exception to this and requires cloud screening to filter out pixels containing clouds.
While cloudy pixels can be filtered out to a certain degree, retrieving aerosols from measurements in the oxygen A band over bright surfaces faces a host of other challenges. From the literature, it is understood that aerosol information content from measured spectra in the oxygen A band reduces as the surface albedo increases (Corradini and Cervino, 2006; Sanghavi et al., 2012). Sanders et al. (2015) report potentially large biases in their aerosol layer height retrievals from the oxygen A band when the surface albedo is fitted. In a previous paper, Sanders and de Haan (2013) also report that certain specific combinations of geometry, aerosol, and surface properties can result in unusually large uncertainties in the retrieved aerosol layer height (see also Fig. 8-2 in Sanders and de Haan, 2016). Such large biases can perhaps be attributed to a phenomenon known as the critical surface albedo regime (Seidel and Popp, 2012), wherein for specific surface albedos, the top-of-atmosphere reflectance becomes independent of the aerosol optical thickness. Sanders et al. (2015) observe that when the surface albedo is not fitted, typical uncertainties in the surface albedo database over land can result in large biases. From our analyses, we understand that for relative errors up to 10 % in the surface albedo, retrievals over dark surfaces are not affected, whereas the same over sufficiently bright surfaces (surface albedo greater than 0.2) can suffer from very large biases.
A combination of all the error sources discussed previously can result in large biases. In fact, we observe that the presence of errors often leads to no convergence in the retrieval, with no concrete predictability on which pixel is likely to yield no result. Because of this, the operational algorithm wastes resources trying to retrieve aerosol layer height from pixels that potentially do not have any usable aerosol information. This is especially problematic in the framework of high-resolution instruments, which demand operational processors to make efficient use of computational time and effort to process large number of spectra (typically several hundred per second). In order to design more efficient operational algorithms, the concept of critical surface albedo needs to be extended beyond the framework provided by Seidel and Popp (2012) into the oxygen A band for aerosol optical thickness as well as aerosol layer height.
This paper analyses simulated measured top-of-atmosphere reflectance spectra in the oxygen A band and provides an explanation for the loss of aerosol information over bright surfaces. Its implication is provided in an optimal estimation framework, specific to the retrieval of aerosol layer height, with results from sensitivity analyses. The analysis is followed up with a demonstration in a real data environment by retrieving aerosol layer height over a bright surface. The case study chosen is the retrieval of optically thick biomass-burning aerosol plumes over the 2010 Russian wildfires, to demonstrate the effect of this loss of aerosol information over land. This paper is one in a series of papers on development of an operational oxygen A-band Aerosol Layer Height retrieval algorithm for Sentinel-4/5/5-P by KNMI, preceded by Sanders and de Haan (2013) and Sanders et al. (2015). The current operational ALH algorithm for S5P is described in Sanders and de Haan (2016). While the results of this paper are relevant for the Sentinel 5-Precursor algorithm as well, the instrument model used in the sensitivity studies is for the UVN spectrometer on the S4 mission.
The next section (Sect. 2) provides a description of the forward model and the optimal estimation framework. Section 3 discusses the concept of aerosol–surface ambiguities in the oxygen A band. Section 4 describes various sensitivities of our retrieval algorithm focusing on the difference between dark and bright surfaces. Section 5 discusses aerosol layer height retrievals over the 2010 Russian wildfires using GOME-2A data. Section 6 concludes this paper with a discussion and the implication of the findings from this paper.
2.1 Forward model
There are three primary parts of the forward model, namely the atmospheric model, the radiative transfer code, and the instrument model. A radiative transfer code is used to model a high-resolution top-of-atmosphere radiance by propagating radiation through the atmosphere described by the atmospheric model. The top of atmosphere reflectance R computed by the forward model is defined as the ratio of the radiance I of the pixel measured by the instrument to the top-of-atmosphere solar irradiance E0 of the pixel on a horizontal surface unit,
where μ0 represents the cosine of the solar zenith angle of the pixel, and λ represents the wavelength.
The top-of-atmosphere reflectance is calculated after the measured radiance and irradiance are convolved with the instrument spectral response function (ISRF) of the hyperspectral sensor in order to simulate measured spectra by a satellite instrument. For simulations, the high-resolution solar spectrum from Chance and Kurucz (2010) is used.
2.1.1 Radiative transfer model
The radiative transfer model is the layer-based orders of scattering (LABOS) method, which is a variant derived from the doubling-adding method (de Haan et al., 1987). Atmospheric properties are calculated line-by-line to compute the reflectance at the top of atmosphere. The radiative transfer code is a part of a software package called DISAMAR (Determining Instrument Specifications and Analysing Methods for Atmospheric Retrievals), which is the main workhorse of the operational algorithm development efforts at KNMI for oxygen A-band aerosol height retrieval with S5P/S4/S5 instruments. Scattering by gases is described by Rayleigh scattering, which has a low scattering cross section in this wavelength region. Because of this, polarization is ignored. Wavelength shifts caused by rotational Raman scattering (RRS) are ignored in order to reduce computational effort, since line-by-line calculations are computationally expensive in the oxygen A band. This is convenient, since the Raman scattering cross section is even smaller than that of Rayleigh scattering. The atmosphere in the forward model is plane-parallel for the Earth radiance, and spherically corrected for the incoming solar irradiance.
2.1.2 Atmospheric model
For cloud-free conditions, the following four absorption and scattering processes are significant in the wavelength range between 758 and 770 nm: scattering by gases, reflection of light by the surface, scattering and absorption by aerosol particles, and absorption by molecular oxygen. Absorption of solar radiation by O3 and H2O are ignored, since they are not dominant absorbing gases in this spectral range.
The surface reflectance is assumed isotropic, described by its albedo. Depending on the surface albedo, a surface can either be bright or dark. Dark surfaces are classified with surface albedo close to 0.05 (or lower), which in the oxygen A-band spectral region typically corresponds to ocean surfaces. Bright surfaces in the oxygen A band on the other hand have a surface albedo of 0.2 (intermediately bright) and higher and are primarily over land. For the oxygen A band at 760 nm, typical values of surface albedo over vegetated surfaces exceed 0.4 since the wavelength band is located beyond the red edge, where absorption of solar radiation by chlorophyll diminishes. Scenes with snow or ice are not processed.
Aerosols are represented as a single layer with a fixed pressure thickness of 50 hPa, containing aerosol particles with a fixed aerosol optical thickness and aerosol single-scattering albedo. Aerosol layer height is defined as the mid-pressure of the aerosol layer – if the aerosol layer extends from 650 to 600 hPa, the aerosol layer height is 625 hPa. In the operational S5P aerosol layer height algorithm, currently the aerosol phase function is a Henyey–Greenstein model (Henyey and Greenstein, 1941) with an asymmetry factor of 0.7, and an aerosol single-scattering albedo of 0.95 (Sanders et al., 2015). While a Mie scattering model could be used instead of the Henyey–Greenstein model, the latter is computationally less expensive and hence more optimal for the operational algorithm.
Oxygen absorption cross sections are derived from the NASA JPL database, following Tran and Hartmann (2008) who indicate that line parameters in the JPL database are more accurate than the HITRAN 2008 database. First-order line mixing and collision-induced absorption by O2–O2 and O2–N2 are derived from Tran et al. (2006) and Tran and Hartmann (2008).
2.1.3 Instrument model
The instrument model is described by the instrument slit function, whose spectral resolution depends on its full width at half maximum (FWHM), and its noise model. For this study, oxygen A band is simulated using specifications of the Sentinel-4 Ultraviolet Visible and Near-infrared (UVN) instrument, which is set to launch in 2022. The instrument is a sounder with a hourly coverage over Europe and northern Africa at a spatial resolution of 8 km × 8 km sampled at 45∘ N and 0∘ E. The instrument has a FWHM of approximately 0.116 nm, oversampled by a factor of 3, effectively giving the instrument a spectral sampling interval of 0.04 nm. Aerosol layer height will be an operational product provided by the Sentinel-4 mission. An example of oxygen A-band spectra at a 0.116 nm resolution is provided in Fig. 1. For retrievals with real data, measurements from the Global Ozone Monitoring Experiment-2 on board the MetOp-A satellite are used. Launched on 16 October 2006, GOME-2A is an optical spectrometer fed by a scanning mirror, which enables across-track scanning in the nadir. The instrument has a spectral sampling interval of approximately 0.21 nm at 758 nm (spectral resolution of 0.48 nm for channel 4), and has a nominal spatial resolution of 80 km× 40 km (Munro et al., 2016). A shot noise model is assumed for the instrument.
2.2 Inverse method
The inverse method is based on the optimal estimation (OE) framework described by Rodgers (2000), which is a maximum a posteriori (MAP) estimator that constrains the least-squares solution with a priori knowledge on the state vector. The method assumes Gaussian statistics for the a priori errors. The iterative method is a Gauss–Newton approach, and the estimation parameters are the aerosol optical thickness τ and the aerosol layer height z. The cost function χ2 is defined as
where y is the measured reflectance, F(x,b) is the vector of calculated reflectance using the forward model, x is the state vector containing fit parameters, b is the vector containing other model parameters, Sϵ is the measurement error covariance matrix, xa is the a priori state vector, and Sa is the a priori error covariance matrix. Sa is diagonal, assuming no correlation between state vector elements. Sϵ is also diagonal, since the measurement error is assumed uncorrelated. is the measurement part of the cost function, whereas is the state vector part of the cost function.
The a posteriori error covariance matrix is computed as
where K is the Jacobian with its columns containing partial derivatives of the reflectance with respect to the state vector elements. DISAMAR calculates the Jacobian semi-analytically, similar to the reciprocity method described by Landgraf et al. (2001). The Jacobian drives the retrieval towards the solution as an integral component in the update to the state vector,
where xn+1 is the next iteration to the nth iteration in the retrieval, and Kn is the Jacobian evaluated at the nth iteration. The Jacobian can become singular if the value of the partial derivative of the reflectance to the state vector parameter is very low, or is correlated to another parameter in the state vector. In these cases, the error covariance matrix does not exist, since the inverse covariance matrix is non-invertible; if it is nearly singular, the problem is ill-conditioned and may result in very large biases in the estimation.
The inverse method reaches a solution if the change in the state vector between iterations is below a convergence threshold. It is possible that during iterations, the inverse method estimates state vector elements beyond boundaries. In such a case, the state vector element is adjusted back to just within its physical limits. If the adjustment is made in two consecutive iterations, the retrieval is stopped and no solution is reached. The upper cap in the number of iterations is set at 12, beyond which the retrieval is said to have failed. In this paper, these failed retrievals are termed as non-convergences. The next section discusses the atmospheric conditions that can potentially lead to these non-convergences.
3.1 Influence of surface reflectance on aerosol information content in the oxygen A band
The top-of-atmosphere reflectance over a surface with an albedo As can be written as the sum of atmospheric path contribution of the photon Rp and surface contribution Rs,
Rp is the top-of-atmosphere reflectance in the absence of a surface. Rs is calculated by subtracting the path contribution from the total top-of-atmosphere reflectance, and represents contributions from photons that have been reflected one or more times by the surface. Rs is dependent on the absorbing and scattering species present in the atmosphere, and also includes aerosol influences. Rp is calculated by substituting As=0.0 and calculating the top-of-atmosphere reflectance in DISAMAR. Rs is calculated by subtracting Rp from R. With increasing viewing angle, Rp increases whereas Rs decreases (Fig. 2). This is in line with expectation, since the slant aerosol optical thickness increases, which increases the amount of contribution that aerosols have in R(λ,As). At steeper geometries, light at the top of atmosphere is more diffuse than direct, which is the primary reason why Rs decreases (assuming a Lambertian surface).
For a model parameter x with two values xa and xb, the difference spectrum ΔRΔx, defined as
can reveal the influence the model parameter x has on the oxygen A band. The spectral shape of ΔRΔx can also show parts of the spectrum that are more sensitive to x. Following Eqs. (5) and (6), ΔRΔx(λ,As) is defined as
If and have opposing signs, ΔRΔx reduces following Eq. (7), which results in a reduction of sensitivity to the parameter x.
Comparing and at two different aerosol layer heights (z) for two different scenes with the same atmospheric conditions (Fig. 3a), it is observed that and have opposite signs and Rp is relatively more sensitive to aerosol layer height than Rs. This is especially the case in the deepest part of the R branch between 759.50 and 761.30 nm and parts of the P branch between 761.30 and 763.00 nm, where the higher absorption cross section reduces the number of photons that can reach the surface. This ultimately reduces the magnitude of Rs to the top of atmosphere for these absorption sub-bands. over ocean and vegetation also shows an increase in its overall magnitude with an increase in surface albedo, and hence an increase in cancellation between ΔRpΔz and ΔRsΔz. Figure 4 represents the variation of the derivative of reflectance with respect to aerosol properties, for increasing surface albedo. Albeit subtle, the consequence of this cancellation between ΔRpΔz and ΔRsΔz is observed in Fig. 4a, where ∂R for the deepest part in the R branch and parts of the P branch diminishes gradually with an increase in surface albedo.
The same experiment is repeated for aerosol optical thickness (τ), and the results are presented in Fig. 3b. and are anti-correlated (Pearson correlation coefficient is −0.99, irrespective of the surface albedo), and the magnitude of increases with an increase in surface albedo. Figure 4b shows the partial derivative of the reflectance with respect to τ for increasing surface albedo. This anti-correlation explains negative derivatives in the higher surface albedo regime.
and of aerosol single-scattering albedo (ω) in Fig. 3c reveals a strong correlation (with a Pearson correlation coefficient of almost unity). This suggests that an increase in surface albedo increases the sensitivity of the model to ω. We suspect that this information predominantly arises from interactions between scattered light by aerosols and surface. The magnitude of the partial derivative of reflectance with respect to ω for increasing surface albedo (shown in Fig. 4c) shows an increase, which is in line with our analysis of Fig. 3c.
For increasing surface albedo, the more dynamic parts of the spectrum in Fig. 4b correspond to spectral points with less absorption by molecular oxygen. These are also the parts of the spectrum with a high signal-to-noise ratio (SNR) and high . From Eq. (4), the inverse method gives a higher priority to spectral points with a higher . Intuitively, low information of τ from the oxygen A-band spectrum will increase the dependence of the inverse method to prior information. This is further discussed in the next section.
3.2 Aerosol–surface interplay in the top-of-atmosphere reflectance
In the inverse method, an a priori error of 100 % is assumed for the aerosol optical thickness, which gives it freedom to vary during iterations. If the a priori aerosol optical thickness is far from the solution, a large a priori error ensures that the retrieval can estimate the parameter in fewer iterations. However, whether the Gauss–Newton optimization reaches the correct solution depends on two primary factors: first, whether the cost function has a global minimum, and second, whether the gradient of the cost function is sufficiently large such that it is minimized significantly at every iteration.
From our analysis of ΔRΔx for aerosol parameters, we have identified aerosol optical thickness to be the parameter most affected by an increasing surface albedo, due to the cancellation between and owing to their similar amplitudes and spectral shapes (but opposing signs). Because of this, the top-of-atmosphere reflectance spectrum becomes independent of aerosol optical thickness for higher surface albedo regimes (Fig. 5).
Over a dark surface such as the ocean, top-of-atmosphere reflectance in the continuum is unique at different aerosol loads (Fig. 5a). The variation in the top-of-atmosphere reflectance in the continuum reduces as the instrument points more towards the nadir. In such geometries, Rs can play a more significant role than Rp and reduce the available information on τ in the R(λ,As) spectrum. For bright surfaces, the variation in the top-of-atmosphere reflectance spectrum is less for steeper geometries relative to the same geometries over the ocean (Fig. 5b, green and blue lines). There can also be cases where, provided sufficiently high aerosol loading, the top-of-atmosphere reflectance spectrum in the continuum can be independent of aerosol optical thickness over very bright surfaces such as vegetation (Fig. 5c, green line). In such cases, more than one value of τ results in the same top-of-atmosphere reflectance. Henceforth in this paper, this phenomenon is termed as aerosol–surface ambiguity.
A loss in aerosol information can have special implications in the minimization of the cost function. As observed in Fig. 6, for lower surface albedo regimes there exists a single minimum of the cost function. For such scenes, if the a priori aerosol optical thickness is far from the true value, the gradient is sufficiently large such that a small change in the state vector between iterations leads to a significant minimization of the cost function. As the surface albedo increases, this gradient decreases significantly, and can also result in the presence of multiple minima in the cost function (Fig. 6, right) if the state vector is far away from the truth. This makes the retrieval dependent on the initial guess of τ.
Because of a model error (described in Fig. 6) in the aerosol layer height between y and F(x,b) (in Eq. 2), the global minimum of the cost function shifts away from the true τ. This shift is biased higher than the truth if the aerosol layer is lower in the atmosphere in comparison to the aerosol layer in the synthetic true spectrum, because the model has to compensate the extra absorption by molecular oxygen. If the aerosol layer is higher in the atmosphere, the minimum of the cost function is situated at a τ lower than the true τ. As observed in Fig. 6 (left, red line), this shift of the cost function minimum from the true τ is larger over bright surfaces for a viewing angle close to nadir, where Rs is more dominant. For the same angle, the global minimum over a dark surface is situated at the true τ value, even with the presence of a model disagreement with the simulated “true” spectrum. As the viewing angle increases over the bright surface, Rp increases and the global minimum of the cost function moves closer towards the true τ.
If the a priori error assigned to aerosol optical thickness is large, presence of aerosol–surface ambiguities can result in non-convergences. Because the a priori part of the cost function has a smaller value than the measurement part, reducing a priori error assigned to the aerosol optical thickness does not necessarily guarantee a solution to this issue since it does not remove the multiple-minima present in the cost function. Since errors between aerosol optical thickness and aerosol layer height are correlated (Sanders et al., 2015), a large error in the optical thickness will lead to a large error in the aerosol layer height estimate. The next section discusses the sensitivity of the aerosol layer height algorithm to this phenomenon by introducing model errors in a simulation environment.
In DISAMAR, forward models for simulation and retrieval have been kept separate so that errors can be introduced into the simulated spectra to mimic errors in a real retrieval scenario. In this section, the instrument model of the Sentinel-4 UVN near-infrared spectrometer is used. The wavelength range for simulations and retrievals is between 758 and 770 nm. A comparative analysis of biases in the retrieved aerosol layer height is conducted over ocean (As=0.03) and land (As=0.25, and As=0.4). Bias in the aerosol layer height is defined as the difference between retrieved and true aerosol layer height (in hPa) – a positive sign indicates that the aerosol layer is retrieved below the true aerosol layer height. The aerosol layer height retrieved is a single layer for the entire atmospheric column, with a fixed thickness of 50 hPa.
4.1 Sensitivity to model error in the aerosol layer thickness
In a typical real-world scenario, aerosol plumes can be as thick as 200 hPa in the atmosphere, or more. We simulate a scene containing an aerosol layer that extends approximately from the surface (1000 hPa) to 800 hPa in the atmosphere. The true τ is 1.0, and the a priori τ is 0.5. The a priori value of the aerosol layer height is 650 hPa, and the aerosol layer thickness is fixed at 50 hPa. In an ideal retrieval instance, the retrieved aerosol layer height (which has a thickness of 50 hPa) should coincide with the height of the simulated thicker aerosol layer. We observe that, in general, the error in the retrieved aerosol layer height reduces as the viewing zenith angle increases (Fig. 7a). This is explained by a reduction in Rs and an increase in Rp (Fig. 2, red line), which explains why difference in errors between retrievals over different surfaces reduces with an increase in viewing angle (Fig. 7a, high viewing zenith angles).
At lower viewing zenith angles, the difference in aerosol layer height errors between retrievals over the different surfaces is the largest, since the effect of Rs cancelling out with Rp is significantly larger (Fig. 2, blue line), which increases with an increase in surface albedo (Fig. 3a). The retrieved aerosol layer is biased towards the surface in all three surface albedo scenarios, with the aerosol layer being placed closer to the surface if the surface albedo is brighter. This should not suggest a sensitivity to the geometrical thickness of the aerosol layer. As the surface albedo increases, the number of photons that pass through the atmosphere to interact with the surface before reaching the detector increases. These photons have a longer path length, which results in an increased absorption by oxygen at specific spectral points with weak oxygen absorption lines. In comparison to photons at wavelengths with strong oxygen absorption lines, these photons have a higher SNR since relatively more of them reach the detector. A higher SNR ensures lower noise, and hence a higher value in the inverse of the measurement error covariance matrix Sϵ. If a spectral point has a higher value in matrix, it has a higher representation in the cost function (in Eq. 2), and hence a higher preference (or weight) in the optimal estimation. Because of this, the retrieval prefers to retrieve an aerosol layer height described by photons that travel through the aerosol layer closer to the surface. If, however, the aerosol optical thickness is so large that the photons cannot penetrate the aerosol layer, the retrieved aerosol layer height would be more accurate. Retrieving the height of optically thin aerosol layers can also be quite challenging, owing to the fact that these layers will allow more photons to pass through and interact with the surface, leading to an increase in Rs, and hence an increase in the cancellation between Rp and Rs. As a result of this, large biases in the retrieved aerosol layer height can be expected for optically thin layers over bright surfaces.
Another consequence of retrieving aerosol layer height over bright surfaces is that the retrieval may become more susceptible to model error in aerosol and surface properties, such as the aerosol phase function anisotropy factor g, the aerosol single-scattering albedo ω, and especially the surface albedo As, which are fixed in the model. These are investigated in the following.
4.2 Sensitivity to model error in the aerosol phase function
The presence of a model error in the aerosol phase function can result in large biases if the surface is bright (Fig. 7b). For a higher surface brightness and a viewing angle close to nadir, this bias is larger. As the viewing angle increases, the biases reduce significantly. The correlation of bias with surface albedo suggests that biases caused by model errors are exacerbated by the surface contribution Rs, which reduces as viewing angle increases (Fig. 2, right).
4.3 Sensitivity to model error in aerosol single-scattering albedo
From Fig. 4, aerosol single-scattering albedo plays an increasingly significant role in the retrieval of aerosol layer height as the surface gets brighter. Because of this, a mis-characterization of aerosol single-scattering albedo in the model can lead to very large biases over bright surfaces (Fig. 7c), and also non-convergences. This is not the case for retrievals over the ocean, since the influence of aerosol single-scattering albedo on the oxygen A-band spectrum is low. It is observed that, as the viewing angle increases, these biases drop significantly. This is again attributed to the decrease in Rs and increase in Rp with increasing viewing angle (again, over a Lambertian surface).
4.4 Sensitivity to model error in surface albedo
Surface albedo is a critical component in the accurate retrieval of aerosol layer height over bright surfaces. Because it is a fixed parameter in the forward model, an error in the surface albedo can result in large biases in the retrieval. To simulate model errors, relative errors of −10 to 10 % are introduced in the retrieval forward model, such that the surface is modelled darker or brighter than the true value. For relative errors of ±10 %, the retrieved aerosol layer height can be biased more than 2 orders of magnitude larger over land than over ocean (Fig. 7d). For retrievals over a bright surface such as vegetation (As=0.4 or greater), the model error can result in non-convergences. As the model error reduces, retrievals over land with a surface albedo of 0.25 become more acceptable. However, over very bright surfaces, an inaccuracy in surface albedo of more than 2 % can result in biases greater than 100 hPa.
The next section demonstrates the implication of these errors in a real retrieval scenario.
The 2010 Russian wildfires began in late July and lasted for several weeks until the beginning of September. Literature reports droughts and record summer temperatures in the same year as a precursor to the wildfires, both of which have been attributed to climate change (Hansen et al., 2012). A consequence of the forest fires were optically thick aerosol plumes over the country, especially over Moscow. In the first few weeks of August 2010, due to the presence of a strong anti-cyclonic circulation pattern in the atmosphere, the impact of biomass-burning aerosols on air quality in Moscow was markedly larger than what was observed from previous wildfire incidences – the ultraviolet aerosol index (UVAI) reported by the Ozone Monitoring Instrument (OMI) on board the NASA Aura mission observed an increase by a factor of 4.1 over previous years (Witte et al., 2011) over Moscow, due to aerosol plumes originating from the south and east of the city.
The aerosol plume above Russia on 8 August 2010 serves as a test case for the aerosol layer height retrieval algorithm, due to fairly cloud-free conditions and the optical thickness of the aerosol plume (see Fig. 8c). Because of this, we do not employ a cloud-screening method. The GOME-2A instrument crosses over the scene at approximately 09:45–09:47 local time. The GOME-2A pixels within the region of interest are recorded between 07:45 and 07:48 UTC, at approximate latitude bounds of 52 and 60∘ and longitude bounds 29 and 45∘. This corresponds to 255 pixels in total. Meteorological information relevant to the retrieval are temperature–pressure profiles and surface pressure, acquired from the European Centre for Medium-Range Weather Forecast (ECMWF) ERA-Interim database (Dee et al., 2011) at the GOME-2A pixel using nearest-neighbour interpolation. Surface albedo is derived using nearest-neighbour interpolation for version 1.3 of GOME-2A LER climatology derived from Tilstra et al. (2017), which is at a grid. Typical values of the surface albedo over the region of interest is around 0.21. In the inverse method, the a priori value of the aerosol layer height is approximately 800 hPa. The a priori aerosol optical thickness is 1.0 at 760 nm.
CALIOP data are used for validation, which provide vertical distribution of aerosols and clouds for a footprint of approximately 70 m, with a 5 km horizontal resolution (Winker et al., 2009). While the coverage of the instrument is not as expansive as the GOME-2 instrument, the level of information available from CALIOP gives a good idea on the vertical position of aerosols in the atmosphere. For a better validation dataset, CALIOP data recorded between coordinates 52.0∘ latitude and 64.0∘ latitude approximately around 10:45 UTC are used for comparison of GOME-2A aerosol layer height retrieval results. The Level-1 CALIOP attenuated backscatter data from 1064 nm is used because lidar in the visible region (532 nm) can get heavily attenuated over optically thick plumes. As can be seen from Fig. 9, the aerosol layer is situated in between the surface and 5 km above the surface. In total, 82 GOME-2A pixels falling within 100 km of the CALIPSO track are considered for comparison.
The operational algorithm retrieves aerosol layer height and aerosol optical thickness, with fixed a priori values, as mentioned in Table 1. Following evaluation of the algorithm on GOME-2A pixels by Sanders et al. (2015), the surface albedo is not included in the state vector. The single-scattering albedo is not fitted in the sensitivity analyses in order to maintain consistency with the current operational algorithms for the Sentinel missions, which currently do not fit this parameter.Tilstra et al. (2017)
5.1 Results from the retrieval algorithm
Out of the chosen 255 GOME-2A pixels, 155 pixels converged and 100 pixels failed to converge to a solution (40 % of the pixels do not converge). The algorithm retrieved aerosol layers primarily in the lower troposphere, roughly within 0–3 km (Fig. 8a). The mean aerosol layer height retrieved is 714 m above the ground with a standard deviation of 647 m and a median of 450 m. The retrieved aerosol layers are optically thick (Fig. 8b), with a mean retrieved aerosol optical thickness of 3.0, a standard deviation of 1.8, and a median of 2.5. The retrievals over the primary aerosol plume do not converge to a solution.
Figure 9a provides results of retrieving aerosol layer height over the chosen 82 GOME-2A pixels co-located to the CALIPSO track. The CALIOP backscatter data show that the aerosol plume extends from the ground to approximately 4 km between latitudes 53 and 60∘. Beyond 60∘ latitude, the aerosol layer is elevated. Of the 82 pixels, 52 converge to a solution. From Fig. 9, it is observed that the retrieved aerosol layer heights are generally biased closer to the surface. This is explained by the increase in surface contribution Rs which represents photons passing through the atmosphere and interacting with the surface before reaching the detector. The spectral points representing these photons have a higher weight in the optimal estimation in comparison to the photons that do not interact with the surface, and hence the aerosol layer height is retrieved closer to the surface.
In Fig. 9, the retrieval does not converge to a solution between latitudes 57 and 60∘. This area also corresponds to the primary biomass-burning plume in Fig. 8. However, the estimated aerosol layer height in the last iteration for these pixels seems to be located within the aerosol plume (Fig. 9a, white crosses between latitudes 57 and 60∘). To investigate this, we retrieve τ from the top-of-atmosphere reflectance in the continuum with different a priori optical thickness values in order to test whether the non-uniqueness of aerosol optical thickness is a potential cause of retrieval non-convergence.
5.2 Retrieving aerosol layer height with multiple a priori aerosol optical thickness values
Aerosol optical thickness (τ) is first retrieved from the continuum before the oxygen A band between 755 and 756 nm. τ is retrieved with two a priori values τa and τb. In these retrievals, the aerosol layer height is kept fixed at any arbitrary value, since its value will hardly affect the continuum.
First, τa=1.0 is chosen, and the retrieved solution is then used to decide the a priori value τb. If the solution for is not reached, then is not calculated. In the case that is retrieved, τb is chosen in the following manner:
If the retrieval for fails, then we can infer a dependence on a priori information. If the retrieval is successful, and are compared to check if they are similar using the following criterion:
where T is a threshold, chosen to be 0.15. Increasing this threshold increases the margin of similarity of and . This method is henceforth called the prefit method.
Applying the prefit method to the GOME-2A pixels processed previously, it is observed that out of 255 pixels, 215 pixels retrieve and 40 pixels do not. Upon analysis of these 40 pixels, it is observed that these pixels do not converge because the retrieved aerosol optical thicknesses are in excess of 10.0, and DISAMAR stops the retrieval since τ reaches boundary conditions (beyond 20.0). Such large optical thicknesses may be attributed to the saturation of the top-of-atmosphere reflectance at very high aerosol loads, observed in Fig. 5. It is also possible that these retrievals do not converge because of the presence of other model errors. Two pixels retrieve above 10.0, and hence are not considered for retrieving .
From these 213 pixels, 209 pixels converge to , whereas four pixels do not converge to a solution. The four pixels that do not converge are confirmed cases of the presence of aerosol–surface ambiguities, since the retrieval toggles between two values at every iteration until the maximum number of allowable iterations is reached. This is also a consequence of a non-unique top-of-atmosphere reflectance at high aerosol load scenarios. Out of the 209 pixels that retrieve both and , 205 pixels have similar retrieved optical thickness values according to the criterion of Eq. (9). The rest have values which are off by more than 2.0.
From Fig. 8c, pixels that contain aerosol–surface ambiguities primarily lie within the main aerosol plume. This is in-line with our expectation of the top of atmosphere being saturated at very high aerosol loads. Interestingly, these pixels also comprise 50 % of the pixels that do not converge for aerosol layer height retrieval. Figure 9b provides a plot of the retrieval of CALIPSO co-located GOME-2A pixels, in which 22 pixels are absent from the plot (relative to Fig. 9a). These are pixels for which the prefit method retrieves different and .
Out of the 100 pixels that do not converge, 50 pixels have been identified which may be affected by aerosol–surface ambiguities. For a majority of these pixels, the retrieved aerosol optical thickness is typically beyond 4.0. It is possible that the true number of pixels that are affected by aerosol–surface ambiguities is higher than 50 pixels – our analysis is represented by a similarity criterion which relies on a similarity threshold T, which we have set at 15 % (Eq. 9). With a stricter criterion, more pixels affected by aerosol-surface ambiguities may be detected. Other non-convergences may be a result of model errors. Comparing our retrievals with the CALIOP attenuated backscatter profile from the infrared channel, we observe that our retrievals are biased closer to the surface, with non-convergences occurring for pixels within the primary biomass-burning plume.
Depending on the surface brightness, the interaction of photons scattered from the atmosphere and the surface can result in a possible reduction of available aerosol information in the oxygen A-band spectrum. Our basis for this assertion depends on the distinction of aerosol information present in atmospheric path contributions Rp and surface contributions Rs to the top-of-atmosphere reflectance in the spectrum (Fig. 2). The reduction of aerosol information increases with increasing surface brightness and decreasing viewing angle.
Our analyses reveal that the derivatives of the atmospheric path and surface contributions with respect to aerosol optical thickness are anti-correlated (see Fig. 3b), which affects the derivative of reflectance with respect to aerosol optical thickness (see Fig. 4). As the surface gets brighter, the magnitude of this derivative decreases, which reduces the sensitivity of the oxygen A-band spectrum to aerosol optical thickness. We expect this anti-correlation behaviour to be strong for viewing angles closer to the nadir, since Rp increases and Rs decreases with an increase in viewing angle (see Fig. 2). One of the consequences of this is the effect on cost function for retrieving aerosol optical thickness. We report that the gradient of the cost function tends to become shallower as the surface albedo increases. This is especially the case when the viewing angle is closer to the nadir (see Fig. 6). We also notice that the cost function reduces at high aerosol optical thickness beyond the local minimum near the truth (Fig. 6, right), which indicates the presence of multiple minima in the cost function. We attribute this behaviour to the saturation of the top-of-atmosphere reflectance at high aerosol loads (see Fig. 5).
Similar analyses on the available information on aerosol layer height in Rp and Rs in the oxygen A band reveals that parts of the oxygen A-band spectrum with a low absorption by oxygen have an increased cancellation of ΔRpΔz and ΔRsΔz (see Fig. 3a) and hence a reduction in aerosol layer height sensitivity in specific parts of the spectrum (see Fig. 7a). This increases as surface albedo increases. It is also observed that the derivatives of ΔRpΔω and ΔRsΔω are both positive (see Fig. 3c), which increases the overall sensitivity of the oxygen A-band spectrum to ω with increasing surface albedo. This is observed in the derivative of reflectance with respect to ω, which increases in magnitude with an increase in surface albedo.
The interaction between photons scattering back from the atmosphere (Rp) to the detector and photons that travel through the atmosphere to the surface and back to the detector (Rs) has direct consequences to the retrieval of aerosol layer height from the oxygen A band. Over bright surfaces, the retrieval algorithm becomes increasingly susceptible to errors in the aerosol layer height estimates as well as non-convergences in the presence of model errors (see Fig. 7). The sign difference of ΔRpΔz and ΔRsΔz also explains why retrieving an aerosol layer over bright surfaces with a 50 hPa thickness for the thicker layer (say 200 hPa thickness) can be biased closer to the ground (see Fig. 7a). To demonstrate this assertion in a real retrieval scenario, we have retrieved aerosol layer height over the 2010 Russian wildfires of 8 August 2010, using measured oxygen A-band spectra recorded by the GOME-2 instrument on board the Metop-A satellite. For validating our retrievals, we refer to lidar measurements by the CALIOP instrument on board the CALIPSO mission which records, among other measurements, attenuated backscatter at 1064 nm over the same wildfires scene a few hours after the GOME-2A acquisition. Comparison of co-located GOME-2A and CALIPSO pixels reveals that, in the case of both boundary and elevated aerosol layers, the retrieved aerosol layer height is biased closer to the surface. For pixels with a high aerosol load, the algorithm fails to converge to a solution (see Fig. 8). Over optically thick plumes, the retrieval becomes dependent on the a priori aerosol optical thickness (see Fig. 8c).
Following the work presented in this paper, our further goal is to apply the knowledge gained from this study in the development of the aerosol layer height retrieval algorithm for retrieving aerosols over land.
All GOME-2 data used for processing in this paper are freely available from the EUMETSAT data centre: https://eoportal.eumetsat.int. The CALIOP lidar profiles are freely available from the ICARE Data and Services Center at http://www.icare.univ-lille1.fr/calipso/. The GOME-2 LER used in this paper can be found at http://www.temis.nl/surface/gome2_ler.html. The software used for processing these data is proprietary and is hence not shared with public.
The authors declare no conflict of interests in the work expressed in this publication.
This research is partly funded by the European Space Agency (ESA) within the
EU Copernicus programme under the project name “Sentinel-4 Level-2 Processor
Component Development”, number AO/1-7845/14/NL/MP. We acknowledge EUMETSAT
for providing the GOME-2 L1b data.
Edited by: Andrew Sayer
Reviewed by: two anonymous referees
Chance, K. and Kurucz, R.: An improved high-resolution solar reference spectrum for earth's atmosphere measurements in the ultraviolet, visible, and near infrared, J. Quant. Spectrosc. Ra., 111, 1289–1295, https://doi.org/10.1016/j.jqsrt.2010.01.036, 2010. a
Chung, C. E. and Zhang, G. J.: Impact of absorbing aerosol on precipitation: Dynamic aspects in association with convective available potential energy and convective parameterization closure and dependence on aerosol heating profile: impact of aerosol on precipitation, J. Geophys. Res.-Atmos., 109, https://doi.org/10.1029/2004JD004726, 2004. a
Corradini, S. and Cervino, M.: Aerosol extinction coefficient profile retrieval in the oxygen A-band considering multiple scattering atmosphere. Test case: SCIAMACHY nadir simulated measurements, J. Quant. Spectrosc. Ra., 97, 354–380, https://doi.org/10.1016/j.jqsrt.2005.05.061, 2006. a
Dee, D. P., Uppala, S. M., Simmons, A. J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M. A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A. C. M., van de Berg, L., Bidlot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A. J., Haimberger, L., Healy, S. B., Hersbach, H., Hólm, E. V., Isaksen, L., Kållberg, P., Köhler, M., Matricardi, M., McNally, A. P., Monge-Sanz, B. M., Morcrette, J.-J., Park, B.-K., Peubey, C., de Rosnay, P., Tavolato, C., Thépaut, J.-N., and Vitart, F.: The ERA-Interim reanalysis: configuration and performance of the data assimilation system, Q. J. Roy. Meteor. Soc., 137, 553–597, https://doi.org/10.1002/qj.828, 2011. a
de Haan, J. F., Bosma, P. B., and Hovenier, J. W.: The adding method for multiple scattering calculations of polarized light, Astron. Astrophys., 183, 371–391, 1987. a
Ingmann, P., Veihelmann, B., Langen, J., Lamarre, D., Stark, H., and Courrèges-Lacoste, G. B.: Requirements for the GMES Atmosphere Service and ESA's implementation concept: sentinels-4/-5 and -5p, Remote Sens. Environ., 120, 58–69, https://doi.org/10.1016/j.rse.2012.01.023, 2012. a
IPCC: Clouds and aerosols, in: Climate Change 2013 – The Physical Science Basis, Cambridge University Press, Cambridge, 571–658, available at: http://ebooks.cambridge.org/ref/id/CBO9781107415324A024, https://doi.org/10.1017/CBO9781107415324.016, 2014. a, b
Landgraf, J., Hasekamp, O. P., Box, M. A., and Trautmann, T.: A linearized radiative transfer model for ozone profile retrieval using the analytical forward-adjoint perturbation theory approach, J. Geophys. Res.-Atmos., 106, 27291–27305, https://doi.org/10.1029/2001JD000636, 2001. a
Munro, R., Lang, R., Klaes, D., Poli, G., Retscher, C., Lindstrot, R., Huckle, R., Lacan, A., Grzegorski, M., Holdak, A., Kokhanovsky, A., Livschitz, J., and Eisinger, M.: The GOME-2 instrument on the Metop series of satellites: instrument design, calibration, and level 1 data processing – an overview, Atmos. Meas. Tech., 9, 1279–1301, https://doi.org/10.5194/amt-9-1279-2016, 2016. a
Rodgers, C. D.: Inverse Methods for Atmospheric Sounding: Theory and Practice, vol. 2, World Scientific, River Edge, NJ, 2000. a
Sanders, A. F. J. and de Haan, J. F.: Retrieval of aerosol parameters from the oxygen A band in the presence of chlorophyll fluorescence, Atmos. Meas. Tech., 6, 2725–2740, https://doi.org/10.5194/amt-6-2725-2013, 2013. a, b
Sanders, A. F. J. and de Haan, J. F.: TROPOMI ATBD of the Aerosol Layer Height product, available at: http://www.tropomi.eu/sites/default/files/files/S5P-KNMI-L2-0006-RP-TROPOMI_ATBD_Aerosol_Height-v1p0p0-20160129.pdf (last access: 9 January 2018), 2016. a, b
Sanders, A. F. J., de Haan, J. F., Sneep, M., Apituley, A., Stammes, P., Vieitez, M. O., Tilstra, L. G., Tuinder, O. N. E., Koning, C. E., and Veefkind, J. P.: Evaluation of the operational Aerosol Layer Height retrieval algorithm for Sentinel-5 Precursor: application to O2 A band observations from GOME-2A, Atmos. Meas. Tech., 8, 4947–4977, https://doi.org/10.5194/amt-8-4947-2015, 2015. a, b, c, d, e, f
Sanghavi, S., Martonchik, J. V., Landgraf, J., and Platt, U.: Retrieval of the optical depth and vertical distribution of particulate scatterers in the atmosphere using O2 A- and B-band SCIAMACHY observations over Kanpur: a case study, Atmos. Meas. Tech., 5, 1099–1119, https://doi.org/10.5194/amt-5-1099-2012, 2012. a
Tilstra, L. G., Tuinder, O. N. E., Wang, P., and Stammes, P.: Surface reflectivity climatologies from UV to NIR determined from Earth observations by GOME-2 and SCIAMACHY: GOME-2 and SCIAMACHY surface reflectivity climatologies, J. Geophys. Res.-Atmos., 122, 4084–4111, https://doi.org/10.1002/2016JD025940, 2017. a, b
Tran, H. and Hartmann, J.-M.: An improved O2 A band absorption model and its consequences for retrievals of photon paths and surface pressures, J. Geophys. Res.-Atmos., 113, D18104, https://doi.org/10.1029/2008JD010011, 2008. a, b
Tran, H., Boulet, C., and Hartmann, J.-M.: Line mixing and collision-induced absorption by oxygen in the A band: laboratory measurements, model, and tools for atmospheric spectra computations, J. Geophys. Res., 111, https://doi.org/10.1029/2005JD006869, 2006. a
Veefkind, J. P., Aben, I., McMullan, K., Förster, H., de Vries, J., Otter, G., Claas, J., Eskes, H. J., de Haan, J. F., Kleipool, Q., van Weele, M., Hasekamp, O., Hoogeveen, R., Landgraf, J., Snel, R., Tol, P., Ingmann, P., Voors, R., Kruizinga, B., Vink, R., Visser, H., and Levelt, P. F.: TROPOMI on the ESA Sentinel-5 Precursor: A GMES mission for global observations of the atmospheric composition for climate, air quality and ozone layer applications, Remote Sens. Environ., 120, 70–83, https://doi.org/10.1016/j.rse.2011.09.027, 2012. a
Winker, D. M., Vaughan, M. A., Omar, A., Hu, Y., Powell, K. A., Liu, Z., Hunt, W. H., and Young, S. A.: Overview of the CALIPSO Mission and CALIOP Data Processing Algorithms, J. Atmos. Ocean. Tech., 26, 2310–2323, https://doi.org/10.1175/2009JTECHA1281.1, 2009. a
Witte, J. C., Douglass, A. R., da Silva, A., Torres, O., Levy, R., and Duncan, B. N.: NASA A-Train and Terra observations of the 2010 Russian wildfires, Atmos. Chem. Phys., 11, 9287–9301, https://doi.org/10.5194/acp-11-9287-2011, 2011. a