Emissivity retrievals with FORUM's end-to-end simulator: challenges and recommendations
- 1ESA – ESTEC, Keplerlaan 1, 2201 AZ Noordwijk, the Netherlands
- 2Space and Atmospheric Physics Group, Department of Physics, Imperial College London SW7 2AZ, United Kingdom
- 3Dipartimento di Fisica e Astronomia, Università di Bologna, Viale Berti Pichat 6/2, 40126 Bologna, Italy
- 4IFAC – CNR, Via Madonna del Piano 10, 50019 Sesto Fiorentino (FI), Italy
- 5IAC – CNR, Via Madonna del Piano 10, 50019 Sesto Fiorentino (FI), Italy
Correspondence: Maya Ben-Yami (firstname.lastname@example.org)
Spectral emissivity is a key property of the Earth's surface, of which only very few measurements exist so far in the far-infrared (FIR) spectral region, even though recent work has shown that the FIR is important for accurate modelling of the global climate. The European Space Agency's 9th Earth Explorer, FORUM (Far-infrared Outgoing Radiation Understanding and Monitoring) will provide the first global spectrally resolved measurements of the Earth's top-of-the-atmosphere (TOA) spectrum in the FIR. In clear-sky conditions with low water vapour content, these measurements will provide a unique opportunity to retrieve spectrally resolved FIR surface emissivity. In preparation for the FORUM mission with an expected launch in 2027, this study takes the first steps towards the development of an operational emissivity retrieval for FORUM by investigating the sensitivity of the emissivity product of a full spectrum optimal estimation retrieval method to different physical and operational parameters. The tool used for the sensitivity tests is the FORUM mission's end-to-end simulator. These tests show that the spectral emissivity of most surface types can be retrieved for dry scenes in the 350–600 cm−1 region, with an absolute uncertainty ranging from 0.005 to 0.01. In addition, the quality of the retrieval is quantified with respect to the precipitable water vapour content of the scene, and the uncertainty caused by the correlation of emissivity with surface temperature is investigated. Based on these investigations, a road map is recommended for the development of the operational emissivity product.
The European Space Agency's 9th Earth Explorer, FORUM (Far-infrared Outgoing Radiation Understanding and Monitoring; Palchetti et al., 2020) is scheduled to launch in 2027. FORUM will provide spectrally resolved measurements of the Earth's top-of-the-atmosphere (TOA) outgoing radiation from 100 to 1600 cm−1, with the goal of filling the observational gap in the far-infrared (FIR; defined here as below 667 cm−1). Even though simulations suggest that around 50 % of the outgoing longwave radiation (OLR) to space is in the FIR in the global mean, due to technical reasons it has never been observed from satellite, spectrally resolved, in its entirety. FORUM's novel measurements will be provided by the mission's core instrument, a nadir-viewing Fourier transform spectrometer (FTS), which will measure the Earth's upwelling spectral radiance. While the primary goal of FORUM is to provide these calibrated spectral radiances, its further aim is to exploit instantaneous radiance observations to retrieve atmospheric and surface properties (Level 2 products).
FORUM clear-sky radiances will be used to retrieve temperature and water vapour profiles, as well as FIR surface emissivity and surface temperature. This work focuses on the retrieval of surface emissivity, which is the material property determining how much thermal radiation a surface emits at a given temperature. For a surface (or skin) temperature Ts, it is defined as the ratio of surface emission to the blackbody emission at Ts. Emissivity is not constant across the spectrum, and the emissivities of different surfaces exhibit distinct spectral variation. The possibility to retrieve spectrally resolved FIR emissivity with FORUM is particularly exciting given its potential influence on the surface and top-of-atmosphere energy budget (Feldman et al., 2014; Kuo et al., 2018).
Surface emissivity across the globe is routinely retrieved in the mid-infrared (MIR) from satellite observations (Susskind et al., 2014; Capelle et al., 2012; Masiello and Serio, 2013; Wan, 2014; Wang et al., 2005). These are complemented by laboratory measurement datasets such as the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) spectral library, which includes more than 2300 different spectral emissivities down to 650 cm−1 (Baldridge et al., 2009). However, in the FIR, no global retrievals of surface emissivity are available due to the absence of spectrally resolved TOA radiance observations, and there is also a lack of laboratory measurements. Bellisario et al. (2017) and Murray et al. (2020) were the first to retrieve FIR snow emissivities from aircraft measurements (during the CIRCCREX/COSMICS projects over Greenland), confirming the feasibility of retrieving FIR surface emissivity from OLR spectral measurements. However, in these studies, no existing theoretical snow/ice model fit the retrieved emissivity values in the MIR and FIR simultaneously, indicating that further testing of the theoretical models using global emissivity retrievals is vital to extend surface emissivity datasets into the FIR. While other planned FIR measurements, such as the Polar Radiant Energy in the Far Infrared Experiment (PREFIRE; L’Ecuyer et al., 2021) and ground-based measurements of snow emissivity (see Palchetti et al., 2021), will contribute to our understanding of FIR emissivity, only FORUM will be able to provide such global retrievals.
The potential value of knowing the spectral variation in surface emissivity in the FIR is significant. In recent years, there has been increasing focus on the inadequate representation of surface emissivity in global climate models (GCMs), which almost all assume blackbody or greybody emissivity (Huang et al., 2018). To test the validity of this assumption in the FIR, Feldman et al. (2014) incorporated spectrally varying FIR emissivity into the Community Earth System Model I (CESM I) and showed significant changes in its predictions after 25 years. At high latitudes, as much as a 2 K change in surface temperature and 10 Wm−2 in the outgoing longwave radiation occurred. The authors also identified a possible feedback mechanism associated with FIR emissivity, namely that in the FIR the emissivity of snow can be substantially higher than that of water (while in the MIR the difference is less significant; see Fig. 1). This means that, as sea ice melts in a warming climate, it exposes a potentially less emissive water surface, exacerbating the warming. Further work with the CESM has confirmed that this feedback is present, if small, and has shown that the inclusion of realistic surface emissivity in fact significantly reduces the persistent cold pole bias of climate models (Kuo et al., 2018; Huang et al., 2018). Critically, by comparing the assumption of ice vs. snow emissivity in the models, it was shown that the size and sign of the feedback depends on the properties of the surface (Huang et al., 2018).
While work has already been done to analyse the performance of the geophysical products (including emissivity) expected from FORUM clear-sky measurements (e.g. Ridolfi et al., 2020; Sgheri et al., 2022), in this study, we focus on spectral surface emissivity and investigate its retrieval using the FORUM mission's end-to-end simulator (FEES) described in Sgheri et al. (2022). In addition to investigating the retrieval parameters, this work focuses on the influence of atmospheric water vapour, as it is one of the most important factors influencing the transmission in the FIR (Harries et al., 2008) due to the dominance of the water vapour rotational band on atmospheric absorption in this region (see Sect. 5).
The paper is structured as follows: Sect. 2 describes the FEES and Sect. 3 the experimental set-up. In Sect. 4, the general FEES retrieval result is introduced, together with the different quantifiers used for its analysis. The parameters are then investigated in the following two steps:
In Sect. 5, the water vapour profile in the forward model is modified to compare emissivity retrieval quality against scene humidity.
In Sect. 6, the parameters of the retrieval algorithm associated with surface temperature and emissivity are investigated, i.e. the retrieval a priori, the a priori uncertainty and the initial guess.
Finally, Sect. 7 summarizes the results, focusing on the main challenges and on the recommendations this study has for further development towards an operational emissivity retrieval for FORUM. The three appendixes provide more detail on the emissivity–surface temperature parameter space (Appendix A), the choice of the emissivity a priori uncertainty (Appendix B) and the spectral dependence of the emissivity–surface temperature correlation (Appendix C).
As this work is meant to provide the first steps towards the development of an operational emissivity retrieval for FORUM, the focus is placed on investigating the effect of various factors on the retrieval of a range of typical scenes. The aim of an operational retrieval is to provide the users with a retrieved product that is transparent and accessible, together with a realistic uncertainty estimate. Thus, the focus of this work is not on extreme cases or on optimizing the retrieval for specific scenes but rather on highlighting general features which need to be investigated in the years up to the expected launch in 2027.
The FORUM mission's end-to-end simulator (FEES) constitutes a chain of modules which simulate the elements relevant to the mission performance. A full description of the FEES can be found in Sgheri et al. (2022), together with a discussion of the geophysical products not shown in this work. Our study uses the following five modules of the simulator: the Geometry Module (GM), the Scene Generator Module (SGM), the FORUM Sounding Instrument (FSI) module, the FORUM Embedded Imager (FEI) module and the Level 2 Module (L2M). For the purpose of this work, the first four modules are run in the default chain (see Sgheri et al., 2022) to generate synthetic FORUM observations for various geographic scenes in clear-sky conditions. The L2M uses these synthetic observations to retrieve the geophysical properties of the scene, and in this work, this retrieval algorithm is tested with a focus on the retrieved spectral surface emissivity.
The L2M retrieves the atmospheric state from the synthetic FORUM measurements using the optimal estimation (OE) method, which deals with the ill-posed nature of the inverse problem using an a priori regularization (Rodgers, 1976, 2000). Starting from an initial guess of the n-dimensional atmospheric state vector x, the algorithm arrives at a best estimate by minimizing the cost function ξ2 as follows:
The first term on the right-hand side is the χ2 of the forward model, which is in essence the difference between the m-dimensional observation vector y and the forward model f(x) calculated from the atmospheric state vector x, where the covariance matrix Sy represents the uncertainty on the observations. The second term is the regularization term, which takes into account the difference of the state vector x from an a priori (model) atmospheric state xa with uncertainty Sa. For more details on the forward model and minimization technique, see Sgheri et al. (2022, 2020), and for the parameters and assumptions used in this work, see Sect. 3.
To understand the parameters influencing the quality of the retrieved emissivity, it is useful to keep in mind the role emissivity plays in the forward model, which is in the simulation of the atmospheric radiative transfer. For nadir-looking observations, the clear-sky TOA spectral radiance Stoa,σ at wavenumber σ can be written as follows:
where B(T) is the Planck function, T(z) is the atmospheric temperature profile, 𝒯(z) is the transmittance between the surface and height z, and the integral is over the height z from the surface z0 to the TOA z1. 𝒯(z1) is the transmittance from the surface to the TOA. The emissivity contributes to the surface part of the radiance as follows:
Here Ld,σ is the downwelling radiance at the surface, Ts is the surface (or skin) temperature, and ϵσ is the emissivity of the surface at wavenumber σ. Following the reasoning from Bellisario et al. (2017), in this work the emissivity is always assumed to have no directional dependence.
All the results presented in this work are the products of FEES runs. A complete description of this simulator and its modules can be found in Sgheri et al. (2022), and unless otherwise stated, the same parameters and settings are used as described in that work for homogeneous clear-sky cases.
3.1 FEES modules
Only the first five modules of the FEES are used in this work. The Scene Generator Module (SGM), which uses geographic coordinates provided by a Geometry Module, computes high-resolution TOA spectral radiances in clear sky conditions using the radiative transfer model LBLRTM version 12.8 (Clough et al., 2005) and auxiliary databases prepared for the FEES. For a detailed description of the auxiliary datasets, see Sgheri et al. (2022), but for reference, note that the water vapour and temperature profiles and the surface temperature are taken from ERA5 reanalysis data (Hersbach et al., 2020). In this work, all scenes used are from 15 January 2018 at 12:00:00 UTC for consistency, and they are identified using their geographic coordinates (see Table 1). The emissivity dataset used by the SGM is based on the geolocated dataset of spectral emissivity by Huang et al. (2016) and uses the 11 surface types defined by Huang et al. (2016) (out of the multiple desert subtypes, the re=30 µm subtype is used). Each scene is generated using the surface type out of these 11 that best match the January value in the geolocated dataset for the given coordinates. A total of 7 of these 11 surface types can be seen in Fig. 1.
The third and fourth modules of the FEES simulate the observing system. The only change made to these modules is to vary a so-called seed used to generate the random noise associated with the FORUM Sounding Instrument (FSI). The synthetic observations thus generated and the variation in random noise is illustrated in Fig. 2.
The final module used is the L2M, which has been described in more detail above in Sect. 2. This is the module used to test emissivity retrieval properties and in which the major modifications were made.
3.2 The baseline retrieval parameters
In this work, the retrieved atmospheric state vector x constitutes the atmospheric water vapour profile, the temperature profile, the spectral surface emissivity and the surface temperature. For the purpose of this study, we define a baseline/default retrieval case, which is used as the basis for all modifications and tests. Unless otherwise stated, all parameters are the standard parameters for clear-sky retrievals in Sgheri et al. (2022). Only two parameters differ between the baseline retrieval in this study and the standard of Sgheri et al. (2022). For the emissivity a priori, this work uses a flat a priori value instead of a perturbed climatological one and a 0.1 uncertainty instead of 0.05 (see Sect. 3.3 and Appendix B for a justification of these choices).
For comparison with later modifications, some of these baseline parameters are listed as follows:
Emissivity initial guess, which is constant and equal to 1
Emissivity a priori, which is constant and equal to 1
Emissivity a priori uncertainty matrix, which is defined using uncertainty Δϵ=0.1 and correlation length (CL) of 50 cm−1 (see Appendix B for an explanation of these terms and a justification of the choice of uncertainty matrix)
Emissivity retrieval grid, which is an evenly spaced 5 cm−1 grid for the full FORUM spectral range
Surface temperature initial guess, which is the climatological value from ERA5 monthly averages (different from the daily value used for the SGM)
Surface temperature a priori, which is a random perturbation of the true value with a 2 K standard deviation (the perturbation is the same for the same geographical scene)
Surface temperature a priori uncertainty, which is 2 K.
In the baseline retrieval, the same instrumental noise is used for all cases (i.e. the seed used to generate the instrumental random noise is kept the same at a value of 0; see Fig. 2).
To test the retrieval of surface emissivity in the FIR, we choose to use geographic scenes with low precipitable water vapour (pwv), which is defined as the depth of water produced if all water in the atmospheric column precipitated as rain. For reference, the full list of scenes used in the tests shown in this work can be found in Table 1, together with some of their relevant atmospheric and surface properties.
3.3 The emissivity a priori and deviation from classical optimal estimation
In the classical optimal estimation method from Rodgers (2000), the solution represents the estimate of maximum a posteriori probability. In the remote sensing community, xa is usually an a priori climatology, and so Sa typically represents the natural variability in these climatologies. However, the formalism of the method can be used without giving a probabilistic interpretation to xa and Sa and simply tuning them to best regularize the retrieval (von Clarmann et al., 2020). For example, the smaller the uncertainties in Sa are, the closer the solution will be on average to xa; this can be thought of as giving the retrieval more or less freedom to converge to the true state.
In this work, we deviate from the classical OE method and do not use climatological datasets for the emissivity part of the a priori vector (climatologies are still used for the rest of xa). Instead, a constant emissivity a priori is chosen in combination with a larger uncertainty. This is done to ensure consistency across cases and allow for easier comparison of different retrieval setups. To cover the range of possible theoretical emissivity model values in the considered spectral region, the emissivity submatrix of Sa is chosen to ensure that the emissivity retrieval has the freedom to converge to any physical value (between 0.7 and 1), provided there is enough sensitivity in the measurements. Therefore, the choice of a constant a priori value is rather arbitrary, and the baseline value is taken to be 1 for simplicity. The chosen parameters defining the emissivity submatrix of Sa are an uncertainty Δϵ=0.1 and correlation length (CL) of 50 cm−1 (see Appendix B for an analysis of these).
Future steps in the development of the FORUM operational retrieval can improve on this approach by the use of scene classification and by developing FIR emissivity climatologies for these scenes, as is already implemented in the MIR for retrievals of emissivity from MODIS (Moderate Resolution Imaging Spectroradiometer) observations (Borbas et al., 2018; Feltz et al., 2018; Loveless et al., 2020).
The retrieval process can only give information on a retrieved quantity where the forward model is sensitive to this quantity and returns the a priori where there is no sensitivity. For retrievals of surface emissivity from TOA spectral measurements, this is determined by the atmospheric transmission. For high water vapour content, the TOA is opaque to the surface in the FIR but becomes more transparent as the atmosphere gets drier. The distinct characteristic of atmospheric transmission in the FIR is that, as the pwv decreases, transmittance does not increase uniformly but in so-called microwindows, which become deeper as pwv decreases. To illustrate the typical pattern of this sensitivity in the FIR, four different quantifiers are defined in this section, and their behaviour for varying water vapour content is analysed.
4.1 The quantifiers
A total of four different quantifiers are described in the following section to illustrate retrieval quality. These are shown together with the TOA transmittance in Fig. 3 for the baseline retrieval of the scene at 67∘ N, 18∘ E.
Figure 3a shows the baseline retrieved emissivity, which is the emissivity part of the best estimate atmospheric state vector introduced in Sect. 2. Note that the emissivity is retrieved on a 5 cm−1 spectral grid, which is much coarser than the ∼0.4 cm−1 resolution of the synthetic observations, and thus, the emissivity ϵσ used in the atmospheric radiative transfer calculations of the forward model (Eq. 3) is in fact a linear interpolation of the emissivity elements of the retrieval vector .
where Sy and Sa are as in Eq. (1), and K is the Jacobian of the full forward model at convergence with respect to the retrieval vector. The retrieval standard deviation σx is the square root of the diagonal of Sx. σx is called the retrieval uncertainty in this work, while the systematic uncertainty is defined as the true value minus the retrieved value.
The second quantifier shown in Fig. 3b makes further use of the information contained in σx, in particular that, in regions where there is no sensitivity, the retrieval vector will equal the a priori, and the σx will equal the a priori uncertainty σa. Recognizing this, Dinelli et al. (2009) defined the information quantifier (IQ) as follows:
where σx is as above, and σa is the square root of the diagonal of the retrieval a priori covariance matrix Sa. The IQ thus tends to 0 in regions with low sensitivity as σx approaches σa. Note that, while the IQ can be defined for the full retrieval vector, in this work it is only used for the retrieved emissivity.
The third quantifier is the Jacobian of the TOA radiances with respect to emissivity. While Eq. (4) uses the Jacobian K with respect to the full forward model, to directly quantify the emissivity retrieval quality, a different Jacobian J is used, which is calculated with respect to the radiative transfer simulation at convergence as follows:
where are the emissivity values used in the radiative transfer calculations of LBLRTM, and are the resulting TOA radiances at wavenumbers σi and σj (see Eqs. 2 and 3 for their physical definitions). From Eq. 3, we can see that, at the measurement spectral resolution Jij is diagonal in the emissivity, and so the diagonal Jii values are plotted in Fig. 3c.
The final quantifier is the averaging kernel A, which is frequently used to evaluate OE retrievals (see Rodgers, 2000; von Clarmann et al., 2020) and gives more information on the retrieval process itself. In the following, A is defined as the derivative of the retrieved atmospheric state vector with respect to the true state vector x (where x is the interpolation of the true atmospheric components onto their respective retrieval grids):
Considering the diagonal submatrix of A that corresponds to emissivity in the retrieval vector, the rows of that submatrix represent the sensitivity of the retrieved emissivity at a particular wavenumber to the true emissivity at all wavenumbers. These emissivity submatrix rows are plotted in Fig. 3d. A approaches the identity matrix I when the contribution of the a priori is negligible with respect to the measurements.
The scene shown in Fig. 3 has a pwv content of 3.55 mm and so, as discussed above, its retrieval is sensitive to the surface in the FIR. The quantifiers in Fig. 3b–d and the transmittance in Fig. 3e thus show the distinct pattern of the TOA's sensitivity to the surface in such dry atmospheric scenes, as follows:
The significant transmission in the FIR below the CO2 absorption band (≲ 600 cm−1), which is the so-called dirty window of the water vapour rotational band where the emission is still strong but the transmission is in microwindows. The microwindow structure can clearly be seen in the Jacobian and is also reflected in the varying strength of the averaging kernel.
The low sensitivity, below 400 cm−1, as the absorption of the water vapour rotational band increases.
The uniform transmittance in the MIR atmospheric window, resulting in an averaging kernel close to 1.
A small decrease in sensitivity in the ozone band around 1000 cm−1.
The decreasing sensitivity at MIR wavenumbers higher than 1200 cm−1 because of a combined increase in noise in the measurements and absorption by water vapour.
The lack of sensitivity in the CO2 band between roughly 600 and 750 cm−1.
4.2 Spectral quantifiers and water vapour content
These quantifiers can be used to investigate how the retrieval quality changes across the spectral range as atmospheric or retrieval parameters are modified. This is illustrated here for varying pwv content. The scene at 67∘ N, 18∘ E was modified by multiplying its climatological water vapour profile by a range of constant factors and generating synthetic observations from these modified scenes (thus resulting in pwv content ranging from 0.4 to 17.8 mm). The baseline retrieval is run for six such modified scenes, and the retrievals and their quantifiers are shown in Fig. 4.
Figure 4 shows that, while the pwv does not effect the basic spectral characteristics of the quantifiers and the retrieval sensitivity in the MIR, it is an important factor determining the sensitivity to emissivity in the FIR. The Jacobians in Fig. 4c show that, while the transmission maintains its microwindow structure in the FIR, these windows gradually weaken and disappear as the pwv content is raised. This is reflected in the averaging kernels in Fig. 4d, where, at low pwv, the retrieved emissivity in the FIR has high sensitivity to the true value, but this sensitivity decreases to almost 0 for the highest pwv content. The consequence of this change for the retrieval result itself can clearly be seen in Fig. 4a. As noted above, where there is no sensitivity to the true value, the retrieval uncertainty will approach the a priori uncertainty (here 0.1), and Fig. 4a shows this. For dry scenes there is a small retrieval uncertainty as low as 300 cm−1, while at high pwv the retrieval uncertainty is equal to the large a priori uncertainty value through most of the FIR. Thus, the spectral region where the emissivity values are in fact retrieved changes depending on the pwv.
4.3 The emissivity product
For clarity of analysis, it is useful to plot and investigate an emissivity product from the retrieval vector that represents only values with information on the true emissivity. In this work, the IQ is used to define such a criterion, following Dinelli et al. (2009), although the diagonal of the averaging kernel could also be used. Here the emissivity range shown and considered as retrieved (i.e. in regions of sensitivity) is that for which, in the following:
While, in practice, FORUM users could be provided with the full retrieval and uncertainty vectors, in this work the criterion in Eq. (8) is used to ease interpretation. Figure 5 shows this retrieved emissivity product for eight dry geographic scenes with different surface emissivities. Only scenes with pwv below 5 mm are shown here to demonstrate the viability of FORUM FIR emissivity retrievals (for FEES emissivity retrievals of scenes with pwv higher than 5 mm, see Sgheri et al., 2022). As already seen in Fig. 3, the emissivity in dry scenes is retrieved in two sections above and below the CO2 band, with the uncertainty in the retrieval highest in the edge regions of these sections. Figure 5 thus illustrates the potential of FORUM to retrieve FIR emissivity for a range of surface types and locations on the globe.
In this section, the analysis of the variation in the retrieval quality with water vapour content shown in Fig. 4 for the scene at 67∘ N, 18∘ E is extended and compared for multiple geographic scenes. The procedure for modifying the pwv content is identical. Leaving all other atmospheric and surface properties untouched, the climatological water vapour profile of the scene was multiplied by a constant value (ranging from 0.05 to 120). The four scenes (25∘ N, 09∘ E; 21∘ N, 15∘ E; 67∘ N, 18∘ E; and 67∘ N, 29∘ E) and the corresponding maximum and minimum water vapour profiles used can be seen in Fig. 6a. The synthetic observations generated from these modified scenes were then used to run the baseline retrieval (see Sect. 3). Although these modified scenes included some non-physical water vapour profiles, there was no significant change in the retrieval quality of the atmospheric profiles.
In Sect. 4, it was seen that as pwv decreased the retrieval quality at a given FIR wavelength improved as microwindows deepened and the retrieval sensitivity extended farther into the FIR as new microwindows opened up. To complement the spectral analysis of Fig. 4 and compare the variation in quality for multiple scenes, in this section three single-value quantifiers are analysed for the retrievals. All three are shown in Fig. 6 and plotted against the true pwv content of the scene.
The first quantifier in Fig. 6b shows the lowest wavenumber of retrieval sensitivity into the FIR by plotting the minimum wavenumber which satisfies the criterion for retrieval (see Eq. 8). The data for 67∘ N, 18∘ E are also listed in Table 2. This wavenumber value decreases as the scene becomes drier and the weaker microwindows become transparent enough for the emissivity to be retrieved at lower wavenumbers. The second quantifier in Fig. 6c shows the root mean square (RMS) error of the retrieved emissivity in the 500–600 cm−1 region for the cases that are fully sensitive in that region. While the region in which the emissivity is being retrieved in the FIR can be larger than 500–600 cm−1 for many of these cases, the RMS is calculated for a constant region to avoid the influence from the fluctuations at the edge of the sensitive regions. Figure 6c shows that, not only does the lowest wavenumber of sensitivity decrease, but the retrieval quality also increases as the scene becomes drier. The final quantifier in Fig. 6d shows the degrees of freedom of the emissivity retrieval in the full 100–667 cm−1 FIR region, calculated from the averaging kernel matrix. It is noteworthy that, unlike the other qualifiers which have occasional plateaus in their trends, the information content in the FIR increases monotonically as the pwv decreases.
All cases individually show the same improvement in quality, with pwv discussed in detail for Fig. 4, and the results are only weakly dependent on the scene. However, there is a small difference in the scene specific behaviour in all three plots, of which Fig. 6d gives the clearest view. In general, for the same value of pwv, 25∘ N, 09∘ E has the best retrieval quality, with 21∘ N, 15∘ E next in quality and 67∘ N, 18∘ E and 67∘ N, 29∘ E lowest and about equal in quality. Although the many parameters of the atmospheric state and the small number of scenes investigated make attribution of this difference difficult, a plausible explanation can still be identified. The difference in surface temperature and surface–atmosphere contrast between these scenes. The hot scenes are 25 and 21∘ N (Ts>300 K; see Table 1), and their higher surface temperatures lead to a larger sensitivity to emissivity through the stronger Ts–emissivity correlation (see Sect. 6 and Appendix A). And though the 21∘ N scene surface temperature is in fact 4 K warmer than the 25∘ N surface temperature, the temperature contrast with respect to the atmosphere is 12.7 and 15 K in the scenes, respectively. A larger difference between the air and surface could mean that the surface emission is easier to separate from the atmospheric emission and would also reduce the reflected downwelling radiation. Further work should extend the analysis to a larger number of geographic scenes to better quantify this effect.
Overall, the analysis of Fig. 6 shows that FORUM measurements will provide significant information on emissivity in the FIR in a range of scenes.
The difficulty in surface emissivity retrieval caused by the connection of emissivity to surface temperature is widely recognized in the field of remote sensing (Li et al., 2013). In many cases, one is only interested in either emissivity or surface temperature, but Eq. (3) shows that, from radiance measurements, these cannot be determined independently. Even if one is only interested in the surface properties, the difficulty in Eq. (3) arises from two sources, namely imperfect knowledge of 𝒯(z), the atmospheric transmittance between the surface and the instrument and at the measurement resolution the degeneracy of the surface emission itself with regards to the parameters of interest. The FEES retrieves the surface temperature and the atmospheric state that defines 𝒯(z) at the same time as the spectral emissivity. The contribution of water vapour to 𝒯(z) was discussed in Sects. 4 and 5. This section focuses on the Ts–emissivity correlation that arises from Eq. (3) and investigates its impact on the retrieved emissivity. To complement the general analysis of this section, the spectral dependency of this correlation strength is discussed in Appendix C.
6.1 Surface temperature and emissivity in the surface emission equation
The surface emission equation (Eq. 3), as written, is degenerate. Even if the atmospheric state is known and so Ld,σ is given, measurements of Ssurf,σ at N wavenumbers still leave N+1 unknowns to solve for, i.e. N spectral emissivity values and the surface temperature Ts. The constraint is that the surface emissivity ϵσ≤1 checks this degeneracy and provides a lower bound for the retrieved Ts. However, for any higher values of Ts, it is possible to find a corresponding surface spectral emissivity ϵσ that produces the correct surface radiance.
Different methods have been developed to deal with this degeneracy in the MIR when it occurs (see Li et al., 2013 for a review). While most methods make assumptions or use empirical relations which cannot be extended into the FIR, as Murray et al. (2020) and Bellisario et al. (2017) have shown, MIR measurements can be used to retrieve a Ts, which can then be used for the FIR emissivity retrieval. Future work could investigate such methods by incorporating independent MIR measurements from synergy with the Infrared Atmospheric Sounding Interferometer – New Generation (IASI-NG) in tandem with the full-spectrum simultaneous OE retrieval used in this work.
In the FEES OE retrieval, the assumption that breaks the degeneracy of Eq. (3) is the retrieval of emissivity on a coarser grid than the measurements. As discussed in Sect. 4, the ∼ 0.4 cm−1 spaced ϵσ used to calculate Stoa,σ is computed by linearly interpolating between the emissivity values retrieved on a coarser 5 cm−1 grid. Thus, the retrieval vector has fewer elements than the observations vector y, and the retrieval is not ill-posed, only ill-conditioned. This interpolation uses the assumption that the emissivity is smooth, and so breaks the degeneracy in a similar way to the retrieval method seen in Murray et al. (2020) and Knuteson et al. (2004). If in the FEES OE forward model the emissivity and Ts move away from the true value, to keep Ssurf,σ the same in Eq. (3), the spectral emissivity would have to take up a shape with sharp high-resolution spectral features corresponding to the spectral pattern of Ld,σ. These cannot be reproduced by the interpolated coarser grid, and so ξ2 is larger value farther away from the correct emissivity. Thus, the smoothing means that an incorrect emissivity introduces errors in the forward model, and this penalization leads the algorithm to nudge the retrieval vector towards the true value.
However, for small shifts away from the true emissivity and true Ts, the errors introduced in Ssurf,σ can be within the FORUM instrumental uncertainty. Thus, to a limited extent, the functional form of the emissivity and surface temperature still allows the retrieval to converge to a range of different emissivities. Such a parameter combination is sometimes called sloppy, as moving along a sloppy direction in the parameter space has little effect on the behaviour of the model (see Transtrum et al., 2011). The combination of Ts and emissivity form a sloppy valley in the model parameter space.
Figure 7 is shown both as an illustration of how surface temperature and emissivity compensate for each other and as a comparison of different a priori constraint scenarios. The retrieval of scene 67∘ N, 29∘ E with instrumental noise seed 0 was specifically chosen for this figure due to the ∼ 0.01 shift seen in the default retrieval, and it is not necessarily a representative case.
As mentioned above, it is likely that, for operational FORUM retrievals, an estimate of Ts will be available either from independent observations, from synergy with IASI-NG, or from a different analysis of the FORUM observations. Thus, the retrieval is run for four different scenarios of the surface temperature a priori information, as follows:
The default FEES retrieval, where a perturbation of the true Ts is used as a priori with a 2 K a priori uncertainty that is characteristic of surface temperature measurements.
To model the ideal scenario of correct and accurate independent measurements, the true Ts is used as both a priori and initial guess, with a smaller 0.5 K a priori uncertainty.
A similar but less realistic scenario in which a high confidence in the independent measurement of the true Ts means that the true value is set as both a priori and initial guess as in (ii), but in this case with a 0.1 K a priori uncertainty.
To test whether using a tight a priori constraint is advisable, the final retrieval uses the perturbed Ts of (i) as the a priori and initial guess, with the 0.1 K a priori uncertainty of (iii).
The first thing to note from the figure is the expected anti-correlation of the surface temperature and emissivity systematic uncertainties in the retrieved values. Out of the four cases, only retrieval (iii) has a retrieved surface temperature centred on the true value, with (i) and (ii) having lower and (iv) higher retrieved surface temperatures. These shifts in Ts cause upward/downward shifts of the whole spectral emissivity, with sign and size anti-correlated with the systematic uncertainty in surface temperature. It is interesting to note that, even though the emissivity retrieval is shifted for the different cases, the emissivity retrieval uncertainty is the same for all of them, and when examined, none of the standard quantifiers (see Fig. 3) show which retrieval is better than the other. The reason can be seen in Fig. 7a. All of these solutions are in the same sloppy valley of the parameter space and so reproduce the observations to the same accuracy within the FORUM goal noise. This illustrates the effect that the functional form (ϵB(Ts)) of Ts and emissivity in the forward model can have on the retrieval.
Are the imposed constraints on Ts useful for mitigating such compensating shifts and reducing the systematic uncertainty on emissivity? There are two points to be made from the cases in Fig. 7, as follows:
Even a constraint of ±0.5 K around the true value of Ts does not correct the shift seen in the default retrieval and can still result in an emissivity retrieval in which the true emissivity is outside the ±1σ retrieval uncertainty range (but it should be noted that it is within both ±2σ and the goal FORUM emissivity uncertainty of ±0.01).
Scenario (iii) shows that a constraint of ±0.1 K is sufficiently small to result in the correct retrieved emissivity. However, scenario (iv) shows that this is too tight of a constraint; if the a priori Ts value is inaccurate even by ±1.5 K, then this already causes a much larger shift in the retrieved emissivity than is seen in the default scenario with more freedom for Ts. It is, therefore, not recommended to use such a tight a priori constraint.
6.2 Impact on the retrieval by the a priori and initial guess choices
The retrievals shown in the previous section investigated possible Ts a priori constraints. This section investigates the impact allowed by the correlation of surface temperature and emissivity when varying the value of the emissivity initial guess and a priori without changing the a priori uncertainty constraints.
To explore the individual effects of the emissivity a priori and initial guess on the retrieved emissivity, their values are varied independently. The baseline retrieval was run for a combination of different constant a priori cases and initial guesses for four different geographical scenes, and the results are shown in Fig. 8. The impact of the different combinations is shown by shading in the range between the maximum and minimum of systematic uncertainties in the retrieved emissivities for three colour-coded scenarios, as well as shading in the maximum retrieval uncertainty range in grey. These scenarios are as follows:
The initial guess is kept constant at 0.9, and the a priori is varied in steps of 0.1 from 0.7 to 1.0.
The a priori is kept constant at 0.9, and the initial guess is varied from 0.7 to 1.0.
The initial guess and a priori take on the same value and are jointly varied from 0.7 to 1.0.
While this is not an exhaustive list of the possible a priori/initial guess combinations in the 0.7–1.0 range, the maximal impact that combinations in this range can have are represented by the difference between the case where both the a priori and initial guess are 0.7 and that when they are both 1.0.
Note, however, that all these retrievals are run for the same default instrumental noise seed, and so the specific higher/lower value of the retrieved emissivity is not necessarily characteristic. An in-depth analysis would average retrievals run for at least 100 different versions of random instrumental noise and varying the L2M random seed, but this is outside the scope of the slow line-by-line forward model used by the L2M (which prioritizes accuracy). On the other hand, the choice of instrumental random noise should not affect the magnitude of the resulting emissivity ranges or their relation to each other, which is what is examined in this section (to confirm this, the above analysis was in fact repeated for a small number of seeds and showed similar results, with the ranges shifted up or down by a small amount). A full analysis would also consider different a priori uncertainties (see Appendix B) and Ts retrieval parameters.
Figure 8 shows the same full-spectrum upward/downward shifts in emissivity that were seen in Fig. 7. In all of the scenes, the impact of the a priori/initial guess variation is not large overall, and the full range of variation amounts to, at most, a 0.015 relative difference in emissivity. The full range also appears to be additive in the impact of the two parameter choices (i.e. the range of the joint variation is the sum of varying each parameter individually).
However, the relative and total size of the ranges show a different behaviour in scenes 67∘ N, 18∘ E and 67∘ N, 29∘ E than in scenes 25∘ N, 09∘ E and 21∘ N, 18∘ E. While in the first two the variation of initial guess has slightly less of an influence than the a priori, in the third and fourth the sensitivity to the initial guess is stronger. This would not, in general, be expected from an OE retrieval, where usually the initial guess has little influence. However, the effect of the initial guess choice seen in Fig. 8c and d is not due to a false convergence of the retrieval, and the final forward model of all the retrievals for a given scene is almost identical. Thus, they have the same final χ2 (see Eq. 1) and reach convergence in the same way. This is the same process that was seen in Fig. 7a, where the shifts in Ts and emissivity compensate for each other in a way that results in the same forward model within the FORUM noise. We can conclude that the sloppy valley of emissivity and Ts allow for a small range of solutions around the true value, and the choice of initial guess gives the retrieval a small nudge within this range.
The different behaviour in the four scenes is likely due to their geophysical characteristics. While 67∘ N, 18∘ E and 67∘ N, 29∘ E both have low surface temperatures and a low surface-to-air temperature contrast, 25∘ N, 09∘ E and 21∘ N, 18∘ E are hot scenes with high surface temperatures and a high surface-to-air temperature contrast (see Table 1). This means that the latter two have a stronger correlation of surface temperature with emissivity, and so the retrieval vector can take larger steps in the parameter space. This effect of the path on the solution is discussed and analysed in more detail in Appendix A. For the purpose of this section, it is sufficient to note that, although the range is at least twice as large for the hotter scenes, even in the worst-case scenario the choice of initial guess and a priori only change the emissivity by about 0.015, which is still close to the FORUM goal accuracy of 0.01.
While these shifts are not in themselves problematic, the cases where the retrieval uncertainty ranges in Fig. 8 are smaller than the parameter-variation-induced ranges require further investigation. This discrepancy occurs because the emissivity a priori is affecting the retrieved emissivity indirectly through the Ts sloppy valley, and such an indirect effect is not represented in the standard uncertainty analysis which only uses the diagonal elements of Sx and the emissivity submatrix of A. Therefore, to produce a reliable emissivity retrieval product, further work is needed to develop an uncertainty analysis which quantifies this indirect effect.
This study follows from previous work on FORUM geophysical retrievals (e.g. Ridolfi et al., 2020; Sgheri et al., 2022), showing that FORUM measurements will be able to provide retrieved surface emissivity in a significant region of the FIR. Using the FEES, factors that influence OE retrievals of FIR emissivity were investigated with an emphasis on the development of operational retrievals for FORUM. More information could be gained from the retrieval by analysing individual scenes in detail and combining the OE retrieval with different methods, and this should be addressed in future work. Additionally, we have only considered the use of FORUM measurements by themselves (see Ridolfi et al. (2020) for a discussion of how synergetic retrievals with IASI-NG observations can improve the FORUM geophysical products).
In Sect. 4, the retrieved emissivity was introduced together with the quantifiers used to analyse it. In Sect. 5, the variation in the quality of the retrieval with pwv content was compared for multiple geographic scenes. Section 6 then investigated the consequences and characteristics of the surface temperature–emissivity correlation that arises from the functional form of the surface emission equation.
This work has shown the following:
Emissivity retrieval quality, degrees of freedom and extent of retrieval sensitivity towards shorter wavenumbers increase as the pwv of the scene decreases.
For the cases investigated here, varying the value of the emissivity a priori and initial guess between 0.7 and 1.0 results in relative differences in the FIR retrieved emissivity of up to 0.015 in the extreme.
In addition, the following recommendations can already be made for FORUM emissivity retrievals based on this work:
When using the FORUM geophysical emissivity product, the spectral extent of the emissivity used for analysis should be decided on a scene-by-scene basis (and not, for example, by applying a latitude cutoff). We recommend using the information quantifier of the scene as a basis for evaluation.
The functional form of the surface emission equation leads to a strong anti-correlation of surface temperature and emissivity in the retrieval. Thus, the retrieval can converge to a small range of solutions around the true value. Attempting to correct this by constraining the surface temperature retrieval (i.e. introducing more a priori information) could lead to larger shifts away from the true emissivity when the a priori for the surface temperature Ts is wrong. Thus, a surface temperature a priori uncertainty of ± 2 K is recommended, as, even in the worst cases investigated here, it only results in an emissivity offset of an acceptable value around 0.01.
In order to best utilize FORUM measurements to retrieve emissivities, the following two recommendations are made for the development of the FORUM emissivity retrieval:
The quality of the retrieval varies greatly depending on scene parameters such as the water vapour content, absolute surface temperature and its contrast to the atmospheric temperature. These scene dependencies should be investigated in order to identify the best conditions for retrieval of FIR emissivity.
The correlation of emissivity with Ts leads to offsets in both retrieval parameters that are not accurately reflected in the standard quantifiers. It is recommended that the systematic uncertainty originating from the Ts–emissivity correlation is evaluated in detail during the development of the operational retrieval. Further work could also look into the possibility of using external constraints on Ts and other methods for Ts retrieval (such as those used in Murray et al., 2020) to complement the OE.
In addition to these two steps, complementary future work would include laboratory and aircraft measurements of emissivity, analysis of additional methods for surface temperature retrieval and an algorithmic optimization of the emissivity retrieval grid. In addition, after the launch of FORUM, a progressively better emissivity product can be obtained as emissivity climatologies are developed both from FORUM radiances and other FIR measurements.
In conclusion, the FORUM mission will be able to provide a unique contribution to our knowledge of surface emissivity in the FIR for many locations on the globe and, potentially, most types of surfaces. In this work, we have taken the first steps towards the development of an operational emissivity geophysical retrieval for the FORUM mission by highlighting possibilities for optimization of the retrieval and the systematic uncertainties that still need to be quantified.
In Sect. 6 of the paper, the concept of the Ts–emissivity parameter space and its sloppy nature was introduced in the context of the surface emission equation (Eq. 3). The OE retrieval algorithm in the FEES minimizes the cost function (Eq. 1) using the Levenberg–Marquardt approach which interpolates between the Gauss–Newton algorithm and the method of gradient descent. The retrievals in this work converge after four to six iterations, and convergence is reached when the normalized change from one iteration to the next in χ2 (the first term in Eq. 1) is less than 0.01. The path that the retrieval takes to convergence is hard to visualize, as the retrieval vector is stepping in a 300+ dimensional parameter space. However, due to the linear contribution of emissivity to the TOA radiance (see Eq. 3), insight can be gained by plotting the steps in the surface temperature–emissivity slice of this parameter space, and two such plots are shown in this appendix. While the full forward model is far too complex and its computation too time-consuming to lend itself to contour plots or manifold visualizations, some insight can also be gained by showing these steps together with the contours of constant surface emission in Eq. 3.
The issue addressed in this paper which benefits most from such parameter space path plots is that of the sensitivity of the retrieval to the initial guess discussed in Sect. 6.2. Figure A1 shows the convergence of the retrievals shown as the light green range in Fig. 8b, in which the emissivity of scene 67∘ N, 29∘ E is retrieved with an a priori of 0.9 and different initial guesses of 0.7, 0.8, 0.9 and 1.0. For each iteration, Fig. A1a plots the value of two retrieval vector components, with the surface temperature on the x axis and the emissivity at 500cm−1 on the y axis. The four different retrieval runs are represented by different geometric shapes. To put the convergence into context, Eq. (3) is used to plot the contours of the true surface emission value for each Ts–emissivity combination. For simplicity, the surface emission is calculated at the surface (without the atmospheric transmission term). Ld is taken to be the true value at 500 cm−1, calculated using a separate run of LBLRTM (version 12.10) using the true atmospheric state of the scene. Note that, as the atmospheric profiles are also being retrieved, both the transmission and Ld in the forward model will not necessarily equal the true values at the early iterations, and so the background contours do not represent the surface emission used in the forward model at that iteration but instead are there to give context to the later iterations (where the retrieval vector is close to the true). In addition, these contours are not directly representative of the forward model, as f(x) includes many additional effects (for example, those associated with the FORUM instrument). Finally, in Fig. A1b, a small region of the space has been magnified so as to better show the behaviour around convergence.
The behaviour of the different retrievals in Fig. A1 is typical of the emissivity retrieval. The retrieval vector starts from an initial guess, which corresponds to a surface emission very different from the true value, and so takes large steps in the parameter space towards the correct surface emission contour (this is the gradient descent part of the Levenberg–Marquardt minimization). The existence of this true surface emission contour is the cause of the sloppy valley in the Ts–emissivity space discussed in Sect. 6. While, in the MIR, the contour is usually reached in one step, in the FIR the retrieval usually takes two to three steps to reach the true emission value, as a change in the water vapour part of the retrieval vector also changes Ld. Once the true surface emission contour is reached, the retrieval proceeds along it, driven mainly by the Ts a priori constraint and by the small forward model discrepancies caused by the emissivity smoothness assumption (discussed in Sect. 6 and difficult to visualize when plotting only the retrieval vector emissivity components). The main point seen in Fig. A1 is that the direction of the shift of the final values from the true ones depends on whether the correct surface emission contour is first reached at a higher or lower value than the true emissivity (so that, even though a retrieval might start from an emissivity at 0.9 that is lower than the true value, due to the structure of the parameter space, it will reach a final value that is higher than the true value).
In Sect. 6.2, Fig. 8 the comparison for the initial guess sensitivity for different geographical scenes revealed a different behaviour of the colder 67∘ N, 18∘ E and 67∘ N, 29∘ E scenes and the warmer 25∘ N, 09∘ E and 21∘ N, 18∘ E scenes. The warmer scenes showed more sensitivity to the initial guess, in that there was a larger difference between the retrieved emissivity for different initial guesses. Figure A2 shows the convergence path of these four scenes in the FIR and MIR for the case when the initial guess emissivity is set to 0.7 (and the a priori is 0.9, as before). In this figure, no surface emission contours are shown, as their different Ld values mean that the contours would differ for the four different scenes. Figure A2 shows that, as discussed in Sect. 6.2, for the warmer scenes, the retrieval vector takes larger initial steps in the parameter space in both spectral ranges. This is what we would expect from the stronger correlation associated with the warmer scenes, which are analysed in detail in Fig. C1. Once the true surface emission contour is reached, the steps are of similar magnitude for the four scenes. Figure A2 illustrates how such larger steps in the first iteration could potentially explain a higher sensitivity to the initial guess. By taking a larger initial step, the retrieval approaches the true value from a lower emissivity value and so also converges to a slightly lower emissivity value.
In summary, plotting the retrieval's path to convergence in the emissivity-Ts parameter space is a useful visualization tool. By comparing the paths of different cases, it can provide further insight into the reasons underlying the sensitivity of the final retrieved product to different parameters.
Throughout this work, the same emissivity a priori uncertainty matrix Sa was used for all retrievals. Its value was chosen as a baseline case, following the sensitivity tests shown in this Appendix.
In the FEES, Sa is calculated using two parameters, namely the uncertainty and the correlation length. For the profiles, the existence of reliable a priori datasets justifies a nuanced calculation of the uncertainty matrix using uncertainty and correlation lengths that change with height (see Sgheri et al., 2022). As there are no such datasets for FIR emissivity, as a starting point for optimizing the uncertainty, the same parameters are used for the full spectral range. Therefore, a constant uncertainty Δϵ and correlation length CL can be defined, and the Sa matrix elements for emissivity are then as follows:
where Δσij is the wavenumber difference between the location of the retrieved emissivity values ϵi and ϵj. In practice, Δϵ defines the freedom of the retrieval discussed in Sect. 3, as a larger value will allow the profile to take larger steps at each iteration and reduces the penalization from x−xa. The CL controls the off-diagonal elements in Sa; its presence means that the regularization term for the retrieved emissivity points is not minimized individually but that the emissivity step at a given wavenumber is also affected by the difference of its neighbouring points from the a priori. In practice, this results in a smoother solution where the retrieval is sensitive to the a priori.
These two parameters were varied in the retrieval set-up, and the results are shown in Figs. B1 and B2. While the sensitivity tests were run for many different scenes, the analysis shown here is of the scene at 67∘ N, 18∘ E, which was chosen as it is representative of the snow emissivity scenes that are the primary goal of FORUM's emissivity retrievals. The range of the uncertainty parameters shown is Δϵ=0.05, 0.1 and 0.12 and CL = 10, 50 and 100 cm−1. Smaller values of Δϵ were also considered but are not shown as they did not give the retrieval the necessary freedom to converge to the right solution and caused a large systematic uncertainty. Figure B1 shows the systematic uncertainty and the retrieval uncertainty in the FIR for all nine cases and the root mean square (RMS) error for the systematic uncertainty values shown. Figure B2 shows the averaging kernels of the nine cases for the full spectral range. The following points can be seen in these figures:
The differences in uncertainties for a given correlation length are only present in the edge regions of the retrieval, where the sensitivity is lower. This is because the a priori uncertainty only matters where information is drawn from the a priori, and for a dry scene such as this (as discussed for Fig. 3) in the centres of the FIR dirty window and of the MIR atmospheric window, the retrieval is fully sensitive to the true state, and thus the choice of a priori uncertainty has no influence.
Examining the averaging kernels shown in Fig. B2, we can again see that the influence of the parameters is strongest at the edge regions of sensitivity. Here it can be seen that increasing CL leads to more information being drawn from regions to which the TOA is not, in reality, sensitive. Analysing the rows in the figure shows that decreasing Δϵ decreases the diagonal averaging kernel values and increases its off-diagonal values.
These averaging kernels show that the lower RMS error of the high CL and low Δϵ cases comes at the price of sensitivity to the true emissivity. In an ideal case, the averaging kernel is a straight diagonal line. The more spread out the edges of this line are, the more a priori information was used.
The main conclusion of this analysis is that there is no abrupt transition in the explored a priori uncertainty parameter space. All the parameter choices produced similar retrieval results, with differences only in less-sensitive regions. Therefore, a choice in either direction will either give slightly more sensitivity or accuracy and can be tuned to match the specific need of the user.
An additional option is to use a posteriori regularization. Using a larger error and a smaller correlation length would be desirable to give the retrieval more precision and freedom. As seen in this Appendix, due to the ill conditioning of the retrieval, the weaker regularization would cause the solution to oscillate more. An a posteriori regularization method, such as the IVS (iterative variable strength) method introduced in Ridolfi and Sgheri (2011) and applied to the FORUM atmospheric profile retrievals in Sgheri et al. (2020), could be used to smooth out these unphysical oscillations.
For the purpose of this study, Δϵ=0.1 and CL = 50 were used as the baseline parameter combination that represents a compromise between the two extremes of sensitivity and accuracy.
To complement Sect. 6, the final step in understanding the variations allowed by the Ts–emissivity sloppy valley is to analyse the correlation strength in different spectral regions. Equation (3) shows that there are two main factors that could cause differences in the correlation in the FIR and MIR. The first originates from B(Ts) having a different shape in different spectral regions. The second is that, even if the downwelling radiation Ld is known, its value still differs significantly between the FIR and MIR. This is for the same reasons as discussed in Sect. 4. In the MIR, the atmospheric window is transparent, and so Ld is negligible, while in the FIR, Ld is higher or lower, depending on the amount of water vapour and on the microwindow structure (see, e.g., Palchetti et al., 2016, 2020 for ground measurements of FIR downwelling radiation).
To investigate these effects, Fig. C1 shows an analysis of both the empirical correlation of the 28 retrieved values for four scenes each and an analytic correlation calculated from the standard OE equations. The same geographic scenes are used as in Fig. 8. For the empirical correlation, the baseline retrieval is run for each scene using instrumental spectra with seven different versions of random instrumental noise (generated with FSI seeds of 0, 1, 2, 3, 4, 5 and 6) and then each is retrieved with equal flat a priori cases and initial guesses set to 0.7, 0.8, 0.9 and 1.0, resulting in 28 cases for each scene. The variation in the instrumental noise and the a priori and initial guess results in a range of different systematic uncertainties (as discussed in Sect. 6.2). These uncertainties are shown in Fig. C1a and b, which plot the average systematic uncertainty of emissivity in a specific spectral range against the systematic uncertainty in Ts. Constant and relatively small spectral ranges are chosen so that the variation in the correlation slope and strength in the averaged range is small enough to allow a meaningful analysis. The spectral ranges of 500–600 and 800–1000 cm−1 are chosen to represent the FIR and MIR, respectively, as these are the spectral intervals with the highest sensitivity in those regions. These are not representative of the variation in the full FIR/MIR but only indicative of the difference between the regions.
As expected, there is a strong anti-correlation between the systematic uncertainties both in the FIR and the MIR. Table C1 lists the slopes of the linear trends fitted to the data in these figures (grouped by scene and spectral region) and the corresponding sample Pearson correlation coefficient R using the standard formula, as follows:
where ΔTs and Δϵ are the data vectors of systematic uncertainties in surface temperature and emissivity, and and mΔϵ are the means of these vectors.
The following three points can be highlighted from these results:
With the exception of 67∘ N, 29∘ E, the slope of the linear fit is steeper in the MIR than in the FIR.
In both spectral regions, the slope of 67∘ N, 18∘ E and 67∘ N, 29∘ E is steeper than that of the other two scenes.
For scenes 67∘ N, 18∘ E; 67∘ N, 29∘ E; and 25∘ N, 09∘ E, the scatter of values is larger in the FIR than the MIR (lower R in Table C1).
A possible cause for the variation in slopes can be found in the form of B(Ts). To see this let Ld=0, which primarily simplifies the analysis but is also a valid assumption for the MIR and for the centres of the FIR microwindows. Equation (3) then becomes the following:
where the σ underscore has been dropped for convenience. Keeping Ssurf constant, the equation is rearranged to get an expression for ϵ and then the derivative is taken with respect to Ts as follows:
The dominating factor in determining (the slope in Fig. C1a and b) for a given scene and wavenumber is the expression in brackets on the right-hand side of Eq. (C3), as although ϵ also varies spectrally and geographically, its average variations are an order of magnitude smaller (20 % as opposed to 800 %). The plot in Fig. C1c shows this expression for the surface temperatures of the four different scenes. This plot shows that the value of this expression increases with wavenumber and is lower for higher Ts. This behaviour could explain the difference in slopes observed in Fig. C1a and b. The larger value of this term and, thus, in the MIR would result in the steeper slope observed for most scenes in the MIR. And as 67∘ N, 18∘ E and 67∘ N, 29∘ E are much colder scenes, it is expected that the slope of their linear relation will be steeper than that of 21∘ N, 18∘ E and 25∘ N, 09∘ E.
Finally, an analytic correlation analysis can shed light on the scatter about the linear trend of the values in Fig. C1a and b (also represented by the size of their R correlation value in Table C1). In Fig. C1d, the analytic Pearson correlation coefficient of the retrieval uncertainties of Ts and emissivity is shown. The uncertainties are given in the retrieval uncertainty covariance matrix Sx defined in Eq. (4). Using the standard formula for the analytic (population) Pearson correlation coefficient, as follows:
where is the retrieval uncertainty standard deviation of Ts (dropping the x in S for visibility), and similarly for ϵi, the ith value in the emissivity retrieval vector. Note that Fig. C1d only shows this value for the baseline retrieval of the four scenes and is thus meant as an illustration of the spectral structure of the correlation and not as a quantitative reference.
Figure C1d shows that, as expected Ts, and emissivity are not correlated to the same extent in different spectral regions. The correlation mirrors the spectral structure seen in the emissivity Jacobian (see Fig. 3). Unsurprisingly, as it is calculated from Sx, which in turn is calculated from the Jacobian (and from Sa). There is a strong uniform correlation of the MIR emissivity points with Ts, while the correlation of the FIR values depends on the microwindow structure and, with that, on the dryness of the atmosphere. The ∼ 750–1250 cm−1 region of the MIR is called the atmospheric window as it is almost fully transparent to the surface, and thus, in most of that region, the strength of the correlation is determined solely by the value of the surface temperature. In the FIR, the difference in correlation strength is harder to attribute precisely, as it is due to a combination of the pwv and the surface temperature. However, its value for the four scenes analysed here can still be used to compare the correlation to the scatter seen in Fig. C1a and b. These show good agreement, as the scenes with a lower retrieval uncertainty correlation coefficient also have a smaller systematic uncertainty correlation and larger scatter around the linear trend.
In summary, the correlation of surface temperature and emissivity behaves as would be expected from the physics of the forward model. The range of systematic uncertainties in Fig. C1a and b confirm what was already shown in Sect. 6.2 and 6.1, namely that this correlation allows for a range of retrieved emissivities depending on the retrieval parameters. The predictability of the behaviour of the correlation is important for the evaluation of this effect, which should be thoroughly quantified during the development of the operational retrieval.
In the FORUM E2ES contract, the open distribution of the code is not mandatory. Each author retains the intellectual property rights, or their portions, and the industrial partners of the consortium do not allow the distribution of their modules. The code for producing the analysis and figures from the FEES outputs is available at https://github.com/mayaby/FORUM_emissivity (last access: 1 February 2022; https://doi.org/10.5281/zenodo.6205876, Ben-Yami, 2022b). The FEES outputs used for the analysis in the paper are available at https://doi.org/10.5281/zenodo.5960223 (Ben-Yami, 2022a).
All authors contributed to the paper through discussions and comments. MBY wrote the paper. MBY and HO devised the experiments. LS, PR, TM, WC and DM wrote the FEES code and helped with running the simulator. DL was the ESA FEES code quality manager. LW and HB provided the downwelling radiation values for Fig. A1.
The contact author has declared that neither they nor their co-authors have any competing interests.
Publisher’s note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The work presented in this paper benefited significantly from discussions with the ESA FORUM Mission Advisory Group. We thank the anonymous reviewers, for their helpful comments.
This research has been supported by ESA (contract no. 4000123810), the ESA Young Graduate Trainee programme and a CASE partnership between the EPSRC DTP and the National Physical Laboratory (grant no. EP/R513052/1).
This paper was edited by Piet Stammes and reviewed by two anonymous referees.
Bellisario, C., Brindley, H. E., Murray, J. E., Last, A., Pickering, J., Harlow, R. C., Fox, S., Fox, C., Newman, S. M., Smith, M., Anderson, D., Huang, X., and Chen, X.: Retrievals of the Far Infrared Surface Emissivity Over the Greenland Plateau Using the Tropospheric Airborne Fourier Transform Spectrometer (TAFTS), J. Geophys. Res.-Atmos., 122, 12152–12166, https://doi.org/10.1002/2017JD027328, 2017. a, b, c
Borbas, E. E., Hulley, G., Feltz, M., Knuteson, R., and Hook, S.: The Combined ASTER MODIS Emissivity over Land (CAMEL) Part 1: Methodology and High Spectral Resolution Application, Remote Sens., 10, 643, https://doi.org/10.3390/rs10040643, 2018. a
Capelle, V., Chédin, A., Péquignot, E., Schlüssel, P., Newman, S. M., and Scott, N. A.: Infrared Continental Surface Emissivity Spectra and Skin Temperature Retrieved from IASI Observations over the Tropics, J. Appl. Meteorol. Clim., 51, 1164–1179, https://doi.org/10.1175/JAMC-D-11-0145.1, 2012. a
Clough, S., Shephard, M., Mlawer, E., Delamere, J., Iacono, M., Cady-Pereira, K., Boukabara, S., and Brown, P.: Atmospheric radiative transfer modeling: a summary of the AER codes, J. Quant. Spectrosc. Ra., 91, 233–244, https://doi.org/10.1016/j.jqsrt.2004.05.058, 2005. a
Dinelli, B. M., Castelli, E., Carli, B., Del Bianco, S., Gai, M., Santurri, L., Moyna, B. P., Oldfield, M., Siddans, R., Gerber, D., Reburn, W. J., Kerridge, B. J., and Keim, C.: Technical Note: Measurement of the tropical UTLS composition in presence of clouds using millimetre-wave heterodyne spectroscopy, Atmos. Chem. Phys., 9, 1191–1207, https://doi.org/10.5194/acp-9-1191-2009, 2009. a, b
Feldman, D. R., Collins, W. D., Pincus, R., Huang, X., and Chen, X.: Far-infrared surface emissivity and climate, P. Natl. Acad. Sci. USA, 111, 16297–16302, https://doi.org/10.1073/pnas.1413640111, 2014. a, b
Feltz, M., Borbas, E., Knuteson, R., Hulley, G., and Hook, S.: The Combined ASTER MODIS Emissivity over Land (CAMEL) Part 2: Uncertainty and Validation, Remote Sens., 10, 664, https://doi.org/10.3390/rs10050664, 2018. a
Harries, J., Carli, B., Rizzi, R., Serio, C., Mlynczak, M., Palchetti, L., Maestri, T., Brindley, H., and Masiello, G.: The Far-infrared Earth, Rev. Geophys., 46, RG4004, https://doi.org/10.1029/2007RG000233, 2008. a
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz-Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., Simmons, A., Soci, C., Abdalla, S., Abellan, X., Balsamo, G., Bechtold, P., Biavati, G., Bidlot, J., Bonavita, M., Chiara, G., Dahlgren, P., Dee, D., Diamantakis, M., Dragani, R., Flemming, J., Forbes, R., Fuentes, M., Geer, A., Haimberger, L., Healy, S., Hogan, R. J., Hólm, E., Janisková, M., Keeley, S., Laloyaux, P., Lopez, P., Lupu, C., Radnoti, G., Rosnay, P., Rozum, I., Vamborg, F., Villaume, S., and Thépaut, J.-N.: The ERA5 global reanalysis, Q. J. Roy. Meteor. Soc., 146, 1999–2049, https://doi.org/10.1002/qj.3803, 2020. a
Huang, X., Chen, X., Zhou, D. K., and Liu, X.: An Observationally Based Global Band-by-Band Surface Emissivity Dataset for Climate and Weather Simulations, J. Atmos. Sci., 73, 3541–3555, https://doi.org/10.1175/JAS-D-15-0355.1, 2016. a, b, c, d, e
Huang, X., Chen, X., Flanner, M., Yang, P., Feldman, D., and Kuo, C.: Improved Representation of Surface Spectral Emissivity in a Global Climate Model and Its Impact on Simulated Climate, J. Climate, 31, 3711–3727, https://doi.org/10.1175/JCLI-D-17-0125.1, 2018. a, b, c
Knuteson, R., Best, F., DeSlover, D., Osborne, B., Revercomb, H., and Smith, W.: Infrared land surface remote sensing using high spectral resolution aircraft observations, Adv. Space Res., 33, 1114–1119, https://doi.org/10.1016/S0273-1177(03)00752-X, 2004. a
Kuo, C., Feldman, D. R., Huang, X., Flanner, M., Yang, P., and Chen, X.: Time-Dependent Cryospheric Longwave Surface Emissivity Feedback in the Community Earth System Model, J. Geophys. Res.-Atmos., 123, 789–813, https://doi.org/10.1002/2017JD027595, 2018. a, b
L’Ecuyer, T. S., Drouin, B. J., Anheuser, J., Grames, M., Henderson, D. S., Huang, X., Kahn, B. H., Kay, J. E., Lim, B. H., Mateling, M., Merrelli, A., Miller, N. B., Padmanabhan, S., Peterson, C., Schlegel, N.-J., White, M. L., and Xie, Y.: The Polar Radiant Energy in the Far Infrared Experiment: A New Perspective on Polar Longwave Energy Exchanges, B. Am. Meteorol. Soc., 102, E1431–E1449, https://doi.org/10.1175/BAMS-D-20-0155.1, 2021. a
Li, Z.-L., Wu, H., Wang, N., Qiu, S., Sobrino, J. A., Wan, Z., Tang, B.-H., and Yan, G.: Land surface emissivity retrieval from satellite data, Int. J. Remote Sens., 34, 3084–3127, https://doi.org/10.1080/01431161.2012.716540, 2013. a, b
Loveless, M., Borbas, E., Knuteson, R., Cawse-Nicholson, K., Hulley, G., and Hook, S.: Climatology of the Combined ASTER MODIS Emissivity over Land (CAMEL) Version 2, Remote Sens., 13, 111, https://doi.org/10.3390/rs13010111, 2020. a
Masiello, G. and Serio, C.: Simultaneous physical retrieval of surface emissivity spectrum and atmospheric parameters from infrared atmospheric sounder interferometer spectral radiances, Appl. Opt., 52, 2428–2446, https://doi.org/10.1364/AO.52.002428, 2013. a
Murray, J. E., Brindley, H. E., Fox, S., Bellisario, C., Pickering, J. C., Fox, C., Harlow, C., Smith, M., Anderson, D., Huang, X., Chen, X., Last, A., and Bantges, R.: Retrievals of High-Latitude Surface Emissivity Across the Infrared From High-Altitude Aircraft Flights, J. Geophys. Res.-Atmos., 125, e33672, https://doi.org/10.1029/2020JD033672, 2020. a, b, c, d
Palchetti, L., Di Natale, G., and Bianchini, G.: Remote sensing of cirrus cloud microphysical properties using spectral measurements over the full range of their thermal emission, J. Geophys. Res.-Atmos., 121, 10804–10819, https://doi.org/10.1002/2016JD025162, 2016. a
Palchetti, L., Brindley, H., Bantges, R., Buehler, S. A., Camy-Peyret, C., Carli, B., Cortesi, U., Bianco, S. D., Natale, G. D., Dinelli, B. M., Feldman, D., Huang, X. L., C.-Labonnote, L., Libois, Q., Maestri, T., Mlynczak, M. G., Murray, J. E., Oetjen, H., Ridolfi, M., Riese, M., Russell, J., Saunders, R., and Serio, C.: FORUM: Unique Far-Infrared Satellite Observations to Better Understand How Earth Radiates Energy to Space, B. Am. Meteorol. Soc., 101, E2030–E2046, https://doi.org/10.1175/BAMS-D-19-0322.1, 2020. a, b
Palchetti, L., Barucci, M., Belotti, C., Bianchini, G., Cluzet, B., D'Amato, F., Del Bianco, S., Di Natale, G., Gai, M., Khordakova, D., Montori, A., Oetjen, H., Rettinger, M., Rolf, C., Schuettemeyer, D., Sussmann, R., Viciani, S., Vogelmann, H., and Wienhold, F. G.: Observations of the downwelling far-infrared atmospheric emission at the Zugspitze observatory, Earth Syst. Sci. Data, 13, 4303–4312, https://doi.org/10.5194/essd-13-4303-2021, 2021. a
Ridolfi, M. and Sgheri, L.: Iterative approach to self-adapting and altitude-dependent regularization for atmospheric profile retrievals, Opt. Express, 19, 26696–26709, https://doi.org/10.1364/OE.19.026696, 2011. a
Ridolfi, M., Del Bianco, S., Di Roma, A., Castelli, E., Belotti, C., Dandini, P., Di Natale, G., Dinelli, B. M., C.-Labonnote, L., and Palchetti, L.: FORUM Earth Explorer 9: Characteristics of Level 2 Products and Synergies with IASI-NG, Remote Sens., 12, 1496, https://doi.org/10.3390/rs12091496, 2020. a, b, c
Rodgers, C. D.: Retrieval of Atmospheric Temperature and Composition From Remote Measurements of Thermal Radiation, Rev. Geophys. Space GE, 14, 609–624, https://doi.org/10.1029/RG014i004p00609, 1976. a
Sgheri, L., Raspollini, P., and Ridolfi, M.: Auto-adaptive Tikhonov regularization of water vapor profiles: application to FORUM measurements, Appl. Anal., 99, 1–11, https://doi.org/10.1080/00036811.2020.1751825, 2020. a, b
Sgheri, L., Belotti, C., Ben-Yami, M., Bianchini, G., Carnicero Dominguez, B., Cortesi, U., Cossich, W., Del Bianco, S., Di Natale, G., Guardabrazo, T., Lajas, D., Maestri, T., Magurno, D., Oetjen, H., Raspollini, P., and Sgattoni, C.: The FORUM end-to-end simulator project: architecture and results, Atmos. Meas. Tech., 15, 573–604, https://doi.org/10.5194/amt-15-573-2022, 2022. a, b, c, d, e, f, g, h, i, j, k, l
Susskind, J., Blaisdell, J. M., and Iredell, L.: Improved methodology for surface and atmospheric soundings, error estimates, and quality control procedures: the atmospheric infrared sounder science team version-6 retrieval algorithm, J. Appl. Remote Sens., 8, 1–34, https://doi.org/10.1117/1.JRS.8.084994, 2014. a
Transtrum, M. K., Machta, B. B., and Sethna, J. P.: Geometry of nonlinear least squares with applications to sloppy models and optimization, Phys. Rev. E, 83, 036701, https://doi.org/10.1103/PhysRevE.83.036701, 2011. a
von Clarmann, T., Degenstein, D. A., Livesey, N. J., Bender, S., Braverman, A., Butz, A., Compernolle, S., Damadeo, R., Dueck, S., Eriksson, P., Funke, B., Johnson, M. C., Kasai, Y., Keppens, A., Kleinert, A., Kramarova, N. A., Laeng, A., Langerock, B., Payne, V. H., Rozanov, A., Sato, T. O., Schneider, M., Sheese, P., Sofieva, V., Stiller, G. P., von Savigny, C., and Zawada, D.: Overview: Estimating and reporting uncertainties in remotely sensed atmospheric composition and temperature, Atmos. Meas. Tech., 13, 4393–4436, https://doi.org/10.5194/amt-13-4393-2020, 2020. a, b
Wang, K., Wan, Z., Wang, P., Sparrow, M., Liu, J., Zhou, X., and Haginoya, S.: Estimation of surface long wave radiation and broadband emissivity using Moderate Resolution Imaging Spectroradiometer (MODIS) land surface temperature/emissivity products, J. Geophys. Res.-Atmos., 110, D11109, https://doi.org/10.1029/2004JD005566, 2005. a