One-dimensional variational retrievals of
temperature and moisture fields from hyperspectral infrared (IR) satellite
sounders use cloud-cleared radiances (CCRs) as their observation. These derived
observations allow the use of clear-sky-only radiative transfer in the
inversion for geophysical variables but at reduced spatial resolution
compared to the native sounder observations. Cloud clearing can introduce
various errors, although scenes with large errors can be identified and
ignored. Information content studies show that, when using multilayer cloud
liquid and ice profiles in infrared hyperspectral radiative transfer codes,
there are typically only 2–4 degrees of freedom (DOFs) of cloud signal. This implies
a simplified cloud representation is sufficient for some applications which
need accurate radiative transfer. Here we describe a single-footprint
retrieval approach for clear and cloudy conditions, which uses the
thermodynamic and cloud fields from numerical weather prediction (NWP) models
as a first guess, together with a simple cloud-representation model coupled
to a fast scattering radiative transfer algorithm (RTA). The NWP model
thermodynamic and cloud profiles are first co-located to the observations,
after which the
Since the early 2000s, a number of high-spectral-resolution, low-noise, very stable new generation hyperspectral infrared (IR) sounders have been deployed onboard Earth-orbiting satellites, providing daily global top-of-atmosphere (TOA) radiance spectra. In principle these TOA radiances can be inverted to estimate atmospheric temperature and humidity profiles, minor gas concentration, surface temperature and some clouds parameters.
IR sounders have rather large nadir footprints of
Presently the NASA AIRS operational soundings are performed using
cloud-cleared radiances (CCRs) coupled with a clear-sky radiative transfer algorithm
Atmospheric Infrared Sounder (AIRS): NASA, using cloud clearing from
a Cross-track Infrared Sounder (CrIS): the NOAA Unique Combined
Atmospheric Processing System (NUCAPS), also using
cloud clearing from a Infrared Atmospheric Sounding Interferometer (IASI): NUCAPS,
using cloud clearing from a Infrared Atmospheric Sounding Interferometer: EUMETSAT,
two-step single-footprint retrievals: piecewise regression for all
scenes nominally exploiting IASI in synergy with AMSU+MHS
(IASI-only is fallback) followed by a physical retrieval using the
optimal estimation method (OEM)
The retrieval approaches mentioned above use various combinations of
training to NWP forecasts from the European Centre for Medium-Range
Weather Forecasts (ECMWF) either by regression (EUMETSAT IASI) or
with neural nets (NASA AIRS) or use climatology (NOAA CrIS). All
utilize co-located microwave soundings when possible. The development
of a formal error estimate computation in the NUCAPS algorithm is
underway (Antonia Gambacorta, personal communication, 2017). The CCR approaches lead
to complicated quality control issues, since cloud clearing can fail,
and the decisions made in assigning quality flags to the retrievals
are not trivial. The cloud-clearing process is especially problematic
Single-footprint retrievals with hyperspectral sounders provide
higher spatial resolution than the
Here we examine some viable first steps in performing operational single-footprint retrievals using the OEM for these sensors using a fast scattering RTA that uses a first guess (and a priori estimates) from the ECMWF forecast model. The OEM methodology provides the user with objective diagnostic information, such as error estimates of the retrieved profiles, averaging kernels (AKs) and the information content of the measurements via the degrees of freedom (DOFs). For example we show later in this paper that our single-footprint retrievals have much lower DOFs under thick clouds than in almost clear scenes, which means our retrieval mostly returns the a priori thermodynamic profiles below thick clouds and mainly adjusts the profile above such clouds.
Radiative transfer algorithms for infrared sounders that include
scattering by clouds and aerosols are now available
This paper concentrates on the accuracy of our relatively simple but fast accurate scattering model, especially when coupled with the representation of cloud features in the profile and initialization of these features in a retrieval. Very few cloud parameters can be retrieved from IR sounder spectra compared to clear-sky geophysical parameters (temperature and humidity), suggesting that simple fast scattering models and cloud representations should be sufficient to radiatively account for cloud and aerosol effects in a retrieval. The paper also demonstrates the utility of using NWP first guess model fields both for thermodynamic and cloud initialization in a high-yield single-footprint physical retrieval, where the computed degrees of freedom are shown to depend on the observed window channel brightness temperature (which itself depends on cloud loading).
In this paper observational data from AIRS are used, while the
principal scattering algorithm is the Stand-alone
AIRS Radiative Transfer Algorithm
The PCRTM–MRO implementation
The SARTA–TwoSlab approach is then applied to single-footprint
retrievals for an AIRS granule and compared to the existing NASA AIRS
level 2 retrievals. As noted above, a key issue is the proper
initialization of the cloud parameters in our RTA. Model fields from
ECMWF are used here to initialize the thermodynamic and scattering
cloud fields. Although NWP models do a reasonably good job at
estimating cloud parameters, it is very unlikely that the positions of
the model clouds are correct on scales near the sounder spatial
resolution, especially given the time mismatch between available
forecast models and the observations (
There are recent papers detailing hyperspectral optimal-estimation-based retrievals in the presence of clouds
The paper is organized as follows. The AIRS instrument and the use of the ECMWF model are summarized first, followed by a detailed description of the RTA models and the cloud-representation schemes. We then examine the computed radiance differences for both clear-sky and all-sky for these two RTAs and discuss radiance differences arising from perturbations to the TwoSlab cloud-representation schemes. Finally we outline a method to reduce the impact of the spatial–temporal mismatch of observed versus modeled clouds and use this together with the TwoSlab cloud representation to perform single-footprint (cloudy) scene retrievals with a priori thermodynamic profiles and cloud fields from the NWP model fields.
The Atmospheric Infrared Sounder (AIRS) onboard NASA's polar-orbiting
Earth Observing System (EOS) Aqua platform has 2378 channels, covering the thermal infrared
(TIR; roughly 649–1613 cm
About 1500 AIRS channels that have remained stable over the life of
the AIRS mission were selected for this paper.
This was done by examining the statistics of the
The core ECMWF 0–10 day forecasts are produced using the Integrated
Forecasting System (IFS)
Here we use 91 level ECMWF model fields, at a horizontal resolution of
0.25
The topmost AIRS RTA pressure layer boundary is 0.005 mb, so US
standard temperature, water vapor and ozone fields
The description of existing cloud-representation and scattering codes
for nadir infrared sounders include those found in
We use two different RTAs, described below, to simulate AIRS infrared
radiances that differ primarily in the scattering radiative transfer.
Both RTAs use the same AIRS 100 pressure layer scheme
The clear-sky version (with gray cloud capability) of SARTA is used
for the NASA AIRS level 2 retrievals. Layer optical depths are
generated using precomputed predictor coefficients
We extended SARTA to handle clouds and aerosols, based on
the parametrization for cloud longwave scattering for use in atmospheric models
We benchmarked the SARTA–TwoSlab model versus the radiance simulator
based on the principal-component-based radiative transfer
model (PCRTM version 2.1)
Here we describe the TwoSlab cloud-representation and the MRO cloud
models. The latter is used exclusively with PCRTM and the former
with SARTA except in Sect.
Our cloud-representation scheme replaces the
Infrared sensors cannot see through optically thick clouds and are mostly sensitive to the emission from the cloud upper boundary while emission throughout the cloud can contribute to the outgoing radiance for less optically thick clouds. The TwoSlab model is very flexible when placing the slabs, for example (a) at the weighted mean or centroid (C) of the cloud ice or cloud liquid profile or (b) near the most prominent cloud profile peak (P), which is best for optically thin clouds.
In practice, the cloud content profile CXWC
Figure
Example of cloud vertical profiles, reduced to one or two slabs. The red and blue curves come from the NWP model, while the cyan and magenta bars are the resulting locations (and loadings) for the slabs.
Assuming one ice and one water cloud slab are produced, the cloud
fractions are constrained as follows:
If there is only one cloud present, its cloud fraction is set to
TCC. For two ice or two water clouds, the fraction for the first
cloud If there is one ice and one water cloud, the cloud fractions are set
according to
For the third case, the water cloud slab fraction comes from weighting
the NWP cloud cover (CC) profile using the cloud liquid water
content profile
Water cloud droplet effective diameters vary with season and
geographic location
The
The MRO cloud processing for the PCRTM model is described in
An earlier inter-comparison of the SARTA and PCRTM clear-sky models is
presented in
We use 1600 randomly chosen nighttime scenes observed by AIRS on
1 March 2009 for an inter-RTA clear-sky simulation comparison. The
locations span all climate zones over ocean and land, as well as all
AIRS scan angles. Nighttime scenes are used to avoid non-local
thermodynamic equilibrium
Figure
Spectral differences for clear-sky
calculations between SARTA and PCRTM for ocean night scenes. Also shown are
the AIRS noise levels. Typical difference between PCRTM and SARTA are less
than 0.25 K, except in the methane region (1300 cm
Here we compare all-sky radiances computed using SARTA and PCRTM but use the same TwoSlab cloud representation in both RTAs. This tests the differences in each RTA's underlying scattering algorithm by keeping the cloud representation the same in both. Thus this directly compares the relative accuracy of the PCLSAM scattering algorithm used in SARTA against the DISORT-based scattering used in PCRTM.
The PCLSAM algorithm approximations in SARTA are more accurate for absorptive clouds that are more likely in the mid-IR. However, the DISORT-based scattering in PCRTM is more accurate if the cloud representation is correct. In general it would be reasonable to expect the differences to increase with optical depth and/or cloud fraction. In addition, in the TIR the single scattering albedo of water clouds is generally larger than that of ice, so we would also expect larger differences for water clouds.
To evaluate the SARTA (PCLSAM) versus PCRTM (DISORT) radiance differences, we used 1000 scenes maximizing cloud variability and spanning all climate types from AIRS on 1 March 2009 (see Appendix C). After matching the thermodynamic and cloud NWP fields to the observations, and subsequent conversion of the input cloud profiles to slab clouds, SARTA and PCRTM were run twice: (a) a clear-sky run where no cloud effects are included and (b) an all-sky run using the TwoSlab cloud representation derived from ECMWF.
In the TIR window region, cloud forcing (difference between observed BT
and surface temperature) can be as large as 100 K (for the deep
convective cloud, DCC, cases). For our sample set the mean (AIRS
observation–SARTA–TwoSlab RTA simulation) difference at 820 cm
The effects of the clouds become less noticeable for channels sensing
high in the atmosphere, such as the 650–700 cm
Biases and standard deviations between 1000 TwoSlab computations using SARTA and PCRTM. See the text for details on these double-difference signals.
Figure
These comparisons indicate that our implementation of the PCLSAM model
is a fast yet simple and effective method to include scattering
effects in the TIR, as has also been shown by
We now compare radiances produced using the TwoSlab
cloud-representation model using SARTA and those produced using the MRO
cloud representation using PCRTM, comparing both to AIRS all-sky
observations. The AIRS data obtained on 11 March 2011 are used in this
section, with co-located thermodynamic and cloud fields from the ECMWF
model. The SARTA–TwoSlab calculations used slab clouds at the weighted
mean (centroid) of the cloud profiles. An
important factor in the comparisons to actual AIRS observations is the
The left panel of Fig.
Figure
Plots of the 1231 cm
Here we explore the similarities and differences between the
observations, SARTA–TwoSlab and PCRTM–MRO by examining the radiance
probability distribution functions (PDFs) and the scene dependence of
the mean BT differences, again for the 1231 cm
Cloud mismatch errors will contribute to a significant portion of the
standard deviation between observations and calculations, as is
evident from Figs. In the tropics the observations have more cold (DCC) scenes, also
seen in The frozen oceans in the northern polar regions and off the
Antarctic coast have BT1231 calculations between 240 and 260 K, which
are quite different than what is observed. A significant portion of the BT1231 calculations between 260 and 280 K come
from the extra-tropical oceans ( The whisker plots (circles are means while the bars are
the standard deviation) show the calculations are generally
slightly warmer than the observations, while the spreads are all
very similar.
The right panel of Fig.
Note that for the highest clouds SARTA (C) is hotter than SARTA (P), which makes sense since these are very optically thick clouds and the TIR weighting function peaks at the cloud top. For warm scenes, the MRO simulations are again closer to SARTA (P) than SARTA (C), indicating that, as expected, placing clouds near the weighting function peak in the TwoSlab algorithm is preferable to the centroid.
Table
Night land and ocean 1231 cm
We note the PDF correlations between observations and all the calculations and also among the calculations themselves are typically 0.9 and above, which reinforces the point that the NWP fields from ECMWF capture much of the atmospheric variability that is observed; however, the mismatch between observed and model cloud tops led to biases on the order of 2–4 K with standard deviations on the order of 10 K.
The implications of this are that, by shifting the position of the model clouds, one could significantly mitigate the differences, and hence have a priori micro-physical properties of the TwoSlab clouds, for a physical single-footprint all-sky retrieval. This idea is exploited later in the section on applying the TwoSlab code for use in single pixel all-sky retrievals.
We close this subsection with Fig.
Brightness temperature PDFs for the night
non-frozen ocean 1231 cm
Finally we compare the observed and calculated spectra for ocean
scenes where the ECMWF model sea surface temperature (SST) is very accurate
and the emissivity is well known. As in Fig.
Mean biases
These comparisons are summarized in
Fig.
The mean tropical SARTA–TwoSlab and PCRTM–MRO calculations are slightly warmer than the observations, partially due to fewer and warmer deep convective clouds in the ECMWF model. The midlatitude window region calculations are on average about 5 K warmer than the observations. The calculations for the polar regions are noticeably warmer than the polar observations, with the SARTA–TwoSlab and PCRTM–MRO clouds simulations much more similar to each other than to the observations.
As expected from studying the 1231 cm
The above plots are similar to all-sky monthly global averaged biases
seen in
One full AIRS granule's (6 min, 12 150 spectra) worth of all-sky retrievals using the SARTA–TwoSlab code is provided here using the OEM, which provides natural diagnostics such as DOF and AK that are extremely helpful in understanding the information content of our retrieval approach in the presence of highly variable clouds. This is a proof-of-concept retrieval which has been separately tested on several days' worth of AIRS observations. While considerable efforts have been put into selection of various regularization matrices and channels, we anticipate additional fine-tuning in the future.
We follow normal OEM notation here, where the observation vector
The state vector consists of surface and profile temperatures
(Kelvin), the logarithm of the water vapor profile (molecules cm
The regularization matrix
For this paper we start with a smoothed climatological profile and use
2 K uncertainties for the temperature profiles, 0.3 K for the surface
temperature, 60 % uncertainty for water vapor profiles and 10 % cloud
loading uncertainty. We start with ECMWF surface temperatures since
they are likely to be better than climatology. Land and ocean surface
emissivity is set from a database (see
The highly nonlinear effect of clouds on infrared radiative transfer makes
retrieval success highly dependent on an accurate linearization point for
cloud parameters. This fact has made it difficult to create
physically based (not statistical) robust single-footprint infrared
hyperspectral retrievals. Since typical infrared all-sky retrievals only
have 2–4 degrees of freedom for cloud fields (see Appendix B), it
is essential that the linearization point and a priori covariances for clouds be as
accurate as possible. Fortunately, NWP model forecasts such as from ECMWF
provide reasonably accurate cloud fields derived using the best physics
possible for an operational model. However, as we have shown earlier,
perfect spatial placement of model clouds is not possible, and winds can
move forecast clouds significantly during the
The ECMWF cloud fields are statistically quite accurate in the sense that they reproduce similar spatial distributions of window channel brightness temperatures as the AIRS observations. Our approach is to compare the observed window channel brightness temperatures to those we compute from the ECMWF model (and cloud) fields using our TwoSlab approximation. We then find the closest spatial match between a particular window region observation and nearby simulated window channel radiance (in a least squares sense). The cloud fields from the ECMWF grid point with the closest matching radiance are then used as the linearization point for the retrieval for the AIRS scene. We only retrieve cloud loading after the linearization, while keeping particle effective size, cloud top and bottom pressure, and cloud fractions unchanged.
For example, Fig.
The 1231 cm
We point out that the thermodynamic fields from ECMWF 3 h forecasts
(and/or analysis) are nearly identical to global radiosonde
measurements (see for example the figures in Sect. 3 of
If a single-footprint retrieval was used for production of a long time
series of AIRS level 2 products, we believe a reanalysis (such as the ECMWF
reanalysis, ERA; or the Modern-Era Retrospective Analysis for
Research and Applications, MERRA;
Three cross sections corresponding to the
black lines in the granule image are shown. The curves are BT1231
cm
This section focuses on AIRS Granule 039 from 11 March 2011, a day scene
over the tropical western Pacific containing many DCCs. Figure
A first step in testing retrieval performance is to examine the final
retrieval residuals and their standard deviations shown in
Fig.
The retrieval standard deviations are far smaller than those for ECMWF in
the region of strong water lines past 1400 cm
Retrieval results for Granule 039 shown in
Fig.
Figure
Retrieved degrees of freedom for Granule 039 show
evident dependence on observed BT1231 (which depends on cloud loading). The
red circles denote locations of the AIRS FORs (fields of regard) (
Temperature and water vapor retrieval statistics are shown in Figs.
These statistics have been separated by the total number of DOFs in the
retrieval. Figure
Retrieval statistics for Granule 039 for
thick clouds, defined by the number of DOFs to be
Same as Fig.
The low-DOF case (Fig.
The high-DOF cases in Fig.
A much more definitive diagnostic of retrieval performance is given in
Fig.
Comparing percent relative humidity
(color bar) along track “B” in Fig.
There is little water vapor structure in the a priori water vapor (fourth panel)
given that it is a monthly average of many years. The retrieved water
vapor (top panel) shows significant structure along this track, with
some small instabilities in the regions of thick high clouds (near
The ability of the retrieval to catch some of the upper-troposphere variability near 150 hPa seen in ECMWF indicates good vertical resolution as well. Note that the retrieval does not use any information from ECMWF, except for the clouds. We also comment that the upper troposphere and lower stratosphere (UTLS) humidity from the AIRS L2 is significantly lower than either ECMWF or our retrieval, with the blanked-out areas indicating where the surface AIRS L2 QA flags were bad.
The higher tropopause RH that was initiated by climatology remained unaffected by the retrieval; this could be alleviated by an improved first guess of the thermodynamic state, as well as choosing WV channels that peak very high in the atmosphere. In the future we plan to use a reanalysis as our a priori thermodynamic profiles, which will be adjusted by the retrieval when there are low to medium optically thick clouds. The use of a fixed shape for the ozone profile is also a limitation of the present retrieval that will be removed in later versions.
Comparisons between the initial ECMWF TwoSlab cloud parameters (found by matching window BTs to nearby ECMWF scenes) and the retrieved cloud parameters have a number of understandable differences. It is well known that NWP models do not produce as many deep convective clouds as observed so it is understandable that the mean ice cloud fraction changed from less than 0.5 to higher values ranging from 0.6 to 0.9 (these can be quite thin ice clouds). The water cloud fractions increased slightly from less than 0.3 and generally decreased for the higher factions. In addition, the frequency of high ice cloud tops (less than 250 mb) increased while the rest decreased; for water clouds the largest increase in frequency of occurrence was seen between 500 and 700 mb.
A quick validation of our ice cloud optical depths is achieved by
comparing AIRS L2 ice optical depths versus our retrieved ice cloud
loading (in g m
Retrieved ice cloud amount (in g m
The retrieval used cloud heights derived from matching to nearby ECMWF
cloud fields. Figure
The left-hand panel shows that the ECMWF ice cloud placement north of the
island of Papua New Guinea is at 8 km (light blue) with a number of high cloud tops
straddling the topmost part of the granule; the center panel shows our
algorithm moved the 8 km high clouds to be northeast of the island plus it
placed some very high cold DCC tops almost on a line along
145
Comparisons between cloud top height: (left)
original ECMWF ice cloud top; (center) ice cloud top heights used in the retrieval
(ice clouds with optical depth
A fast infrared radiative transfer algorithm with the ability to handle two scattering layers (from clouds, aerosols, volcanic dust) has been described and compared to a more sophisticated and often slower approach (maximum random overlap). Our ultimate goal is to perform single-footprint retrievals with hyperspectral IR sounder radiances. In particular we wish to handle the very common case of two cloud layers (water, ice) in order to provide accurate, higher spatial resolution retrievals of temperature and water vapor (and other minor gases). This approach uses the observed radiances in the retrieval rather than derived equivalent cloud-cleared radiances that are presently used for the NASA AIRS level 2 products. The complexity of true cloud structures cannot be retrieved with hyperspectral IR radiances, and we have shown that only a maximum of 2–4 cloud parameters can be derived from a single scene, suggesting that only a simple RTA is needed.
However, if the a priori cloud parameters are not sufficiently accurate, it can be very difficult for the retrieval to converge quickly, if at all. Our approach uses NWP model fields (here ECMWF) to initialize the cloud model fields. Four sub-columns (at most) are needed to compute a radiance for one scene, which is a small speed penalty in fast radiative transfer models, where most of the time is spent in computing the atmospheric optical depths. The TwoSlab model can be an order of magnitude faster than typical implementations of MRO and has nearly the same accuracy, both in terms of mean spectral radiances and radiance PDFs. The spectral bias between all-sky AIRS observations and calculations are dominated by spatial location mismatches between actual and forecast clouds. Both approaches used the ECMWF cloud fields, and in general both differed from observations similarly. For example, PCRTM–MRO is slightly more accurate in the tropics than SARTA–TwoSlab, while the opposite is true in polar regions. However, the comparisons of RTA simulations to observations are both limited by the accuracy of the NWP model fields and especially by small spatial mismatches between NWP and observed clouds. The larger errors of both RTA approaches in the polar regions indicate that ECMWF clouds have too-low optical depths.
We demonstrated the feasibility of the SARTA–TwoSlab approach by
performing single-footprint retrievals using an AIRS tropical granule.
Our approach to the retrieval cloud initialization and a priori thermodynamic and cloud parameters,
which is key to the successful results shown here, was to use NWP
cloud fields in the region of the footprint of interest based on
matching simulated and observed radiances. These matched cloud fields
are then converted from
A major advantage of single-footprint retrieval using the OEM approach is the retrieval quality diagnostics that are provided within the OEM framework. We demonstrated that the retrieval DOFs are reduced in the presence of thicker clouds. However, we were able to reproduce much of the water vapor variability in ECMWF (assumed to be relatively accurate, partly because it agrees with our retrievals) when using a climatology for the water vapor and temperature a priori.
Existing AIRS L2 retrievals fail in scenes with thick clouds and where
the 3-by-3 set of radiances used for cloud clearing are too homogeneous
(which is not always the case in thick clouds as seen in
Fig.
This work does not represent a rigorous analysis of the accuracy of our retrieval approach, but only a proof of principle that the technique appears viable. In particular, the temperature retrievals are not stressed in a tropical environment, although our results suggest significant skill for water vapor. The retrieval tests shown here were mostly all over ocean where the surface emissivity is well known. Over land, we will need to include a variable surface emissivity into the retrieval. The time taken to retrieve one single footprint (at the 100-layer native SARTA vertical resolution) is on average under 2.5 s, which includes matching the AIRS L1b radiances to climatology and NWP cloud fields and converting the NWP cloud profiles to slab clouds. This is very competitive with the official AIRS L2 product which takes about 1.5 second per field of regard using 20 trapezoid vertical functions (but does retrieve profiles of some additional trace gases and computes outgoing longwave radiation).
Data used for this study are available at
Clear-sky biases are likely to arise from inaccuracies in the
geophysical parameters, such as highly variable water vapor fields and
surface temperatures. The radiance measured by and simulated for the 1231 cm
A uniform mixing ratio ice cloud (
The SARTA–TwoSlab model has four parameters per cloud plus a cloud slab fraction and cloud slab overlap parameter that are derived from NWP model fields. The four parameters are the vertical placement and width of the slabs, the cloud loading (integrated CIWC or CLWC amounts) and the effective particle size. Since there are only 2–4 degrees of freedom for clouds in the spectra, for the retrieval only the cloud amounts were varied while the vertical placement, fraction and effective particle size were kept fixed, after the “closest” cloud was found.
Here we briefly explain the changes in the simulated radiances as the
cloud vertical placements are changed. An observation dataset of 7377
AIRS observations from 1 March 2009 is used here, as it was chosen to
provide maximum variability due to clouds, over land and ocean, and
span all climate regions (personal communication, George Aumann, Jet
Propulsion Laboratory, CA). The BT1231 cm
As can be seen from the whisker plots of Figs.
The authors declare that they have no conflict of interest.
We acknowledge the use of ECMWF model fields to
compute radiances. The hardware used in the computational studies is
part of the UMBC High Performance Computing Facility (HPCF). The
facility is supported by the US National Science Foundation through
the MRI program (grant nos. CNS-0821258 and CNS-1228778) and the
SCREMS program (grant no. DMS-0821311), with additional support from
the University of Maryland, Baltimore County (UMBC). See