Interactive comment on “ Profiles of CH 4 , HDO , H 2 O , and N 2 O with improved lower tropospheric vertical resolution from Aura TES radiances ”

Abstract. Thermal infrared (IR) radiances measured near 8 microns contain information about the vertical distribution of water vapor (H2O), the water isotopologue HDO, and methane (CH4), key gases in the water and carbon cycles. Previous versions (Version 4 or less) of the TES profile retrieval algorithm used a "spectral-window" approach to minimize uncertainty from interfering species at the expense of reduced vertical resolution and sensitivity. In this manuscript we document changes to the vertical resolution and uncertainties of the TES version 5 retrieval algorithm. In this version (Version 5), joint estimates of H2O, HDO, CH4 and nitrous oxide (N2O) are made using radiances from almost the entire spectral region between 1100 cm−1 and 1330 cm−1. The TES retrieval constraints are also modified in order to better use this information. The new H2O estimates show improved vertical resolution in the lower troposphere and boundary layer, while the new HDO/H2O estimates can now profile the HDO/H2O ratio between 925 hPa and 450 hPa in the tropics and during summertime at high latitudes. The new retrievals are now sensitive to methane in the free troposphere between 800 and 150 mb with peak sensitivity near 500 hPa; whereas in previous versions the sensitivity peaked at 200 hPa. However, the upper troposphere methane concentrations are biased high relative to the lower troposphere by approximately 4% on average. This bias is likely related to temperature, calibration, and/or methane spectroscopy errors. This bias can be mitigated by normalizing the CH4 estimate by the ratio of the N2O estimate relative to the N2O prior, under the assumption that the same systematic error affects both the N2O and CH4 estimates. We demonstrate that applying this ratio theoretically reduces the CH4 estimate for non-retrieved parameters that jointly affect both the N2O and CH4 estimates. The relative upper troposphere to lower troposphere bias is approximately 2.8% after this bias correction. Quality flags based upon the vertical variability of the methane and N2O estimates can be used to reduce this bias further. While these new CH4, HDO/H2O, and H2O estimates are consistent with previous TES retrievals in the altitude regions where the sensitivities overlap, future comparisons with independent profile measurement will be required to characterize the biases of these new retrievals and determine if the calculated uncertainties using the new constraints are consistent with actual uncertainties.

rors.This bias can be mitigated by normalizing the CH 4 estimate by the ratio of the N 2 O estimate relative to the N 2 O prior, under the assumption that the same systematic error affects both the N 2 O and CH 4 estimates.We demonstrate that applying this ratio theoretically reduces the CH 4 estimate for non-retrieved parameters that jointly affect both the N 2 O and CH 4 estimates.The relative upper troposphere to lower troposphere bias is approximately 2.8 % after this bias correction.Quality flags based upon the vertical variability of the methane and N 2 O estimates can be used to reduce this bias further.While these new CH 4 , HDO/H 2 O, and H 2 O estimates are consistent with previous TES retrievals in the altitude regions where the sensitivities overlap, future comparisons with independent profile measurement will be required to characterize the biases of these new retrievals and determine if the calculated uncertainties using the new constraints are consistent with actual uncertainties.

Introduction
Investigating the processes controlling the water and carbon cycles and their linkages require multiple tracers that are sensitive to the vertically distributed sources, sinks, and processes controlling the water and carbon cycles.Measurements of water vapor profiles (e.g., Dessler et al., 2007 Published by Copernicus Publications on behalf of the European Geosciences Union.and references therein), upper tropospheric water (e.g., Reed et al., 2008) and the vertical distribution of clouds (e.g., Stephens and Vane, 2007;Su et al., 2008) have been used to examine the exchange and transport processes controlling tropospheric humidity.Measurements of the isotopic ratio of water can provide an additional constraint for quantifying the distribution of the sources and exchange processes through the sensitivity of this composition to that of the moisture source, to changes in phase, and to transport and mixing processes (e.g., Kuang et al., 2003;Worden et al., 2006Worden et al., , 2007;;Risi et al., 2008;Nassar et al., 2007;Payne et al., 2007;Brown et al., 2008;Noone et al., 2008;Frankenberg et al., 2009;Herbin et al., 2009;Steinwagner et al., 2010;Schneider and Hase, 2011).Satellite measurements such as those from the Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) instrument, the Aura TES instrument, the Atmospheric Chemistry Experiment (ACE), the Infrared Atmospheric Sounding Interferometer (IASI), and the SCanning Imaging Absorption SpectroMeter for Atmospheric CHartographY (SCIAMACHY) have been used for this purpose.Similarly, any of the dynamical processes controlling the water cycle such as surface exchange, mixing, advection, and convection also affect the carbon cycle.As with water, mixing processes in the free troposphere (e.g., Jiang et al., 2008;Sarrat et al., 2010;Li et al., 2010, Lee et al., 2007;Risi et al., 2008) and boundary layer (e.g., Stephens et al., 2007a, Picket-Heaps et al., 2011;Querino et al., 2011) affect the tropospheric distribution of CO 2 (Nassar et al., 2011) and CH 4 and must be accounted for when estimating fluxes and emissions.
Consequently, in order to investigate the processes, sources, and sinks affecting the global carbon and water cycles it is useful to have vertically resolved trace gas profiles.It is with this motivation that we seek to improve the vertical resolution of the TES H 2 O, HDO, and CH 4 products, especially in the lowermost troposphere and boundary layer where many of the exchange processes between the surface, boundary layer, and free troposphere have significant impact on the tropospheric distribution of these gases.

The TES instrument and trace gas retrieval overview
The TES instrument is an infrared, high spectral resolution, Fourier Transform spectrometer covering the spectral range between 650 to 3050 cm −1 (15.4 to 3.3 µm) with an apodized spectral resolution of 0.1 cm −1 for the nadir view (Beer, et al., 2001).Spectral radiances measured by TES are used to infer atmospheric profiles using a non-linear optimal estimation algorithm that minimizes the difference between these radiances and those calculated with the equation of radiative transfer (Clough et al., 2006), subject to the constraint that the parameters are consistent with a statistical a priori description of the atmosphere (Rodgers, 2000;Bowman et al., 2006).TES provides a global view of tropospheric trace gas profiles including ozone, water vapor and its isotopes, carbon monoxide and methane, along with atmospheric temperature, surface temperature, surface emissivity, effective cloud top pressure, and effective cloud optical depth (Worden et al., 2004;Kulawik et al., 2006b;Eldering et al., 2007).

Spectral windows
A common approach when performing retrievals from high resolution Fourier transform spectrometers such as TES is to select spectral windows for each target atmospheric constituent that maximize information gained from a spectral measurement and minimize the systematic errors related to incorrect knowledge of temperature, emissivity, spectral errors, or radiative interference from un-retrieved species (e.g., Echle et al., 2000;Dudhia et al., 2002;Worden et al., 2004;Kuai et al., 2010).The details of the approach for the TES spectral window selection are described in Worden et al. (2004).The general procedure is to first compute an error budget for a set of spectral windows using the following equation: where x is the estimate of interest and the subscript "a" indicates that a priori knowledge is used for the corresponding vector.The A xx is the averaging kernel matrix describing the sensitivity of the estimate to the true state: A = ∂ x ∂x .The A xy is the sensitivity of x to other parameters (y) that are jointly estimated with x.The M is a mapping matrix relating retrieval parameters z back to the full profile x, x = Mz (many retrievals use a subset of parameters as a hard constraint to regularize the retrieval as discussed in Worden et al., 2004 andBowman et al., 2006).The vector m is the measurement noise as a function of wavelength.The b term represents unretrieved parameters that affect the observed radiance with K b being the Jacobian or sensitivity of those terms to the radiance.The G is the gain matrix, which is the partial derivative of the retrieval parameters to the radiance (F) where S m is the covariance of the measurement noise for an ensemble of measurements and z is a constraint matrix used to regularize the retrieval.Note that z can take on different forms such as a Tikhonov-type squared nth order difference matrix (e.g., Steck, 2001), a hybrid constraint (e.g., Kulawik et al., 2006a) or the inverse of a climatology (Rodgers, 2000).The last term in Eq. ( 1) is the sum over all terms that are not retrieved with the state vector x but which also affect the measured or modeled radiance.Since in general the noise vector and the errors in these parameters are not exactly known we instead use their known second order statistics to calculate the errors in x from each term: where these four terms correspond to the terms in Eq. ( 1): S tot is the total error, the first term that is dependent on S a is an estimate of the "smoothing error" which describes how well the estimate can infer the natural variability of the atmosphere (Rodgers 2000).In principal, the S a term should describe the true statistics of the atmosphere for observed air parcel, presumably using in situ data.In practice, these statistics are typically un-available for most remotely sensed observations so that we use other calculations, such as global models, to calculate S a .For a retrieval that has perfect resolution on the prescribed vertical grid, the Averaging kernel matrix (A xx ) would be the identity matrix and the smoothing error would be zero.The second term depending on S y , is similar to the smoothing error and characterizes the impact of the natural variability of jointly estimated parameters on the parameters of interest (Worden et al., 2004).The third term depending on S m term describes the statistics of the observation (in this case radiance) error due to noise (m); the mapping and gain matrices (M and G) are then used to calculate the impact of this noise on the estimate.The last term is a summation over all non-retrieved parameters (b) which could include spectroscopic uncertainties, temperature, or non-retrieved species.In our case, the included parameters are temperature and cloud top height.
In general, spectral window selection involves calculating whether a measurement adds information (using a definition of Shannon information content that is related to a decreased uncertainty) using the following equation: where H is a scalar and is the information content, S x1 is the error covariance before adding a measurement and S x2 is the error covariance after adding a measurement.In order to make full use of the available spectral information without negatively adding information content it is necessary to jointly retrieve all constituents together (Worden et al., 2004).If all constituents are jointly retrieved then the last term in Eq. ( 3) becomes zero and all data points increase the information content.Similar to Schneider and Hase (2011), our approach then is to use effectively the entire 8 micron spectral range shown in Fig. 1 to jointly estimate HDO, H 2 O, N 2 O, and methane.However, we currently avoid a 10 cm −1 wide spectral region centered around 1280 cm −1 and a 2 cm −1 wide spectral region centered at 1308 cm −1 which contains a strong CFC absorption feature and the methane Q branch respectively.Other interfering species such as CO 2 , O 3 , and HNO 3 are included in our forward model.

State vector
The new state (column) vector for this joint estimate is: where the column vectors x are on a 67 level pressure grid ranging from 1000 hPa to 0.1 hPa (Worden et al., 2004), T surface is the surface temperature, and τ cloud is the cloud effective optical as a function of frequency (e.g., Kulawik et al., 2006b;Eldering et al., 2007).As discussed earlier the retrieval vector elements corresponding to the trace gasses and the cloud optical depth are actually the log of the trace gas amount or cloud optical depth respectively.The atmospheric species are retrieved on a subset of the 67 level pressure grid used in the TES forward model; this effective hard constraint is described by the mapping matrix "M" and the retrieval levels "z" in Eq. ( 1) (Worden et al., 2004;Bowman et al., 2006) and must formally be included in the error analysis; however, for the sake of brevity we exclude this term in subsequent equations.Figure 1b.Same as in Figure 2a but for H 2 O and HDO.2a.Averaging kernels for a TES water retrieval using old (spectral windows shown in Fig. 2b) and new (using almost all the radiance shown in Fig. 2b).The diamonds indicate the pressure level for the averaging kernel.Color coding is to help the reader follow the variability of each averaging kernel with pressure.

Constraints
A primary objective for these new TES retrievals is to increase the vertical resolution and information content of methane, H 2 O, and the HDO/H 2 O ratio in the lower troposphere.The added spectral data cannot by themselves allow for these objectives to be met because the choice of regularization in the previous versions of the TES data limited sensitivity at specific altitudes in order to reduce impacts of nonlinearity on the retrieval due to low sensitivity.Consequently, we need to change both the hard constraint (or retrieval levels and mapping matrices) as shown by Eq. ( 5) and the soft constraints (constraint matrix shown in Eq. 2).Previously, the retrieval levels (z) for H 2 O and HDO in the lower troposphere (surface to 500 hPa) tropospheric were defined as every other forward model level (x); with the mapping matrix using linear in (log) pressure and (log) mixing ratio to interpolate between retrieval levels and forward model levels.The new retrieval levels in the lower troposphere now have a one-to-one mapping with the TES forward model levels for H 2 O and HDO.For methane, the retrieval level density has been increased from every 3rd level to every 2nd forward model level for CH 4 .The constraints were selected based on the altitude-dependent Tikhonov constraints as described in Kulawik et al. (2006a).
In optimal estimation, the constraint matrix is typically calculated from the known a priori statistics of the atmosphere (e.g., Rodgers 2000).These statistics are most easily generated from global chemical or climate models.However, covariances from these models are not typically invertible, can vary from model to model, and may not replicate actual correlations for molecules such as HDO that are not Figure 2a: Averaging kernels for a TES water retrieval using old (spectral windows shown in Figure 2b) and new (using almost all the radiance shown in Figure 2b).
The diamonds indicate the pressure level for the averaging kernel.Color coding is to help the reader follow the variability of each averaging kernel with pressure.well observed.We therefore modify the derived correlations from the models by the sensitivity of the radiances to each geophysical parameter (e.g., Kulawik et al., 2006b) or from insight derived from more recent data sets such as water vapor isotope data at the Mauna Loa observatory (Worden et al., 2011).For the new TES retrievals of H 2 O, HDO, and CH 4 , the correlation length scales in the constraint matrices (not shown as the larger variance and negative correlations make these plots difficult to generate) have been reduced between the mixing layer (typically surface to 825 hPa) and lower troposphere to reflect conclusions drawn from recent in situ and satellite based observations of these constituents (e.g., Frankenberg et al., 2005Frankenberg et al., , 2009;;Worden et al., 2011;Pickett-Heaps et al., 2011;Noone et al., 2011).

Comparison of previous (Version 4) and new profile retrievals
The effective vertical resolution (as characterized by the averaging kernels) and the calculated uncertainties of these new data are compared to the earlier retrieval approach.We also compare old versus new retrievals for the altitude region in which the vertical sensitivities overlap.

H 2 O
Figure 2a shows the averaging kernels for the new and old H 2 O retrievals for a tropical ocean case and Fig. 2b shows the square-root of the diagonals of the corresponding a priori, a posteriori error, and observation covariances (measurement + interference error).As discussed earlier, the averaging kernels (or rows of the averaging kernel matrix) describe the sensitivity of estimate to the true state, e.g., A = ∂ x ∂x where x is the estimate and x is the true state.As shown in Eq. ( 1), in the absence of uncertainties, the estimate is related to the true state via the a priori constraint and the averaging kernel matrix (Rodgers, 2000): An "ideal" averaging kernel would approach the identity matrix.The rows would exhibit narrowly defined peaks, with the peak value of each row located at the pressure of the retrieval level assigned to that row.In the absence of error, the retrieved estimate would then approach the true state.Figure 2a shows that the H 2 O averaging kernels have narrower vertical extent and are more distinct for the new retrievals, while Fig. 2b shows that the uncertainties for the new retrieval are overall reduced, except near pressures around 700 hPa for this retrieval.
Figure 3 shows the RMS difference and bias between the new (TES Version 5) and older (TES Version 4) H 2 O profile retrievals.The RMS difference is consistent with the random uncertainties in the estimate as seen in the previous figure.In addition, the bias between the versions is effectively zero except at the lowermost pressures where the sensitivity has increased in the tropics through the mid-latitudes.The poorer precision at 700 hPa is due to modification of the constraint which reduced the correlation length scales between the boundary layer (surface to 825 hPa) and the free troposphere (825 hPa to ∼150 hPa).

HDO/H 2 O ratio
The TES HDO and H 2 O retrieval approach is designed to reduce the uncertainties in the HDO/H2O ratio estimate as opposed to HDO or H 2 O separately.(e.g., Worden et al., 2006;Schneider et al., 2006;Schneider and Hase, 2011).Consequently, the constraint used to regularize this retrieval is based on an a priori covariance that characterizes the HDO/H 2 O ratio variability, under the assumption that HDO and H 2 O are jointly estimated, i.e.: where S H a is the a priori covariance for H 2 O and S R a is the a priori covariance for the HDO/H 2 O ratio.The a priori covariance for water, S H , is constructed using statistics from the MOZART (e.g., Brasseur et al., 1998;Horowitz et al., 2003) model but scaled to the expected uncertainty of NCEP water content predictions (Worden et al., 2004).The a priori statistics for S R are originally based on a version of the National Center for Atmospheric Research (NCAR) Community Atmosphere Model (CAM) that has been modified to predict the isotopic composition of water using the approach developed by Noone and Simmonds (2002).However, we now error covariance is the sum of the observation error and smoothing err covariances.adjust S R to reduce correlations between the PBL and the lower troposphere and increase the variance in the boundary layer and free troposphere, consistent with recent observations of the PBL and free troposphere in the subtropics at Mauna Loa (Worden et al., 2010, Noone et al., 2011).This is an ad hoc change to the S R covariance that we implement to obtain separation of the HDO averaging kernels so as to allow profiling of the HDO/H 2 O ratio; once we have enough in situ data of HDO/H 2 O profiles or our confidence in model estimates of the HDO/H 2 O ratio improves at the fine-scale (∼10 km), we will change this covariance in order to make the constraint "more optimal".Note that we also only use a single S R matrix for the HDO/H 2 O constraint globally for similar reasons.
There is no unique averaging kernel for the estimate of the HDO/H 2 O ratio (Worden et al., 2006) because the sensitivity of the HDO/H 2 O ratio depends on both HDO and H 2 O.However, the averaging kernels for the HDO estimate will typically span a subset of the averaging kernels for the H 2 O estimate.Therefore, the HDO averaging kernel is a good approximation of the vertical sensitivity for the HDO/H 2 O estimate characteristics.On the other hand, because the HDO averaging kernels do not perfectly span that of the H 2 O averaging kernels, the true sensitivity of the HDO/H 2 O estimate is likely smaller than that of the HDO estimate.
The HDO averaging kernel matrix and square root of the diagonal of the HDO/H 2 O error covariances are shown in Fig. 4 for the same tropical case shown in Fig. 2. The degrees-of-freedom for signal (DOFS) for the HDO estimate has greatly increased; we find in general that approximately half of the increase in sensitivity is due to the changed Figure 4a: Averaging kernel for the old and new HDO TES retrievals.As in Figure 2a, the symbols and colors indicate the pressure level and variation with pressure of each row of the averaging kernel matrix.Figure 4b: Same as in Figure 2b but for the HDO/H 2 O ratio.The units are approximately the fractional uncertainty as the TES trace gas retrievals estimates the log of the concentrations.constraint and the other half due to the increased number of radiance measurements used for the retrieval.There is a net increase in the precision error in the boundary layer due to temperature and noise of approximately 3 %; whereas the previous precision was only about 0.5 percent in the boundary layer.On the other hand, the smoothing error in the boundary layer has decreased because of the increased sensitivity.For convenience we have used here the ad hoc S r covariance used to develop the new constraint to calculate the smoothing error; however, the user of this data can better calculate the smoothing error by providing their own S r covariance and using the averaging kernels provided by the TES data products.
This poorer precision in the boundary layer is a consequence of our new and more relaxed constraint.It is quite likely that the total error in the tropical oceanic boundary layer is made worse relative to the prior version that applied a stronger constraint.At least we would expect that the isotopic composition in the boundary layer should only range from 70-80 ‰ under quiescent conditions (Lawrence et al., 2004).Note that the HDO/H 2 O ratio is typically given in parts per thousand (‰) relative to the isotopic composition of ocean water (‰) or δ −D = 1000(R/R std −1.), where R is the HDO/H 2 O mole ratio and R std = 3.11 × 10 −4 is 2 times the isotope ratio of the Vienna Standard mean Ocean water reference for the D/H.However, we believe this new constraint is reasonable for estimating the variability of the HDO/H 2 O ratio over continents and at higher latitudes because of the the larger expected variability of the isotopic composition in the boundary layer in these regions (e.g., Risi et al., 2010;Yoshimura et al., 2011).
Figure 4a: Averaging kernel for the old and new HDO TES retrievals.As in Figure 2a the symbols and colors indicate the pressure level and variation with pressure of each row of the averaging kernel matrix.Figure 4b: Same as in Figure 2b but for the HDO/H 2 O ratio.The units are approximately the fractional uncertainty as the TES trace gas retrievals estimates the log of the concentrations.

Global comparison of Version 5 and previous HDO/H 2 O estimates
TES products prior to version 5 have been validated in the lower troposphere by comparing TES estimates to in situ measurements of HDO and H 2 O at the Mauna Loa observatory (Worden et al., 2011).While there is insufficient data to provide direct validation of the profiles of the new TES HDO/H 2 O estimates in the free troposphere, we can compare the new TES estimates in the lower troposphere to the older estimates in the lower troposphere where the sensitivities overlap.This comparison is shown in Fig. 5.The first panel of Fig. 5 shows the latitudinal distribution of the HDO/H 2 O ratio between the old and new HDO/H 2 O estimates for the vertical range between 825 and 500 hPa for all scenes in which the degrees of freedom for signal (or trace of the averaging kernel) are larger than 1.0.For a log-based retrieval, the DOF is a good metric for retrieval sensitivity as it indicates how well an ensemble of estimates captures the range of variability of the true distribution.For example, if the DOFS is 0.5 for some altitude range then that means a distribution of estimates, averaged over that altitude, could be expected to capture half the natural variability of the true distribution.The data in the top panel of Fig. 5 are taken from one TES global survey in July 2005.As can be seen in this figure, there are many more retrievals at higher latitudes that meet this DOF's criteria as the sensitivity of the new retrievals have improved.The bottom panel shows the difference between the new and old estimates, averaged between 825 and 500 hPa, for all retrievals with DOFS larger than 1.0 in order to reduce uncertainty in the comparison due to differences in the retrieval sensitivity.

Global Estimates of the HDO/H 2 O ratio for July 2006
A limited number of TES global surveys have been processed with the new retrieval approach and the results are shown in Fig. 6.The top panel of Fig. 6 shows the HDO/H 2 O ratio for the altitudes approximately corresponding to the free troposphere (800 to 300 hPa) and the bottom panel shows the HDO/H 2 O ratio for altitudes that approximately corresponds to the boundary layer (surface to 800 hPa) regions.Values of the HDO/H 2 O ratio are given in ‰ and have been corrected for the estimated TES bias discussed in the previous section (Worden et al., 2011).Only data in which the DOFS for the HDO estimate is larger than 1 and where the cloud optical depth is less than 0.4 are shown.Note that even though the DOFS can be approximately one, the HDO/H 2 O profile can still distinguish boundary layer variability from free tropospheric variability of the HDO/H 2 O ratio as long as the peak values of the averaging kernels (rows of averaging kernel matrix) in these regions are separated; this condition should be met for most clear-sky regions.In the boundary layer above the ocean, mean values of the HDO/H 2 O ratio are approximately −74 ‰ with an RMS variance of 37 ‰, consistent with the 3 % uncertainty shown for the tropical case in Fig. 4b (for isotopic values near 0.0 a 3 % uncertainty corresponds to 30 ‰ uncertainty).The −74 ‰ mean value for the mean tropical ocean boundary layer is consistent with in situ measurements for boundary layer water vapor (e.g., Lawrence et al., 2004;Galewsky et al., 2007;Worden et al., 2011) and therefore suggests that the bias correction calculated for the previous TES HDO/H 2 O estimates are applicable for these data.

CH 4 profiles
In this section we describe the changes in the vertical resolution and error characteristics of the new TES CH 4 methane retrievals as well as biases in the profiles.We then discuss approaches for correcting or accounting for this bias including averaging, or correcting the methane estimate using the co-retrieved N 2 O estimate.However subsequent analysis using independent methane data sets will be needed in order to determine the optimal approach for this bias correction.and 300 hPa indicating that methane variations at these altitudes can theoretically be distinguished from one another provided the vertical variations are larger than the expected uncertainties.This increased sensitivity to the lower and middle troposphere is due to use of the methane lines around 1230 cm −1 (Fig. 1a) because the lower optical thickness at these wavelengths allows for improved sensitivity to lower tropospheric methane; Fig. 8 shows the DOF's for the new and older methane retrievals.Typically there are about 0.5 DOFS more for the new retrieval than the old with the increased sensitivity in the middle/lower troposphere.

CH 4 error characteristics
Error characteristics for the TES methane estimate using the radiance from the tropical scene shown in Fig. 1 are presented in the left panel of Fig. 9.For the TES methane retrieval we assume an a priori 5 % uncertainty in methane but with significant (>50 %) cross-correlations between adjacent    levels (not shown) because methane is a well mixed gas in the free troposphere (e.g., Fung et al., 1991;Wofsy et al., 2011).The a priori variability of 5 % used to generate the CH 4 constraint is in part an ad hoc choice and is based on the measured variability of the latitudinal gradient which ranges from 1700 to 1900 ppb.However, we expect that this variability is a non-optimal choice for the TES methane retrievals because methane variability in any one location is typically smaller than 5 % (Wofsy et al., 2011); however, much of this variability will be decreased by averaging such as used for the TES CO 2 retrievals (Kulawik et al., 2010;Nassar et al., 2011).We will likely change this covariance and constraint  in the next TES algorithm release based on experience using the TES CH 4 data with global models and more direct comparisons to in situ data to inform our a priori choices.
For this case, the observation error describes the estimated error from noise and from co-retrieved geophysical parameters such as H 2 O, HDO, surface temperature, and clouds.Because temperature is retrieved from a previous step using the CO 2 ν2 band around 700 cm −1 , its error estimate is shown separately.As can be seen in this Fig. 9, uncertainty due to temperature is the largest component of the methane retrieval error budget in the lower/middle troposphere.

Global distribution of TES observed methane and biases
Because of the long life-time of approximately nine years for methane (e.g., Frankenberg et al., 2005) we would expect that methane should be a vertically well mixed gas in the free troposphere (e.g., Wofsy et al., 2011;Pickett-Heaps et al., 2011) but showing a latitudinal gradient that depends on inter-hemispheric mixing, the preponderance of northern hemispheric methane sources relative to the southern hemisphere, and the distribution of OH which is the primary sink for CH 4 (e.g., Fung et al., 1991).Consequently, it is reasonable to show a two-dimensional figure of the vertical profile of methane as a function of latitude, averaged over all longitudes as well as ocean and land scenes, in order to infer any vertical biases in the TES methane estimates.Figure 10 shows the TES estimated vertical distribution of methane as a function of latitude for all data taken during July 2006.A feature of this distribution is that methane is biased high in the upper troposphere and lower stratosphere.This upper  tropospheric bias was suspected for previous TES methane estimates that were only sensitive to methane in the upper troposphere (Payne et al., 2009).Based on these observations we suspect that either a systematic bias in temperature is affecting the TES methane estimates, or that temperature dependent uncertainties in the methane spectroscopic line strengths are affecting these estimates.The fact that MI-PAS retrievals based on the same CH 4 band are also biased high (e.g., von Clarmann et al., 2009) seems to support the later hypothesis.Another possibility is that the bias is partly due to anti-correlations of the estimated upper tropospheric methane with the middle/lower tropospheric methane estimate as shown in the methane averaging kernels (right panel Fig. 7); in order to determine if this anti-correlation could account for some of this bias we show a global map of the middle troposphere at 618 hPa versus a global map using an information based averaging approach described by Payne et al. (2007) which maps each profile to one or two levels that best represent the altitude where the estimate has the most sensitivity; this approach limits the impact of the a priori on an average because the averaging kernel approaches unity for the re-mapped estimate.For the approach using the Payne et al. ( 2007) algorithm we only choose methane estimates for which the pressure of the re-mapped (or information averaged) estimate is greater than 450 hPa.While both maps show an expected latitudinal gradient, the map using the methane estimate from the TES 562 hPa pressure level shows un-physically high methane at around −50 degrees relative to the tropics; however, the map derived from the averaged values shows a more realistic latitudinal gradient as compared to previous measurements (e.g., Frankenberg et al., 2006).This result suggests that the anti-correlations Figure 11: (Top) TES estimated methane at 562 hPa.(Bottom) TES estimated methane at approximately 500 hPa using an "information" averaging approach.in the profile estimate accounts for part of this bias.Future comparisons between the TES data and independent methane measurements will be needed to further characterize this bias so that this data can be used for understanding the global methane cycle.In the next section, we describe an additional approach (e.g., Razavi et al., 2009) in which we correct the methane estimate using co-retrieved N 2 O estimates.The theoretical calculation of errors using this approach is promising but depends on accurate a priori knowledge of the tropospheric and stratospheric N 2 O distribution.

Methane profile correction using N 2 O estimate
In this section we describe an approach for reducing errors in the methane estimates using the co-retrieved N 2 O estimates.Although N 2 O varies much less than CH 4 in the troposphere, the magnitude of the sensitivity of the radiance to variations in N 2 O and CH 4 are nearly the same in the 8 micron spectral region as shown by their normalized column Jacobians in Fig. 1a.Consequently, errors that affect estimates of N 2 O will have a similar radiative effect as errors that affect estimates of CH 4 .For this correction approach using N 2 O, we therefore assume that the tropospheric N 2 O profile is well represented by the a priori profile, and that deviations in the retrieved N 2 O from the prior are a result of systematic error.For these estimates we use a priori N 2 O profiles from the Whole Atmosphere Community Climate Model (WACCM) (e.g., Tilmes et al., 2007).Interference error from temperature, clouds, and emissivity should therefore affect both CH 4 and N 2 O very similarly, and correction of CH 4 by N 2 O should therefore reduce the CH 4 errors.This correction takes the following form: where x c is the estimate for (log) methane, x n is the (log) estimate for N 2 O, and the adj superscript means "adjusted" or corrected.Because this is simply the ratio of two numbers (for a logarithm) modified by an a priori constraint we can use the same derivation for the errors in the HDO/H 2 O estimate as described in Worden et al. 2006or Schneider et al. (2006).For the methane estimate this leads to: Results show that each term of the cross averaging kernels for the N 2 O and CH 4 estimates are small relative to the averaging kernels for N 2 O and CH 4 (A nc A cc and A cn Figure 12: TES CH4 as a function of latitude after applying N2O correction.A nn ); consequently we can ignore the cross averaging kernels.Under the assumption that the variability of N 2 O in the atmosphere is much smaller than the variability of CH 4 (Wofsy et al., 2011) in the atmosphere we can ignore the term associated with S nn .This leads to an error estimate for methane, corrected by the N 2 O estimate of: The right panel of Fig. 9 shows the error budget for these terms.While the observation error (error due to noise and from jointly estimated parameters such as H 2 O, clouds, etc) and smoothing error is approximately the same for CH 4 with and without the N 2 O correction, the temperature error is much smaller after applying the N 2 O correction, providing evidence of the reduction in errors due to parameters that are not retrieved.For example, Fig. 12 shows the two-dimensional (latitude versus altitude) distribution of TES estimated CH 4 methane.In addition to correcting the CH 4 estimate using the coretrieved N 2 O estimate, we also only keep CH 4 estimates in which the vertical variability is less than 1.5 % or the corresponding N 2 O estimate is less than 0.5 %.These choices of 1.5 % and 0.5 % are a compromise between finding data that is most physically plausible (i.e., we expect the vertical variability of methane to be less than 1.5 %) and the need for enough data to look at the global methane distribution.Using the N 2 O correction and this quality flag, we find that the bias in the upper troposphere is greatly reduced and the vertical variability has been decreased.

Summary
This manuscript documents improvements to the Aura TES profile estimates of H 2 O, HDO/H 2 O, and CH 4 by using a joint retrieval over a wide spectral range and new, less stringent constraints.These new products are now being generated and are called Version 5.In general, the vertical resolution of H 2 O has increased in the lower troposphere with improved capability to distinguish between boundary layer variability of H 2 O and that of the free troposphere.Previous (version 4 or less) retrievals could not profile the HDO/H 2 O ratio but were instead sensitive to an average over the lower troposphere between 550 and 825 hPa.New TES estimates of the HDO/H 2 O profile can now distinguish between the boundary layer/lower troposphere and the middle troposphere around 550 hPa with uncertainties of approximately 30 ‰ for the HDO/H 2 O ratio in the boundary layer.We show that the new and old estimates for the HDO/H 2 O estimates are consistent within the expected uncertainties in the regions where the vertical sensitivity overlaps.The new profiling capability is useful for examining isotopic variability for high latitude and continental scenes where there is large variability in the lowermost atmospheric levels.On the other hand the less stringent constraint increases the total error of the HDO/H 2 O estimate in the tropical oceanic boundary layer.
The new TES methane estimates are now sensitive to methane variability from approximately 800 hPa to 200 hPa whereas previous TES retrievals were only sensitive to methane in the mid-to upper troposphere.However, there is clearly a bias in the upper tropospheric methane that must be better characterized with respect to other parameters that affect the TES methane estimates before this profile information can be used.The effect of this bias on the latitudinal distribution of the TES methane profiles is mitigated if the profiles are averaged to account for the vertical distribution of the sensitivity of the estimate to methane (Payne et al., 2007).We also show both theoretically and empirically that the bias in the estimated methane can be further mitigated using the co-retrieved N 2 O estimate.Validation of the new H 2 O, HDO/H 2 O, and CH 4 profiles in regions with increased vertical sensitivity will require comparisons to independent measurements and will be presented in subsequent papers.
Figure 1a: (top) Example of radiance measured by TES over a tropical ocean scene.(Middle) Sensitivity of TOA radiance to (log) CH4, integrated over the whole atmospheric column and normalized by the TES NESR.(Bottom) Same as middle but for N 2 O.The red shaded area indicates the spectral region used for TES Version 4 methane retrievals.

Fig. 1a .
Fig. 1a.(Top) Example of radiance measured by TES over a tropical ocean scene.(Middle) Sensitivity of TOA radiance to (log) CH 4 , integrated over the whole atmospheric column and normalized by the TES NESR.(Bottom) Same as middle but for N 2 O.The red shaded area indicates the spectral region used for TES Version 4 methane retrievals.

Fig. 1b .
Fig. 1b.Same as in Fig. 1a but for H 2 O and HDO.
Figure 2b.The square root of the diagonal of the error covariances.The units are approximately the fractional uncertainty as the TES trace gas retrievals estimates the log of the concentrations.Observation error includes uncertainties from jointly retrieved parameters affecting the radiance and error due to noise.The a posteriori Fig. 2b.The square root of the diagonal of the error covariances.The units are approximately the fractional uncertainty as the TES trace gas retrievals estimates the log of the concentrations.Observation error includes uncertainties from jointly retrieved parameters affecting the radiance and error due to noise.The a posteriori error covariance is the sum of the observation error and smoothing error covariances.

Figure 3 :
Figure 3: The RMS and Bias of the fractional difference between the ne H 2 O retrievals.Tropics indicate all latitudes less than 20 degrees (Nort Mid--latitudes are between 20 and 50 degrees (North and South) and H are all latitudes greater than 50 degrees North (to avoid Antarctica wh have very low sensitivity).

Fig. 3 .
Fig. 3.The RMS and Bias of the fractional difference between the new and old TES H 2 O retrievals.Tropics indicate all latitudes less than 20 degrees (North and South).Mid-latitudes are between 20 and 50 degrees (North and South) and High Latitudes are all latitudes greater than 50 degrees North (to avoid Antarctica where retrievals have very low sensitivity).

Fig. 4a .
Fig. 4a.Averaging kernel for the old and new HDO TES retrievals.As in Fig.2a, the symbols and colors indicate the pressure level and variation with pressure of each row of the averaging kernel matrix.

Fig. 4b .
Fig. 4b.Same as in Fig. 2b but for the HDO/H 2 O ratio.The units are approximately the fractional uncertainty as the TES trace gas retrievals estimates the log of the concentrations.
Figure 5: (top) Comparison of the new and old (Version 6) HDO/H 2 O estimates.A DOFS threshold of 1.0 is used for the data in the top panel for both releases.(bottom) Difference between old and new HDO/H 2 O estimates for the overlapping data shown in the top panel.δ--D = 1000(HDO/H 2 O/3.11x10 --4 --1.).

Fig. 5 .
Fig. 5. (top) Comparison of the new and old (Version 6) HDO/H 2 O estimates.A DOFS threshold of 1.0 is used for the data in the top panel for both releases.(bottom) Difference between old and new HDO/H 2 O estimates for the overlapping data shown in the top panel.δ − D = 1000(HDO/H 2 O/3.11 × 10 −4 −1.).

Figure
Figure 7a and b shows the averaging kernels for the previous and new CH 4 estimate for the same tropical case shown in Figs. 2 and 4 for H 2 O and HDO.The new CH 4 methane profile estimates generally show increased sensitivity to the lower and mid troposphere between 825 and 450 hPa.In addition, the averaging kernels generally peak around 650 hPa

Figure 6 (
Figure 6 (bottom panel) Averaged TES HDO/H 2 O estimates for pressures greater than 800 hPa.(top panel) Averaged TES HDO/H 2 O estimates for pressures between 300 and 800 hPa.

Figure 7 :
Figure7: Averaging kernels for the old (Version 6 or less) TES methane retrievals and the new TES methane retrievals.The diamonds indicate the pressure level for the averaging kernel.Color coding is to help the reader follow the variability of each averaging kernel with pressure.

Fig. 7 .
Fig. 7. Averaging kernels for the old (Version 6 or less) TES methane retrievals and the new TES methane retrievals.The diamonds indicate the pressure level for the averaging kernel.Color coding is to help the reader follow the variability of each averaging kernel with pressure.

Figure 8 :
Figure8: DOFS for the new methane retrieval (yellow) and the old methane retrieval (black).

Fig. 8 .
Fig. 8. DOFS for the new methane retrieval (yellow) and the old methane retrieval (black).

J.
Figure9: Error budget for the methane estimate before and after correcting the methane profile with the ratio of the TES estimated N 2 O and the TES N 2 O a priori.Observation error is the sum of the measurement error related to noise and due to jointly estimated parameters.The total error is the sum of the smoothing, observation, and temperature error.

Fig. 9 .
Fig. 9.Error budget for the methane estimate before and after correcting the methane profile with the ratio of the TES estimated N 2 O and the TES N 2 O a priori.Observation error is the sum of the measurement error related to noise and due to jointly estimated parameters.The total error is the sum of the smoothing, observation, and temperature error.
Figure 11 (bottom panel) shows global methane estimate from TES for July 2006 for re-mapped estimate.The average pressure for this re-mapped estimate is approximately 500 hPa. Figure 11 (top panel) shows the TES global methane estimate for July 2006 for the 562 hPa pressure level.

)
Note that the full averaging kernel contains entries for the joint estimate of CH 4 , N 2 O, H 2 O, HDO, surface temperature, clouds and emissivity.The A cc term is the component of this averaging kernel that just corresponds to the (log) CH 4 estimate.The A cn term is the component of the averaging kernel that represents how the (log) N 2 O estimate affects the jointly retrieved (log) methane estimate (using indices n for N 2 O and c for CH 4 ).The term G r is the gain matrix for the CH 4 methane part of the retrieval vector minus that of the N 2 O part of the retrieval vector (G r = G c − G n ).The term G R m is the impact of measurement noise on the estimate.The index j is for jointly retrieved parameters such as H 2 O or HDO and the index i refers to un-retrieved parameters such as atmospheric temperature, spectroscopy or calibration.Taking the expectation of the adjusted CH 4 methane estimate minus the true CH 4 methane distribution (e.g.,Bowman et al., 2006) yields:

Fig. 12 .
Fig. 12. TES CH 4 as a function of latitude after applying N 2 O correction.