The AIRS instrument is a nadir-viewing, scanning infrared spectrometer (Aumann et al., 2003; Pagano et al., 2003; Irion et al., 2018; DeSouza-Machado et al., 2018) that is onboard the NASA Aqua satellite and was launched in 2002. AIRS measures the thermal radiance between approximately 3–12 µm, with a resolving power of approximately 1200. For the 8 µm spectral range used for the HDO, H2O, and CH4 retrievals, the spectral resolution is ∼1 wavenumber (cm-1), with a gridding of ∼0.5 cm-1, and the signal-to-noise (SNR) ranges from ∼400 to ∼1000 over the 8 µm region for a typical tropical scene. A single footprint has a diameter of ∼15 km in the nadir; given the ∼1250 km swath, the AIRS instrument can measure nearly the whole globe in a single day. The Aqua satellite is part of the A-Train that consists of multiple satellites and instruments, including TES, in a sun-synchronous orbit at 705 km with an approximately 01:30 and 13:30 Equator crossing time. In this paper, we use only daytime data to match the validation observations.

Measurements from the HIPPO (Wofsy et al., 2012) and ATom (Wofsy et al., 2018) aircraft campaigns provide excellent datasets for satellite validation, due to their wide latitudinal coverage, the large vertical extent of the profiles (up to 9–12 km), and the availability of campaigns over a wide range of months. Each of the five HIPPO campaigns flew south then north over a period of weeks, often using a different path for the northern and southern legs, with campaign dates in 2009–2011. Atmospheric methane concentrations were measured with a quantum cascade laser spectrometer (QCLS) at 1 Hz frequency, with an accuracy of 1.0 ppb and precision of 0.5 ppb (Santoni et al., 2014). HIPPO methane data are reported on the WMO X2004 scale and have been used in several other studies to evaluate satellite retrievals of methane (e.g., Alvarado et al., 2015; Wecht et al., 2012; Crevoisier et al., 2013). Comparisons with NOAA flask data showed a mean positive bias of 0.85 ppb for the QCLS during the HIPPO campaigns, which is consistent with the estimated QCLS accuracy of 1.0 ppb (Santoni et al., 2014; Kort et al., 2011). We used 396 QCLS CH4 profiles from the HIPPO campaigns. Using coincidence criteria of ±9h and ±50km, 22 271 AIRS observations were processed, of which 5537 passed quality flags. The latitude of the matches ranges from 57∘ S to 81∘ N.

We compare AIRS to observations from the ATom aircraft campaigns 1–4 (Wofsy et al., 2018). This comparison provides validation ∼7 years after HIPPO, between 2016 and 2018. Similar to HIPPO, these observations include observations in the Pacific Ocean, but ATom also includes observations in the Atlantic (as seen in Table A1 and Fig. 1). ATom methane data are reported on the WMO X2004A scale. We used 289 profiles from the ATom campaigns from the NOAA Picarro instrument (Karion et al., 2013). For more information on the instrument, see https://espo.nasa.gov/sites/default/files/archive_docs/NOAA-Picarro_ATom1234_readme.pdf (last access: 21 December 2020). Using coincidence criteria of ±9 h and ±50 km, 21 225 AIRS observations were processed, of which 4913 passed quality flags. The latitude of the matches ranges from 65∘ S to 65∘ N.

The NOAA GML aircraft network observations (Cooperative Global Atmospheric Data Integration Project, 2019) are taken twice per month at fixed sites primarily in North America and also Rarotonga (RTA) at 21∘ S (Sweeney et al., 2015). NOAA aircraft network methane data are reported on the WMO X2004A scale. Although HIPPO data are not reported on the same scale as ATom and NOAA aircraft network data, differences in values of calibration tanks used for HIPPO (Santoni et al., 2014) on the two different scales are <1 ppb. We match AIRS and NOAA aircraft observations between 2006 and 2017, with coincidence criteria of 50 km and 9 h, finding ∼43000 matches, and 18 000 good-quality matches following the retrieval, to 719 aircraft measurements, at sites ACG (67.7∘ N, 164.6∘ E; 401 matches), ESP (49.4∘ N, 126.5∘ E; 2743 matches), NHA (43.0∘ N, 70.6∘ E; 2682 matches), THD (41.1∘ N, 124.2∘ E; 1551 matches), CMA (38.8∘ N, 74.3∘ E; 3269 matches), TGC (27.7∘ N, 96.9∘ E; 1944 matches), and RTA (21.2∘ S, 159.8∘ E; 810 matches).

Figure 1 shows the locations of all the aircraft data used for the comparisons described in this paper. Most of the ocean measurements are from the HIPPO and ATom campaigns that span a range of latitudes, whereas most of the land measurements are taken over North America.

Detailed descriptions of the use of optimal estimation (OE) to infer trace gas profiles from remote sensing radiance measurements' retrieval is included in numerous publications (e.g., Rodgers, 2000; Worden et al., 2006; Bowman et al., 2006). However, we present a partial description here as it is relevant for comparing the AIRS methane retrievals and aircraft profile measurements. As discussed in Rodgers (2000), the estimate for a trace gas profile inferred (or inverted) from a radiance spectrum is described by the following linear equation: 1x^=xa+Ax-xa+GKbberror+Gn, where x^ is the estimate of log⁡(VMR), xa is the log of the a priori concentration profile used to regularize the inversion, G is the gain matrix, and berror represents errors in systematic parameters, with Kb the sensitivity of the radiance to changes in b. We split x into [x,y], where x is the quantity of interest, the methane profile, and y denotes the jointly estimated quantities (such as temperature, water vapor, clouds, and surface properties), which results in the cross-state error (Worden et al., 2004; Connor et al., 2008). 2x^=xa+Axxx-xa+GKbberror+Axyy-ya+Gn For the AIRS (and TES) OE methane retrievals, xa comes from the MOZART atmosphere chemistry model (e.g., Brasseur et al., 1998). The vector x is the “true state”, or in this case the (log) concentration profile. The matrix A is the averaging kernel matrix or A=∂x^∂x and describes the vertical sensitivity of the measurement. Axx describes the dependence of x^ on the true state x, and Axy describes the dependence of x^ on the true state y, which is non-zero because of correlations in the Jacobians, K, for x and y. The matrix G relates changes in the radiance (L) to perturbations in x, G=∂x∂L. The vector n is the noise vector, the matrix K is the sensitivity of the radiance to changes in (log) concentration K=∂L∂x=∂L∂log⁡(VMR), and the set of vectors bi represent interference errors not estimated from the observed radiances. The true state, noise vector, and interference errors as described here are the “true” values and are therefore not actually known but are represented in this form so that we can calculate how their uncertainties affect the estimate x^. An example averaging kernel matrix is shown in Fig. 2 and shows that AIRS-based estimates of methane are most sensitive to methane in the free troposphere and lower stratosphere as demonstrated previously for AIRS and other TIR-based estimates of tropospheric methane (e.g., Xiong et al., 2016; de Lange and Landgraf, 2018).

The degrees of freedom, DOFs, describe the sensitivity of x^ to the true state and are equal to the trace of Axx. The degrees of freedom in the troposphere are equal to the trace of the averaging kernel corresponding to the troposphere, and the degrees of freedom in the stratosphere are equal to the trace of the averaging kernel corresponding to the stratosphere. The troposphere is defined using the tropopause height parameter from version 5 of the NASA Global Modeling and Assimilation Office (GMAO) Goddard Earth Observing System (GEOS-5) model (Molad et al., 2012).

Finally, we look at the quantity of interest, x^=hx. The vector h combines all the necessary operations that map the (log) concentration profiles to whatever quantity is needed such as selecting one particular pressure level (e.g., h=[0,0,0,1,0,0,0,…], selecting a column average, h=pressure weighting function – see Connor et al., 2008, or Kulawik et al., 2017) or selecting the VMR mean (e.g., h[:]=1/m, where m is the number of pressure levels to average). 3ax^=hx^x^=hxa+hAxxx-xa+hGxKbberror3b+hAxyy-ya+hGxn In Eq. (3a), the vector x^ (denoted in bold) is converted to the scalar of interest, x^ (non-bold, italic). In our validation comparisons, h is used to select (1) a specific pressure level that is measured by the aircraft, (2) the partial column XCH4 VMR within the pressure levels measured by the aircraft, and (3) the partial column XCH4 between 750 hPa and the top of the atmosphere.

A challenge in comparing the satellite-based AIRS measurements to aircraft data is that the aircraft will typically measure only a section of the atmosphere (e.g., the troposphere), whereas the AIRS measurements are sensitive, to varying degrees (see Fig. 2), to the entire atmosphere. To account for these differences, we divide the atmosphere into two parts x=[xc,xs], where xc is the part measured by the aircraft (denoted c for airCraft), and xs is the part not measured by the aircraft (denoted s for stratospheric): x^c=hcxa+hcAccxc-xac+hGcKbberror4+hcAcyy-ya+hcAcsxs-xas+hcGcn, where the term Acs is the cross term in the averaging kernel that describes the partial derivatives of the aircraft-measured levels (e.g., the troposphere) to the unmeasured levels (e.g., the stratosphere). Equation (4) describes how the AIRS measurement x^c responds to the true state [xc,xs]. So, if for example, the aircraft measured indices [0:9] and did not measure pressure levels [10:*], then Acc=A[0:9,0:9] and Acs=A[0:9,10:65], where A is the full averaging kernel.

We compare our AIRS observation, x^c in Eq. (4), to our aircraft observation, xaircraft. To compare this directly to the aircraft observation (without accounting for AIRS sensitivity), we would compare it to x^aircraftc=hcxaircraft. The expected total error includes the smoothing error, which is the covariance of the hcAccxc-xac (Rodgers, 2000), where the covariance of xc-xac is the a priori covariance, Saxx. The smoothing error is as follows: 5Smoothing error=hcAccSaxxAccThcT. We estimate the smoothing error for the partial column XCH4 VMR within the pressure levels measured by the aircraft to be 30 ppb, using Eq. (5). This estimate strongly depends on Saxx, the a priori covariance, which is the same as in Worden et al. (2012), briefly 5 % diagonal variability with correlations in pressure set from the MOZART model. In Eq. (6a), we apply the AIRS averaging kernel to the aircraft measurement to fully account for the AIRS sensitivity: x^aircraftc=hcxa+hcAccxaircraftc-xac6a+hcAcsxaircrafts-xas6bx^aircraftc=hcxa+hcAccxaircraftc-xac. One issue is that we do not actually have aircraft observations in the “s” part of the atmosphere, xaircrafts, which is used in the second term of Eq. (6a). We have aircraft observations in the “c” part of the atmosphere only, so we apply the averaging kernel to this part of the atmosphere only. Equation (6a) accounts for all of the AIRS smoothing error, whereas Eq. (6b) (the equation used in this work, other than Sect. 3.3) only accounts for the smoothing error from the part of the atmosphere measured by the aircraft profile. The difference from Eqs. (6a) and (6b) is discussed in Sect. 3.3.

Equation (7a) is the predicted bias between x^c (the measured AIRS value) and x^aircraftc (the aircraft value with the AIRS averaging kernel applied) and is the expected difference of Eqs. (4) and (6b). Equation (7b) is the covariance of Eq. (7a) and estimates the predicted error: 7aE(x^c-x^aircraftc)=hcGcKbE(berror)+hcAcyE(y^-ya)+hcAcsE(x^s-xs)+hcGcE(nerror)7bE||(x^c-x^aircraftc)||=hc(GcKbSabbKbTGcT+AcySayyAcyT+AcsSassAcsT+Smcc)hcT. Equation (7a) represents the propagation of mean biases from (1) non-retrieved parameters and assumptions, e.g., spectroscopy (b); (2) jointly retrieved parameters, e.g., temperature (y); (3) “unknown stratospheric true”, describing the impact of the part of the atmosphere not covered by the aircraft on the measured part (xs); or (4) measurement errors (n) into biases of x^c. The mean bias from Eq. (7a) is difficult to characterize theoretically and is characterized during validation. It is assumed to be primarily from the first term (e.g., spectroscopy). Equation (7b) is the covariance of the terms in 7a, where, e.g., the covariance of berror is Sabb. Equation (7b) represents the “observation covariance”. The square root of Eq. (7b) is the predicted observation error. Although Eq. (7b) has overall zero bias, it can produce regional and temporal biases, e.g., as seen in Connor et al. (2016), where these biases approach zero over long enough spatial or temporal scales. The error covariances all represent fractional errors, in log⁡(VMR). Because of the retrieved quantity log(VMR), the error in ppb is approximately the fractional error times the methane value in ppb.

For the purpose of evaluating the AIRS methane measurement uncertainties and comparing the AIRS methane to aircraft in situ measurements, we refer to the four terms on the right side of Eq. (7b) as follows:

Sbcc is the systematic error due to terms that are not accounted for in the retrieval state vector, such as spectroscopy and calibration; these terms are estimated by comparisons with the aircraft data. A pressure-dependent bias correction, described in Sect. 3.4, of approximately -60 ppb is used to correct this systematic bias.

Figure 3 shows the predicted errors for the AIRS partial column XCH4 VMR within the pressure levels measured by the aircraft. The measurement error (light green) is 18 ppb (from the last term of Eq. 7b), and the total error for a single observation (including smoothing error) is 41 ppb. A component of the total error, the cross-state error, is estimated to be 21 ppb (from Eq. 7b).

A typical aircraft profile will only measure part of the troposphere and rarely measure into the stratosphere. However, the AIRS methane profile measurements are sensitive to methane variations over the whole atmosphere, as shown by the averaging kernel matrix in Fig. 2. Similarly, the true state in the troposphere influences retrieved values in the stratosphere. Options for dealing with this are (a) extending the true profile with the AIRS prior or (b) extending the true profile with a model profile value.

This section estimates this uncertainty by calculating the difference of xaircraftc for Eq. (6a) minus Eq. (6b) when extending the aircraft using two different “true” profiles taken from two different global atmospheric chemistry models, the Laboratoire de Météorologie Dynamique (LMDz) model (e.g., Folberth et al., 2006) model and the Goddard Earth Observing System (GEOS-Chem) model (e.g., Maasakkers et al., 2019). So, if the model value equaled the AIRS prior in the stratosphere, this difference would be zero. The differences for xaircraftc from the LMDz model and GEOS-Chem are shown in Fig. 4 for all HIPPO ocean and land data; these differences show that model–model differences in the stratosphere can contribute significantly to the differences between AIRS and aircraft validation.

These differences provide an estimate for how knowledge error in the stratosphere projects to uncertainties in our methane retrievals. For example, this uncertainty varies with latitude, similar to the residual bias between the AIRS estimate and aircraft (next section). Furthermore, the variability over small latitudinal ranges of 10∘ or less suggests that the random part of the stratospheric error is smaller than this latitudinal variability. Our estimate for this error is the average of these two errors, 10 ppb, and places an upper bound on the ability to validate AIRS CH4. Our estimate for this error agrees with the 10 ppb estimate for the impact of stratospheric uncertainty on column estimates from aircraft profiles (Wunch et al., 2010). Appendix A shows further analysis of mean differences of AIRS minus aircraft for different profile extension choices. The bias varies by ∼5 ppb for different profile extension choices when comparing at 700 hPa, ∼10 ppb for different profile extension choices when comparing at 500 hPa, and ∼11 ppb for different profile extension choices when comparing the column above 750 hPa.

The methane profile has a strong variable negative vertical gradient in the stratosphere. Models in general have a positive bias in the extratropical stratosphere (Patra et al., 2011). In GEOS-Chem 4×5, the column bias is shown in Fig. 2c of Turner et al. (2015) and further discussed in Maasakkers (2019), which resolves the bias to the stratosphere, and model stratospheric accuracy is an active research area (Ostler et al., 2016; Maasakkers et al., 2019).

AIRS CH4 shows a persistent high bias of 25 to 90 ppb vs. aircraft observations in Fig. 5. Previous studies using remotely sensed measurements suggest that a bias correction to the AIRS methane profile measurement must account for the vertical sensitivity (e.g., Worden et al., 2011). For example, in the limit where the AIRS measurement is perfectly sensitive to the vertical distribution of methane, the bias correction could be a simple scaling factor. However, in the limit where the AIRS measurement is completely insensitive (e.g., DOFs=0.0), then the bias correction is zero. We therefore use the bias correction approach described in Worden et al. (2011), where a bias profile (which varies by pressure) is passed through the averaging kernel to account for the AIRS sensitivity, as seen in Eq. (8). The form of the bias profile, δbias, is set in Eq. (9).

We use HIPPO-4 observations to set a bias correction which we then evaluate with the other HIPPO campaigns and NOAA aircraft network data. HIPPO-4 was selected as it covers a wide range of latitudes and so that the bias correction can be set and tested with two independent datasets. To set the bias, we use Eq. (6b) to estimate the aircraft observation as seen by AIRS then compare this to AIRS observations. The result (by pressure level) is shown in Table 1. Then a bias was applied to AIRS using Eq. (8), with the bias term δbias in the form of Eq. (9). 8x^corrected=x^orig+Aδbias, where x^=ln⁡(VMR) because the retrieved quantity is ln⁡(VMR), δbias is a vector, and A is the averaging kernel matrix for x^=ln⁡(VMR). We fit a single bias function for all AIRS measurements by minimizing the difference between AIRS and HIPPO-4, with δbias constrained to have a slope with pressure and two pressure domains. We specify that δbias cannot jump more than 0.05 (5 %) between the two domains. δbias=c+dP(P>Po)9δbias=e+fP(P<Po) where P is pressure in hPa. The optimized bias correction parameters were c=0.0; d=-6.1×10-5; P0=400hPa; e=-0.09; f=0.00018. These bias correction results are shown for HIPPO-4; HIPPO-1, HIPPO-2, HIPPO-3, and HIPPO-5; and NOAA observations in Table 1. The remainder of the paper, unless specified, uses data bias-corrected by Eqs. (8) and (9).

Figure 6 shows the effect of bias correction on the average of all HIPPO (1, 2, 3, and 5) AIRS profiles. The bias correction improves the mean AIRS–aircraft difference and improves the pressure-dependent skew in the bias (Table 1). The HIPPO data are shown before and after the AIRS averaging kernel is applied (using Eq. 6b), which has the effect of bringing the HIPPO observations towards the AIRS prior. This is to match the imperfect sensitivity of satellite-based observations, which are similarly influenced by the prior.

Figure 5 shows a comparison between all AIRS measurements within 50 km and 9 h of an aircraft measurement and the aircraft measurement. The quantity compared is the partial column XCH4 VMR within the pressure levels measured from the aircraft. There is a mean bias of 57 ppb overall, ∼52 ppb for ocean, and ∼76 ppb for the land. The rms difference is ∼26 ppb. Furthermore, there appears to be latitudinal variations in the bias. For example, the mean difference between the AIRS and aircraft over the ocean for latitudes less than 20∘ S is ∼74 ppb, and for latitudes between 20∘ S and 20∘ N, this bias is ∼56 ppb.

Figure 7 shows the same comparisons as Fig. 5 after bias correction (described in Sect. 3.4). The mean bias is 1 ppb, and the rms difference is 24 ppb. The overall land bias is 12 ppb, and the overall ocean bias is -2 ppb. The bias calculated in Fig. 7 weights every point equally. Table A1 shows a slightly different result for these biases, where the bias is calculated by the campaign then averaged over all campaigns. In Table A1 the partial column XCH4 VMR within the pressure levels measured by the aircraft has a bias of 16 ppb for land and -2 ppb for ocean. Note that the HIPPO land observations are primarily in Australia, New Zealand, and North America, whereas the ocean comparisons are in the mid-Pacific, as seen in Fig. 1. We expect the rms difference to be similar to the observation error, as the terms that make up the observation error are the primary source of variability in the observations (e.g., Worden et al., 2017b). The predicted observation error from Fig. 3 is 27 ppb and is consistent with the rms difference seen here, 23 ppb. However, knowledge of the stratosphere and validation uncertainty is potentially a large component of the latitudinal variability in the difference seen in the bottom panel of Fig. 7.

We also compare AIRS CH4 observations to the NOAA aircraft network and ATom observations and find similar results as for HIPPO. Figure 8, discussed in Sect. 4.2, shows ATom results, and Fig. 9, discussed in Sect. 4.2, shows comparisons to a NOAA aircraft time series. The biases for different pressure ranges, campaigns, and surfaces are shown in Table A1. Table A3 shows the SD of AIRS minus validation by pressure and surface type, for single observations and daily and seasonal averages.

Satellite data are typically averaged in order to improve the precision of a comparison between data and model. However, as shown in the previous figure, these data contain errors that vary with latitude. For example, knowledge error of the true profile in the stratosphere as well as errors in the jointly retrieved AIRS temperature and water vapor retrievals have both a random and a bias component, both of which vary with latitude. The bias component is approximately the same for all AIRS methane measurements taken on the same day within 50 km, as we do not expect large variations in temperature and water vapor errors over these scales, which we presume to be a driver of these correlated errors. To quantify the component of the accuracy that cannot be reduced by averaging, we compare averages of AIRS measurements to HIPPO and ATom measurements. We average over 1 d the AIRS observations matching a single HIPPO or ATom measurement, within ±50 km and 9 h of the measurement. We specify that there needs to be at least nine AIRS observations for each comparison so that the systematic error, and not the precision (or measurement error), is the dominant term. These daily AIRS averages contain, on average, 20 AIRS observations. Figure 8 shows the predicted error, assuming that the error is random, which is calculated by dividing the single observation error (24 ppb rms shown in Fig. 7) by the square root of the number of observations that are averaged. The mean predicted error for the averaged data, assuming random errors, is 6 ppb. The actual SD between the averaged AIRS and HIPPO or ATom data is ∼17 ppb, which is much larger and indicates that the errors within 1 d and 50 km are correlated. Note that the same-colored adjacent points (i.e., adjacent observations from the same campaign) often show similar biases. Because this rms difference is much larger than what would be expected if the errors were purely random, this shows the presence of systematic errors, either in the AIRS data or in the validation uncertainty. We therefore report 17 ppb as the limiting error when averaging AIRS data within 1∘ grids and 1 d for the purpose of comparisons with models or other methane profiles.

On the other hand, averaging AIRS data seasonally can reduce the error further because geophysical errors such as temperature and water vapor vary over longer timescales. We demonstrate this aspect of the AIRS uncertainties by comparing averaged AIRS data to the NOAA aircraft methane profiles taken off the coast near Corpus Christi, Texas (27.7∘ N, 96.9∘ W, site TGC). We screen for at least three observations per day, fewer than the nine observations per day used for HIPPO and ATom daily averages in order to get enough daily averages to explore how the errors reduce with monthly and seasonal averages, since the aircraft make one–two measurements per month. Figure 9 shows daily, monthly, 90 d, and seasonal averages of the partial column XCH4 VMR within the pressure levels measured from the aircraft at TGC. The seasonal averages are created by converting all AIRS–aircraft matched pairs to 2012 by adding 5.4 ppbyr-1 multiplied by the year minus 2012 to account for the mean annual growth rate. The growth rate of 5.4 ppbyr-1 is the mean increase during the AIRS record time period (2002–2019) estimated from the NOAA Global Monitoring Laboratory global surface measurements (https://esrl.noaa.gov/gmd/ccgg/trends_ch4/, last access: 21 December 2020). Since we are converting matched pairs of aircraft and AIRS to 2012, the differences between these matched pairs are unaffected by the accuracy of the conversion to 2012.