Real-time remote detection using AVIRIS-NG measurements is traditionally based on the MF method (Frankenberg et al., 2016). In this method, the background spectra are assumed to be distributed as a multivariate Gaussian N with covariance matrix Σ and background mean radiance μ. If H0 is a scenario without CH4 enhancement and H1 is one with CH4 enhancement, the MF approach is equivalent to a hypothesis test between the two scenarios: 1H0:Lm∼Nμ,Σ,2H1:Lm∼Nμ+tα,Σ, where Lm is the measurement radiance; t is the target signature, which is defined in Eq. (4); and α is the enhancement value, denoting a scaling factor for the target signature that perturbs the background μ. If x is a vector of measurement spectra with one element per wavelength, α(x) can be written, based on maximum likelihood estimates (Manolakis et al., 2014), as follows: 3α(x)=x-μTΣ-1ttTΣ-1t. We utilize the same definitions as in Frankenberg et al. (2016). Specifically, the enhancement value α(x) denotes the thickness and concentration within a volume of equivalent absorption and has units of parts per million × meter (ppm × m). The target signature t refers to the derivative of the change in measured radiance with respect to a change in absorption path length due to an optically thin absorbing layer of CH4. Note that this definition has the disadvantage that the accuracy of the result degrades when the absorption is strong and further attenuation becomes nonlinear. At a particular wavelength λ, t can be expressed as 4t(λ)=-κ(λ)μ(λ), where κ is the absorption coefficient for a near-surface plume with units of ppm-1 m-1. This is different from the units of m2 mol-1 traditionally used for the absorption coefficient κtrad in trace gas remote sensing. Using the ideal gas law to express the volume V (in liters) occupied by 1 mol of CH4 at the temperature and pressure corresponding to the plume altitude (V=22.4 at standard temperature and pressure), and the relations 1 L =10-3 m3 and 1 ppm =10-6, we obtain the following expression for unit conversion (units in parentheses): 5κtrad[m2mol-1]=κ[ppm-1m-1]×V[Lmol-1]×10-3[m3L-1]/10-6[ppm-1].

Figure 1 shows the target signature, which is calculated based on HITRAN absorption cross sections (Rothman et al., 2009). The background mean radiance μ used in Eq. (4) is based on the AVIRIS-NG measurement shown in Fig. 2; this is described in more detail in Sect. 3.

The OE method is widely used for the remote sensing retrieval of satellite measurements, such as from the Orbiting Carbon Observatory-2 (OCO-2; O'Dell et al., 2018), the Spinning Enhanced Visible and InfraRed Imager (SEVIRI; Merchant et al., 2013), and the Greenhouse Gases Observing Satellite (GOSAT; Yoshida et al., 2013). It combines an explicit (typically nonlinear) forward model of the atmospheric state, a (typically Gaussian) prior probability distribution for the variabilities and a (typically Gaussian) distribution for the spectral measurement errors. In addition, the Bayesian framework used by the OE approach allows new information (from measurements) to be combined with existing information (e.g., from models). In many applications, the forward model is nonlinear, and obtaining the optimal solution requires iterative techniques such as the Levenberg–Marquardt method (Rodgers, 2000), which has been routinely applied to study the impacts of measurement parameters on the retrieval process (see, e.g., Zhang et al., 2015). The iteration in this algorithm follows the procedure below. 6xi+1=xi+1+γSa-1+KiTSϵ-1Ki-1KiTSϵ-1y-Fxi-Sa-1xi-xa, where x is a state vector of surface and atmospheric properties, Sa is the a priori covariance matrix, Sϵ is the spectral radiance noise covariance matrix, K is the Jacobian matrix, xa is the a priori state vector, and γ is a parameter determining the size of each iteration step. The measured spectral radiance is denoted as y; F(x) is the simulated radiance obtained from the forward model. For the retrieval of CH4 from AVIRIS-NG measurements, the state vector includes the total column amounts of CH4 and H2O, while for the retrievals from synthetic spectra, the H2O concentration is fixed and the state vector only includes the CH4 total column. The a priori values are within 10 % of the true values; a priori errors are assumed to be 20 % for all state vector elements. The retrieved results are shown as the column-averaged mixing ratio (XCH4, ppm). Aerosols are not included in the state vector for both the real and synthetic retrievals. They are, however, considered in the forward model for the synthetic simulations. Table 1 (WCRP, 1986) lists optical properties for four basic aerosol types (dust, water soluble, oceanic, and soot). Table 2 (WCRP, 1986) shows the corresponding properties for three aerosol models that are defined as mixtures of the basic components from Table 1. We employ the Henyey–Greenstein phase function (Henyey and Greenstein, 1941), where aerosol composition is determined by two parameters: single-scattering albedo (SSA) and asymmetry parameter (g). The surface albedo is also not retrieved; for both real and synthetic retrievals, it is held fixed and assumed to be independent of wavelength.

In a real AVIRIS-NG observation, the exact column concentration of CH4 cannot be controlled. However, synthetic simulations allow us to manipulate parameters such as CH4 concentration, surface albedo, AOD, g, and SSA and thereby test aerosol impacts on CH4 retrievals. The 2S-ESS RT model is used to simulate the AVIRIS-NG spectral radiance. In this model, a prior atmospheric profile with 70 layers from the surface up to 70 km is derived from National Center for Environmental Prediction reanalysis data (Kalnay et al., 1996); absorption coefficients for all relevant gases are obtained from the HITRAN database (Rothman et al., 2009). Monochromatic RT calculations are performed at a spectral resolution of 0.5 cm-1; the radiance spectrum is then convolved using a Gaussian instrument line shape function with a wavelength-dependent full width at half maximum (FWHM) from a calibrated AVIRIS-NG data file. The signal-to-noise ratio (SNR) is set to be 300, with Gaussian white noise added. This procedure results in a wavelength grid with a resolution of about 5 nm. The spectral wavelength range used to retrieve CH4 is from 2100 to 2500 nm.

The additional atmospheric and geometric variables included in the model are listed in Table 3, which are held constant unless otherwise mentioned. The observation geometry parameters are taken from a real AVIRIS-NG measurement. Recent AVIRIS-NG flight campaigns have sensor heights ranging from 0.43 to 3.8 km; we choose a value of 1 km, the same as the highest level where aerosol is present in our simulations. The influence of AOD on CH4 retrieval as a function of SSA and g is analyzed in Sect. 4.3; in all other cases, SSA and g are held constant at 0.95 and 0.75, respectively, which is representative of aerosols in the Los Angeles region (Zhang et al., 2015).

We simulate synthetic spectra at different AOD, surface albedo, and CH4 concentration values; use the MF method to obtain the CH4 enhancement; and compare differences in α between scenarios without and with aerosol. The covariance matrix and background mean radiance are calculated from a simulated zero AOD background with surface albedos from 0.1 to 0.5 and XCH4 set at the typical background value of 1.822 ppm used in Sect. 3. Figure 5a shows the enhancement value as a function of XCH4. As the CH4 concentration increases, the enhancement value obtained by the MF method at first increases approximately linearly. However, the absorption changes in a nonlinear fashion with concentration, whereas the MF method applies a linear formalism to the change. Therefore, the enhancement value (which is correlated with the absorption signature) also shows a deviation from linear behavior at larger XCH4. Two aerosol scenarios (AOD =0, 0.3) are compared in Fig. 5a, which reveals that the effect of aerosol loading is similar to an underestimation of CH4 in the retrieval. The underestimation, which is due to the shielding of CH4 absorption below the aerosol layer and the fact that multiple-scattering (MS) effects between the aerosol and the surface are ignored, is clearly shown in Fig. 5b, where the enhancement value for fixed CH4 concentration (same concentration as the background) decreases from 0 ppm × m to -1532 ppm × m with increasing AOD. To clarify the impact of AOD at different surface albedo values, zoomed-in versions of α as a function of XCH4 are presented in Fig. 5c–f. For the AOD =0 scenario, the results are independent of surface albedo. This is because there are no MS effects between the surface and the atmosphere (Rayleigh scattering is negligible in the retrieval wavelength range) when there is no aerosol loading. For the scenarios with aerosol loading, the dispersion in the zero-enhancement XCH4 value between different surface albedos indicates that results from the MF method are biased more at large AOD and surface albedo values (Fig. 5d–f). This is a consequence of increased multiple scattering between the aerosol layer and the surface that is not accounted for by the retrieval algorithm. The maximum bias value is close to -700 ppm × m (equivalent to -0.06×1.822 ppm relative to the background concentration of 1.0×1.822 ppm) for an AOD of 0.3 and surface albedo of 0.5 (Fig. 5f). The implication of these results is that accurate knowledge of the surface albedo is important for MF retrievals, especially when the aerosol loading is large.

A quantitative analysis of underestimation of CH4 concentration due to aerosol scattering is presented in Fig. 6. The color bar shows the α bias – which is defined as the difference between the enhancement value without aerosol (true α value) and that with aerosol – for different CH4 concentrations, surface albedos, and AODs. A positive bias means that CH4 is underestimated. The α bias increases with increasing surface albedo and AOD, reaching a maximum value of about 700 ppm × m for the simulated cases. However, it is interesting that the bias decreases with increasing CH4 concentration, which is different from the results obtained by the OE method (discussed in Sect. 4.3). This surprising behavior is a direct consequence of the physical basis of the MF method. The rate of increase in enhancement becomes smaller as XCH4 becomes larger (Fig. 5a). Therefore, at higher XCH4 values, the addition of aerosols (which has a similar effect as a reduction in XCH4) results in a lower reduction in enhancement compared to that at lower XCH4 values, resulting in a net decrease in the enhancement bias.

For the simulation of the synthetic spectra, we assume nonzero aerosol loading below 1 km elevation. The OE method is then used to perform retrievals using the same configuration (including, in particular, the same surface albedo) except that AOD is set to zero. This approach is similar to neglecting aerosol scattering in the CH4 retrieval; the retrieval bias is defined as the difference between the true XCH4 in the simulation and the retrieved value (positive values refer to underestimation). First, we study the retrieval bias caused by different aerosol types and mixtures. Figure 7a shows CH4 retrieval biases as a function of SSA and g; surface albedo and AOD are kept constant at 0.3 and XCH4 is assumed to be 1.0×1.822 ppm. The retrieval bias increases with SSA and decreases with g, with a maximum bias ratio (ratio of retrieval bias to the true value) of about 20 %. This behavior can be explained as follows. At higher SSA values, there are more MS effects (that are ignored in the retrieval). On the other hand, larger values of g imply greater anisotropy of scattering (preference for forward scattering), leading to a reduction in MS effects. Since the retrieval bias is large for high SSA and low g, the water-soluble aerosol type (Table 1) and the maritime aerosol model (Table 2) can be expected to induce greater biases in the retrieval. In order to compare the impacts of SSA and g in further detail, retrieval results due to a ±5 % change in SSA and g for the three aerosol models from Table 2 are shown in Fig. 7b and c. Note that for the maritime aerosol model, the SSA is set to 0.999 for the +5 % scenario to ensure physicality. It is clear that (1) the maritime aerosol model induces larger retrieval biases than the other aerosol types, and (2) the retrieval results are more sensitive to changes in g than those in SSA.

We then simulate synthetic spectra for different values of CH4 concentration, surface albedo, and AOD. The impacts of aerosol scattering on the retrievals for these scenarios are demonstrated in Fig. 8. Figure 8a shows a 5×5 panel of boxes. Within each box, XCH4 is constant, while surface albedo increases from top to bottom and AOD increases from left to right. The variation in XCH4 across the boxes is shown in Fig. 8b. We also show a zoomed-in plot of the bottom right box (XCH4 =5.8×1.822 ppm) in Fig. 8c, which illustrates the AOD and surface albedo changes within a box. These changes are identical for all boxes. Figure 8a indicates that OE retrievals produce larger CH4 biases at higher XCH4 values, in contrast with MF results. In addition, it is evident that the retrieved CH4 bias increases with increasing AOD. The CH4 bias induced by differences in the surface albedo is not as large as that due to AOD variations, but surface albedo effects are noticeable at large AOD. Figure 8d shows the sensitivity of retrieval biases to changes in AOD and surface albedo, again demonstrating the greater impact of AOD than surface albedo in the retrieval.

The effects of changing the a priori, a priori error, and RT simulation spectral resolution on the retrieved XCH4 are shown in Fig. 9. For these calculations, the other parameters are set as follows: SSA =0.95, g=0.75, AOD =1.0, surface albedo =0.5, and true XCH4 =5.8×1.822 ppm. The parameters were chosen to correspond to the scenario with the largest retrieval bias in Fig. 8c (bottom right box in Fig. 8c). Figure 9a shows that the retrieved XCH4 changes by about 9 ppb as the a priori changes from half to twice the true XCH4 value. Similarly, the XCH4 difference is less than 4 ppb when the a priori error changes from 0.05 to 0.5 (Fig. 9b). Compared to the bias of about 923 ppb induced by neglecting aerosol scattering for this scenario, it is clear that the impacts of the a priori and a priori error are very small. The effect of spectral resolution is larger, but XCH4 still changes by only about 100 ppb when the spectral resolution is changed from 0.5 to 0.1 cm-1 (Fig. 9c).

Figure 10 presents the bias ratios for the two retrieval techniques at different AODs (surface albedo =0.3). In the MF method, the bias ratio is defined as the ratio of the bias to the true value of α. On the other hand, in the OE method, it is the ratio of the bias to the true XCH4. From Fig. 10 it is clear that the bias ratio decreases with increasing CH4 concentration and has higher values at larger AODs. The bias ratio for the MF method (1.3 %–4.5 %) is up to 53.6 % less than that for the OE method (2.8 %–5.6 %) for AOD =0.3 when the CH4 concentration is high (2–5 times typical background values). On the other hand, the OE method performs better when enhancements are small and XCH4 is close to the background value. For example, the bias ratio for the MF method has a high value of about 42.6 % at AOD =0.3 for a 10 % enhancement (XCH4 =1.1×1.822 ppm); the OE value for the same scenario is 8.6 %. For scenarios where scattering is ignored, the two retrieval techniques seem to be complementary, with differing utilities for different enhancements. On the other hand, when RT models that account for scattering effects are employed, the MF technique is suboptimal. Further, MF retrievals rely on accurate characterization of the surface albedo, especially when the aerosol loading is large. Finally, the MF method does not retrieve concentrations, which are necessary to infer fluxes. Therefore, the OE technique is in general superior due to its ability to support simultaneous retrieval of aerosols, surface albedo, and CH4 concentration.