Methane retrieved from TROPOMI: improvement of the data product and validation of the first 2 years of measurements

The TROPOspheric Monitoring Instrument (TROPOMI) on board the Sentinel 5 Precursor (S5-P) satellite provides methane (CH₄) measurements with high accuracy and exceptional temporal and spatial resolution and sampling. TROPOMI CH₄ measurements are highly valuable to constrain emissions inventories and for trend analysis, with strict requirements on the data quality. This study describes the improvements that we have implemented to retrieve CH₄ from TROPOMI using the RemoTeC full-physics algorithm. The updated retrieval algorithm features a constant regularization scheme of the inversion that stabilizes the retrieval and yields less scatter in the data and includes a higher resolution surface altitude database. We have tested the impact of three state-of-the-art molecular spectroscopic databases (HITRAN 2008, HITRAN 2016 and Scientific Exploitation of Operational Missions – Improved Atmospheric Spectroscopy Databases SEOM-IAS) and found that SEOM-IAS provides the best fitting results. The most relevant update in the TROPOMI XCH₄ data product is the implementation of an a posteriori correction fully independent of any reference data that is more accurate and corrects for the underestimation at low surface albedo scenes and the overestimation at high surface albedo scenes. After applying the correction, the albedo dependence is removed to a large extent in the TROPOMI versus satellite (Greenhouse gases Observing SATellite – GOSAT) and TROPOMI versus ground-based observations (Total Carbon Column Observing Network – TCCON) comparison, which is an independent verification of the correction scheme. We validate 2 years of TROPOMI CH₄ data that show the good agreement of the updated TROPOMI CH₄ with TCCON (−3.4 ± 5.6 ppb) and GOSAT (−10.3 ± 16.8 ppb) (mean bias and standard deviation). Low- and high-albedo scenes as well as snow-covered scenes are the most challenging for the CH₄ retrieval algorithm, and although the a posteriori correction accounts for most of the bias, there is a need to further investigate the underlying cause.

1 Introduction

Methane (CH₄) is the second most important anthropogenic greenhouse gas after carbon dioxide (CO₂). The global warming potential of CH₄ for a 20-year horizon is more than 80 times higher than that of CO₂ (Myhre et al., 2013) and, together with its relatively short lifetime of about 10 years, makes it an ideal short-term target for climate change mitigation strategies via reducing CH₄ emissions. CH₄ has both natural (e.g. wetlands) and anthropogenic sources (e.g. agriculture and waste together with fossil fuels), and globally 60 % of the total emissions are attributed to anthropogenic sources (Saunois et al., 2020). Natural sources are the most uncertain components of the CH₄ budget because of their magnitude and variability, which at the same time depend on characteristics that are vulnerable to changes in the state of the Earth's climate. Furthermore, the interpretation of observed CH₄ trends is hampered by the uncertainties in the estimates of CH₄ emissions (Turner et al., 2019).

Satellite observations of CH₄ are highly valuable to constrain emission inventories and for trend analysis, not only at global scale but also at regional and local scales. CH₄ measurements from satellite instruments like GOSAT (Greenhouse gases Observing SATellite) have been used to infer CH₄ emissions from natural sources (e.g. tropical wetlands Lunt et al., 2019) and anthropogenic sources (e.g. coal mining in China Miller et al., 2019) and also to map emissions and trends at global scale (e.g. Maasakkers et al., 2019). However, the spatial and temporal resolution at which these emissions can be resolved are limited by the capabilities of the instrument, preventing daily estimations or source attribution at fine scales.

A unique perspective for the long-term monitoring of CH₄ is provided by the TROPOMI (TROPOspheric Monitoring Instrument) instrument on board the Sentinel 5 Precursor (S5-P) satellite, with its daily global coverage at an unprecedented resolution of 7 × 7 km² since its launch in October 2017 (upgraded to 5.5 × 7 km² in August 2019). The high resolution together with the high signal-to-noise ratio allows for the detection and quantification of CH₄ emissions from localized (e.g. Pandey et al., 2019) to larger scale sources (e.g. Permian basin by Schneising et al., 2020; Zhang et al., 2020). Furthermore, assimilating TROPOMI CH₄ has shown great potential (e.g. in the Copernicus Atmosphere Monitoring Service (CAMS) ECMWF Integrated Forecasting System (CAMS-IFS) data assimilation system; Barré et al., 2020). The main challenge of CH₄ remote sensing is that high data quality is required for data assimilation and flux inversion applications. For TROPOMI, strict mission requirements are formulated, with a single sounding precision and accuracy both below 1 % (Veefkind et al., 2012).

TROPOMI CH₄ operational data were already proved to be of good quality by comparisons shortly after launch with both GOSAT observations (Hu et al., 2018) and ground-based measurements from the Total Carbon Column Observing Network (TCCON; Hasekamp et al., 2019). However, the CH₄ data product can now be further improved using real measurements after TROPOMI has been measuring for more than 2 years. A detailed analysis of the data provides insight into which aspects of the processing chain regarding the input data or retrieval algorithm can be further improved. The long-term record also allows us to explore possibilities of correcting biases independent of any reference data (e.g. ground-based or other satellite measurements).

In this study we present the improvements that we have developed to retrieve CH₄ from TROPOMI measurements using the full-physics approach, and we validate the TROPOMI CH₄ product with satellite and ground-based measurements. Section 2 describes the data and analysis methods that we use, and Sect. 3 focuses on the main improvements related to the regularization scheme of the inversion, the choice of the spectroscopic database for the absorption cross-sections, the surface elevation database and an a posteriori bias correction derived using only TROPOMI data. Finally, Sects. 4 and 5 show a detailed validation of the improved TROPOMI CH₄ data. The study concludes in Sect. 6, with an outlook for future steps regarding CH₄ data retrieved from TROPOMI.

2 Retrieval algorithm and validation datasets

2.1 TROPOMI CH₄ retrieval algorithm

The methane total column-averaged dry-air mole fraction (XCH₄) is retrieved from TROPOMI measurements of sunlight backscattered by Earth's surface and atmosphere in the near-infrared (NIR) and shortwave-infrared (SWIR) spectral bands with the retrieval algorithm RemoTeC. This algorithm has been extensively used to retrieve both CO₂ and CH₄ from measurements of OCO-2 and GOSAT (e.g. Wu et al., 2018; Butz et al., 2011), and it is the Sentinel 5-P and Sentinel 5 operational algorithm for CH₄ (Hasekamp et al., 2019; Landgraf et al., 2019).

The S5P RemoTeC algorithm uses the full-physics approach that simultaneously retrieves the amount of atmospheric CH₄ and the physical scattering properties of the atmosphere. The algorithm aims at inferring the state vector x that contains all the parameters to be retrieved from the radiance measurements y in the SWIR (2305–2385 nm) and NIR (757–774 nm) spectral bands, where the forward model F simulates the TROPOMI measurements,

$$\begin{array}{}\text{(1)}& \mathit{y}=\mathit{F}\left(\mathit{x}\right)+{\mathit{ϵ}}_y+{\mathit{ϵ}}_F.\end{array}$$

Here, ϵ_y and ϵ_F are the measurement noise error and the forward model error respectively. The forward model employs the LINTRAN V2.0 radiative transfer model in its scalar approximation to simulate atmospheric light scattering and absorption in a plane parallel atmosphere (Schepers et al., 2014; Landgraf et al., 2001). Accurate modelling of absorption by molecules relies on spectroscopic databases, which provide the absorption cross-section of the target absorber CH₄ as well as of the interfering gases CO, H₂O and O₂.

The inversion to estimate the state vector x requires the use of regularization methods, as measurements typically do not contain sufficient information to retrieve every state vector element independently. The RemoTeC retrieval algorithm uses the Philips–Tikhonov regularization scheme, which aims to find the state vector by solving the minimization problem

$$\begin{array}{}\text{(2)}& \widehat{\mathit{x}}=\min \left(\right|\left|{\mathbf{S}}_y^{-\mathrm{1}/\mathrm{2}}\right(\mathit{F}\left(x\right)-\mathit{y}\left)\right|{|}^{\mathrm{2}}+\mathit{\gamma }\left|\right|\mathbf{W}(\mathit{x}-{\mathit{x}}_a)\left|{|}^{\mathrm{2}}\right),\end{array}$$

where $\left|\right|\cdot \left|\right|$ describes the Euclidian norm, S_y is the measurement error covariance matrix that contains the noise estimate, γ is the regularization parameter, W is a diagonal weighting matrix that renders the side constraint dimensionless and ensures that only the target absorber CH₄ and the scattering parameters contribute to its norm (Hu et al., 2016) and x_a is the a priori state vector.

The retrieval state vector contains CH₄ partial sub-column number densities at 12 equidistant pressure layers. The total columns of the interfering non-target absorbers CO and H₂O are also retrieved, together with the effective aerosol total column, size and height parameter of the aerosol power law distribution. A Lambertian surface albedo in both the NIR and SWIR spectral range together with its first-order spectral dependence is also retrieved, as well as spectral shift and fluorescence in the NIR band.

The TROPOMI CH₄ data product is given in the form of total column-averaged dry-air mole fraction, XCH₄. It is calculated from the methane vertical sub-column elements x_i and the dry-air column V_air,dry calculated with meteorology input from ECMWF (European Centre for Medium-Range Weather Forecasts) operational analysis product and surface topography from a high-resolution database:

$$\begin{array}{}\text{(3)}& {\mathrm{XCH}}_{\mathrm{4}}=\sum _{i=\mathrm{0}}^n{\displaystyle \frac{x_i}{V_{\mathrm{air},\mathrm{dry}}}}.\end{array}$$

The precision ${\mathit{\sigma }}_{{\mathrm{XCH}}_{\mathrm{4}}}$ available in the data product is defined as the standard deviation of the retrieval noise, which follows from the error covariance matrix S_x that describes the effect of the measurement noise on the retrieval (Hu et al., 2016):

$$\begin{array}{}\text{(4)}& {\mathit{\sigma }}_{{\mathrm{XCH}}_{\mathrm{4}}}={\displaystyle \frac{\sqrt{{\sum }_{i,j=\mathrm{0}}^n{S}_{x,i,j}}}{V_{\mathrm{air},\mathrm{dry}}}}.\end{array}$$

The algorithm has been designed to provide accurate and precise retrievals for clear-sky scenes with minor scattering by aerosols and optically thin cirrus. To fulfil this criterion, a strict cloud filter is applied based on observations of the Visible Infrared Imaging Radiometer Suite (VIIRS) aboard the Suomi-NPP satellite that observes the same scene as TROPOMI approximately 5 min earlier. In fewer than 1 % of the cases when VIIRS data are not available, we use a back-up filter based on a non-scattering H₂O and CH₄ retrieval from the weak and strong absorption bands (Hu et al., 2016). These cases are flagged accordingly by the quality value indicator. Table A1 summarizes the filters applied in the retrieval process and in the TROPOMI data selection used in this study.

The CH₄ total column-averaged dry-air mole fraction retrieved from TROPOMI with the operational retrieval algorithm (version 1.2.0) largely complies with the mission requirement of precision and accuracy below 1 %, with significantly improved data quality of the bias-corrected product (Hasekamp et al., 2019). In Sect. 3 we present recent updates that further improve the quality of the data. The TROPOMI XCH₄ scientific data product from SRON retrieved with the updated algorithm serves as a beta version of the operational processing. Another scientific retrieval algorithm using the weighting function modified differential optical absorption spectroscopy (WFM-DOAS) method to retrieve CO and CH₄ from TROPOMI was presented by Schneising et al. (2019). Comparison of both retrieval approaches is foreseen as part of ongoing verification activities.

2.2 TCCON reference dataset

To validate XCH₄ retrieved from TROPOMI we use independent ground-based XCH₄ measurements from the Total Carbon Column Observing Network (TCCON) (Wunch et al., 2011a) as a reference (data version GGG2014). Table 1 contains the information of the 13 different stations located in North America, East Asia, Europe and Oceania used for the validation. In regions where there are multiple TCCON stations, we have selected those located at flat terrain in relatively remote areas, which minimizes the errors due to assumptions on the vertical CH₄ distribution used to correct for differences between the surface elevation of TROPOMI particular pixels and the ground altitude at the TCCON sites.

Kivi and Heikkinen (2016)Kivi et al. (2017)Wunch et al. (2017)Hase et al. (2017)Warneke et al. (2017)Wennberg et al. (2017a)Wennberg et al. (2017b)Wennberg et al. (2017c)Iraci et al. (2016)Kawakami et al. (2017)Griffith et al. (2017a)Griffith et al. (2017b)Sherlock et al. (2017)Pollard et al. (2019)

Table 1 Overview of the stations from TCCON used in this study.

* For the Lauder station, the ll instrument was replaced on October 2018 by the lr instrument.

To evaluate the quality of the retrieved TROPOMI XCH₄, we average TROPOMI XCH₄ data within a collocation radius around each station of 300 km. The average retrieved TROPOMI XCH₄ within the specific radius is compared with measurements of the matching TCCON station within ±2 h of the TROPOMI overpass (${\mathrm{XCH}}_{\mathrm{4},\mathrm{TROPOMI}}-{\mathrm{XCH}}_{\mathrm{4},\mathrm{TCCON}}$). For all paired collocations at each station, we compute the mean bias defined as the mean of the difference of individual collocations (ΔCH₄) and its standard deviation (σ) as a measure of the spread in the data. We then compute the average of the station biases ($\stackrel{\mathrm{‾}}{b}$) and its standard deviation ($\mathit{\sigma }\left(\stackrel{\mathrm{‾}}{b}\right)$) as a measure of the station-to-station variability. The station-to-station variability is an important diagnostic parameter as it indicates regional biases in our data, and it might be used as an overall uncertainty estimate.

2.3 GOSAT reference dataset

XCH₄ measurements by the Thermal and Near Infrared Sensor for Carbon Observation Fourier transform spectrometer (TANSO‐FTS) on board the Greenhouse gases Observing SATellite (GOSAT) satellite are used for the validation of the TROPOMI XCH₄ data. GOSAT was launched in 2009, and it performs three-point observations in a cross-track swath of 790 km with 10.5 km resolution on the ground at nadir, which results in global coverage approximately every 3 d.

We use the GOSAT proxy XCH₄ data product produced at SRON in the context of the ESA GreenHouse Gas Climate Change Initiative (GHG CCI) project (Buchwitz et al., 2019, 2017). This XCH₄ product is retrieved using the RemoTeC/proxy retrieval algorithm. The proxy approach (Frankenberg et al., 2005) infers a CO₂ and CH₄ total column from observations at 1.6 µm, ignoring any atmospheric scattering in the retrieval. Substantially, the XCH₄ product is derived by

$$\begin{array}{}\text{(5)}& {{\mathrm{XCH}}_{\mathrm{4}}}^{\mathrm{proxy}}={\displaystyle \frac{V_{{\mathrm{CH}}_{\mathrm{4}}}}{V_{{\mathrm{CO}}_{\mathrm{2}}}}}\cdot {{\mathrm{XCO}}_{\mathrm{2}}}^{\mathrm{mod}},\end{array}$$

where the column-averaged dry-air mole fraction XCO${{}_{\mathrm{2}}}^{\mathrm{mod}}$ is taken from the Carbon Tracker data assimilation system, and $V_{{\mathrm{CH}}_{\mathrm{4}}}$ and $V_{{\mathrm{CO}}_{\mathrm{2}}}$ are the vertical column densities. This approach assumes that light path modifications due to scattering in the atmosphere are the same for the target absorber (i.e. CH₄) and the proxy absorber CO₂, whose prior is assumed to be known with high accuracy.

In the validation in Sect. 5 we found that there is no bias between the GOSAT proxy and full-physics products. However, for the comparison we have selected the GOSAT proxy product over the full-physics because of its higher data yield. The proxy approach cannot be applied to retrieve XCH₄ from TROPOMI since it does not cover the 1.6 µm CH₄ and CO₂ absorption bands. Schepers et al. (2012) compared both the physics and proxy retrievals applied to GOSAT measurements to retrieve XCH₄ and concluded that both retrievals performed similarly with respect to bias variability and precision when validating the retrieved XCH₄ with ground-based TCCON measurements. This study also concluded that both methods can retrieve XCH₄ in aerosol-loaded scenes with retrieval errors of less than 1 %.

3 TROPOMI CH₄ retrieval updates

The TROPOMI XCH₄ scientific data product from SRON retrieved with the updated algorithm will be suggested for use in the operational processing in the next processor update. The updates to the S5P-RemoTeC retrieval algorithm relate to the regularization scheme, the selection of the spectroscopic database, the implementation of a higher resolution digital elevation map (DEM) for surface altitude and a more sophisticated a posteriori correction for the albedo dependence. In this section we present the updates and quantify the improvements, and we use the comparison with TCCON and GOSAT as a benchmark to test the performance of the retrieval after implementing the updates.

3.1 Regularization scheme

Hu et al. (2016) determined the regularization parameter γ in the inversion (Eq. 2) using the L-curve criterion (Hansen, 1998; Hu et al., 2016) in each iteration of the TROPOMI measurement inversion. As TROPOMI has been measuring for more than 2 years, it is possible to select a constant regularization optimized for real observations. This includes a dedicated regularization parameter for the target absorber CH₄ and one for each of the scattering parameters (effective aerosol distribution height and size parameter, and effective aerosol column). The advantage of the constant regularization is a more stable performance compared to the L-curve method in which the regularization strength changes at each iteration for every scene. The regularization parameters are selected such that the degrees of freedom for CH₄ are between 1 and 1.5 and that retrieved scattering parameters follow a distribution that we would expect.

The main improvement of the constant regularization is that the dispersion in the retrieved XCH₄ is significantly reduced. This is noticeable in the XCH₄ distribution over small regions where we do not expect large gradients of XCH₄. At regions with relatively low albedo, the decrease in the spread of the data can reach 10 %–20 % (e.g. from 18 to 14 ppb over Canada and 11 to 9 over Australia). Furthermore, the validation with TCCON shows a decrease in the station-to-station variability of 4 ppb (25 % decrease of the 15 ppb station-to-station variability using the L-curve approach) when analysing 1 year of data. The comparison with GOSAT shows that the new regularization scheme reduces the standard deviation of the difference between collocated GOSAT and TROPOMI XCH₄ observations by 9 % (from 21.5 to 19.7 ppb).

3.2 Spectroscopy database

The TROPOMI CH₄ retrieval uses external spectroscopic information to simulate the molecular absorption lines of the target absorber CH₄ as well as of CO and H₂O. The baseline retrieval algorithm employs the HITRAN 2008 spectroscopic database (Rothman et al., 2009) with updated spectroscopy parameters for H₂O from Scheepmaker et al. (2013). In preparation for the Sentinel 5 Precursor mission, Birk et al. (2017) established an improved spectroscopic database, the so-called Scientific Exploitation of Operational Missions – Improved Atmospheric Spectroscopy Databases (SEOM-IAS hereafter) for the interpretation of TROPOMI observations. The release of the HITRAN 2016 database already included some of the updates from the SEOM-IAS project regarding H₂O (Gordon et al., 2017). We have tested the effect of the three spectroscopic databases on the retrieved XCH₄ using 1 year of TROPOMI data (September 2018–September 2019).

The TCCON validation shows that after substituting HITRAN 2008 by HITRAN 2016 and SEOM-IAS for all the molecules in the CH₄ retrieval, the station-to-station variability does not change significantly (less than 1 ppb; see Table 2). The change in the mean bias shows that the different spectroscopy databases introduce an overall bias in the retrieved XCH₄ with respect to HITRAN 2008 (+20.3 ppb for HITRAN 2016 and −14.7 ppb for SEOM-IAS), but the correlation of the bias with other retrieved parameters (surface albedo, H₂O) does not improve or worsen with any of the spectroscopic database. The spectral fitting quality parameters (e.g. the root mean square of the spectral fit residuals, rms, and the corresponding χ²) show a slight improvement over TCCON stations when using the SEOM-IAS spectroscopic database, similar to what was found for the CO retrieval from TROPOMI (Borsdorff et al., 2019). The comparison with XCH₄ measured by GOSAT also shows that different spectroscopic databases introduce an overall bias, but the standard deviation of the bias does not change significantly (Table 2).

On a global scale, we see that both the rms and χ² improve significantly when using the SEOM-IAS database, with HITRAN 2008 giving the worst fitting results. Global mean χ² improves by 19 % with SEOM-IAS cross-section and by 7 % with HITRAN 2016 with respect to HITRAN 2008. Figure 1 shows the latitudinal distribution of XCH₄ retrieved with HITRAN 2008, SEOM-IAS and HITRAN 2016, referenced to the value at 0^∘ latitude. XCH₄ retrieved with HITRAN 2016 shows the least latitudinal variation at latitudes higher than 55^∘ where differences between the datasets are largest; however the global distribution does not point to a better performance of any of the spectroscopic databases. The validation with TCCON observations including Eureka (80.05^∘ N) and Lauder (45.04^∘ S) just reflects the overall bias but does not point to any latitudinal bias of XCH₄ retrieved with any of the spectroscopic databases (not shown).

Figure 1 Latitudinal distribution of TROPOMI XCH₄ retrieved using HITRAN 2008 (blue), HITRAN 2016 (pink) and SEOM-IAS (green), referenced to the value at 0^∘ latitude. Daily measurements from September 2018–September 2019 are gridded into a $\mathrm{0.2}{}^{\circ }\times \mathrm{0.2}{}^{\circ }$ grid, averaged longitudinally and then binned in 1^∘ latitude.

The results of the sensitivity tests do not point to an improved data quality when the HITRAN 2016, SEOM-IAS or HITRAN 2008 spectroscopic database is used. Each of them introduces an overall bias that cannot be used as an independent argument to favour a specific database. In view of the better spectral fitting results in the retrieved XCH₄, we have decided to use the SEOM-IAS spectroscopy database.

Table 2 Overview of the TCCON and GOSAT validation results (mean bias and its standard deviation) for the TROPOMI XCH₄ retrieved with different spectroscopic databases.

* $b=\mathrm{TROPOMI}-\mathrm{ref}$

3.3 Surface elevation

Satellite remote sensing of XCH₄ requires accurate knowledge of surface pressure and thus of surface elevation, which is specially relevant for the spatially highly resolved measurements of TROPOMI. The effect is twofold: (1) through the pressure dependence of the absorption cross-sections and (2) through the dry-air column used to calculate the dry-air mixing ratio from the retrieved column (Eq. 3).

In a first preprocessing step of the retrieval, the elevation data from a digital elevation map (DEM) are interpolated in space to the ground pixel. Then a correction is applied to the atmospheric variables (i.e. surface pressure and model pressure levels) based on the difference between the coarse resolution ECMWF altitude and the surface elevation from the DEM. To minimize errors, a filter is applied on terrain roughness, which excludes scenes with a standard deviation of the surface elevation higher than 80 m within the observed area. The default source for surface elevation information for all TROPOMI products is the Global multi-resolution terrain elevation data 2010 DEM (GMTED2010), with an aggregation radius of 5 km and a sampling of around 2 km, which results in a resolution of approximately 2 km (S5P-DEM hereafter).

The updated retrieval scheme uses the Shuttle Radar Topography Mission (SRTM) (Farr et al., 2007) digital elevation map with a resolution of 15 arcsec, approximately 400 m. To match the DEM surface elevation with the ground pixel, we perform a spatial sampling of 0.5 km and compute the mean altitude and its standard deviation for each scene. Figure 2a shows altitude differences between S5P-DEM and SRTM collocated to TROPOMI pixels (before altitude correction) on 5 May 2019 over the United States. In this specific area, 5 % of the pixels have differences in altitude greater than 45 m, with the highest differences over mountain regions. For these scenes the differences in retrieved XCH₄ are up to 7 ppb. On a yearly average (and after correction and quality filtering), 1 % of the retrievals present altitude differences greater than 50 m, which result in surface pressure differences above 5 hPa and XCH₄ differences above 10 ppb (Fig. 2b–d). The terrain roughness within TROPOMI pixels has not changed significantly with the SRTM DEM, so it does not affect the data yield due to the 80 m threshold. Although globally the average altitude difference is small, the analysis of small-scale XCH₄ enhancements will benefit from this update. Due to its higher resolution, the SRTM DEM is a better representation of elevation not only over mountains, but also close to coastlines and over rough terrain (e.g. Greenland, Sahara).

Figure 2 (a) Altitude difference between S5P-DEM and SRTM collocated to TROPOMI pixels on 5 May 2019 (orbits 8077, 8078, 8079). (b–d) Altitude, surface pressure and XCH₄ differences averaged over a year, with custom quality filtering for the TROPOMI XCH₄ retrievals, in a $\mathrm{0.2}{}^{\circ }\times \mathrm{0.2}{}^{\circ }$ grid over the United States (20–50^∘ N, 65–120^∘ W).

3.4 Posteriori correction

Greenhouse gas concentrations retrieved from satellite instruments like TROPOMI generally show systematic biases with different instrumental or geophysical parameters. Retrieved CO₂ and CH₄ from GOSAT and OCO-2 are typically corrected for dependencies on goodness of fit, surface albedo or aerosol parameters (e.g. Guerlet et al., 2013; Inoue et al., 2016; Wu et al., 2018). In the approach that O'Dell et al. (2018) derived for OCO-2 CO₂ retrievals, such parametric bias is part of a more complex correction that also accounts for footprint-level and global biases using a set of four “truth proxies” as a reference.

The comparison of TROPOMI and TCCON XCH₄ measurements shows a dependence of the bias (i.e. difference between TROPOMI and TCCON) on surface albedo retrieved in the SWIR spectral range, while for the other retrieved parameters the dependence is negligible (compared to that of the surface albedo; see Fig. 3 with solar zenith angle and aerosol optical depth as an example). Figure 3a shows that for low-albedo values, TROPOMI XCH₄ strongly underestimates TCCON measurements, while for relatively high-albedo values TROPOMI overestimates TCCON measurements. The comparison of TROPOMI XCH₄ with XCH₄ retrieved from measurements of GOSAT shows the same dependence of the bias with the retrieved surface albedo. For scenes with low-albedo values, generally the retrieval's sensitivity is low due to the large measurement noise, therefore errors from unaccounted light path modification due to scattering processes can be more significant than for scenes with a relatively higher albedo. For low-albedo scenes, this effect leads to an underestimation in the retrieved trace gas (Guerlet et al., 2013; Aben et al., 2007), resembling the TROPOMI XCH₄ underestimation in Fig. 3a.

Figure 3 Ratio of XCH₄ measurements by TCCON and TROPOMI as a function of (a) retrieved surface albedo in the SWIR spectral range, (b) retrieved effective aerosol optical depth (AOD) in the SWIR spectral range and (c) solar zenith angle (SZA).

To account for the albedo dependence, we apply an a posteriori bias correction to the retrieved XCH₄. In the baseline operational algorithm a few months after TROPOMI was operational, we applied a correction based on the comparison of TROPOMI XCH₄ with GOSAT retrievals (Hasekamp et al., 2019). After more than 2 years of measurements, we have sufficient data to derive a new correction using only TROPOMI XCH₄ measurements. We use a similar approach to the “small area approximation” applied to OCO-2 (O'Dell et al., 2018), assuming a uniform XCH₄ distribution as a function of albedo in several regions. This approach makes the correction completely independent of any reference data (e.g. GOSAT, TCCON) that could introduce additional biases when applying the correction and does not allow for an independent verification of the correction.

The new correction is derived as follows:

1. We select areas at several latitudes and longitudes throughout the globe, small enough so we can assume that XCH₄ does not vary but large enough to cover scenes with a wide range of albedo values. Figure B1 shows the different regions.

2. For each region we estimate a XCH₄ reference value for a surface albedo around 0.2, and then we calculate the ratio of the retrieved XCH₄ to the reference value to obtain the albedo dependence. The specific value for surface albedo is selected because XCH₄ retrieval errors are lower in the SWIR for that albedo range: errors because of unaccounted light path modifications due to scattering and surface albedo are minimal around a surface albedo of 0.2 (e.g. Guerlet et al., 2013; Aben et al., 2007).

3. We combine the albedo dependence for all the areas, we fit the curve using B-spline interpolation and least-squares fitting.

The B-spline method fits piece-wise polynomials that are continuous at the preselected knots. The knots and the order of the polynomials are chosen such that the residual rms of fit residuals is minimum and that the shape of the fit at the edges of the surface albedo range does not vary sharply.

Figure 4 shows the distribution of the reference to TROPOMI XCH₄ ratio for all the areas and the result of the B-spline fit. We observe two distinct features: (1) the strong underestimation for low-albedo values (already shown in the TCCON comparison in Fig. 3a), for which the B-spline fit corrects more strongly than the regular polynomial fit that was previously derived for the operational product and (2) an overestimation for high-albedo values, not captured by TCCON due to the limited albedo range values but reported in the TROPOMI and GOSAT comparisons.

Figure 4 Ratio of reference XCH₄ to TROPOMI XCH₄ as a function of the retrieved surface albedo as explained in step 3 in the derivation of the bias correction. Green dots show the average ratio in 0.05 albedo bins, and the orange line shows the B-spline fit used to derive the bias correction. Data are averaged from 1 January 2018 until 31 December 2019 in a $\mathrm{0.1}{}^{\circ }\times \mathrm{0.1}{}^{\circ }$ grid.

The correction applied to the retrieved XCH₄ can be expressed as

$$\begin{array}{}\text{(6)}& {\mathrm{XCH}}_{\mathrm{4}i}^{\mathrm{corr}}={\mathrm{XCH}}_{\mathrm{4}i}\cdot f\left(A_{\mathrm{s}i}\right).\end{array}$$

The correction function f depends on the retrieved surface albedo A_s in the SWIR spectral range at each pixel i.

Figure 5 shows the global distribution of XCH₄ before and after applying the correction. Distinctive features that correspond with low and high surface albedo areas are visible in the difference map. After correction, for example, the XCH₄ underestimation for low-albedo values (e.g. over high latitudes over Canada and Russia) is corrected. Similarly, the XCH₄ overestimation for high-albedo values over desert areas like Sahara is accounted for in the correction. The change in XCH₄ induced by the bias correction is in the range of 2 %, in agreement with the errors observed in the TCCON comparison.

Figure 5 (a) Global TROPOMI XCH₄ distribution before correction, (b) after correction and (c) their difference (${\mathrm{XCH}}_{\mathrm{4}}-{\mathrm{XCH}}_{\mathrm{4}}^{\mathrm{corr}}$) for 2019 averaged in a cylindrical equal-area grid with $\mathrm{0.3}{}^{\circ }\times \mathrm{0.5}{}^{\circ }$ resolution at the Equator.

As the correction is derived using only TROPOMI XCH₄ data, the comparison with TCCON and GOSAT is an independent verification of the approach. The validation with TCCON shows a reduction of 5.9 ppb (50 %) in the station-to-station variability and of 13.6 ppb in the bias due to the albedo correction. The dependence of the bias on surface albedo is removed (Fig. 3a vs. Fig. 6a), and the dependence on other parameters remains negligible (not shown). The comparison with GOSAT measurements shows that bias dependence on albedo is removed after applying the correction (Fig. 6b), which reduces by 4 ppb the scatter of the differences in XCH₄ measured by the two satellites. In the remainder of the paper, the corrected XCH₄ product will be used.

Figure 6 Ratio of daily XCH₄ measurements by (a) TCCON and TROPOMI and (b) GOSAT and TROPOMI as a function of retrieved surface albedo in the SWIR spectral range. Data for the period 1 December 2018–31 December 2019 are shown.

4 Comparison of TROPOMI and TCCON

4.1 TCCON validation

We perform a detailed comparison of the TROPOMI XCH₄ corrected with XCH₄ measured at 13 TCCON stations selected for the validation (Table 1). TROPOMI is able to capture the temporal XCH₄ variability, both the seasonal cycle and the year-to-year increase. This is clearly visible in the time series (e.g. Pasadena or Lamont) in Fig. 7, which shows the time series of daily average XCH₄ measured at each TCCON station and by TROPOMI for the period 1 December 2018–31 December 2019, with a collocation radius of 300 km.

Figure 7 Time series of daily averaged XCH₄ measurements from TROPOMI (red) and TCCON (blue) over the selected stations for the period 1 December 2018–31 December 2019. TROPOMI measurements around a circle of 300 km radius around each station have been selected for the comparison.

The mean bias is below 1 % for all stations; the validation results are summarized in Table 3. The average bias for all stations is −0.2 % (−3.4 ppb), and the station-to-station variability is 0.3 % (5.6 ppb), both parameters below the mission requirements for TROPOMI XCH₄ retrievals. Compared to the uncorrected TROPOMI XCH₄, the mean bias is reduced significantly (from −3.4 % to 0.2 %), even though the correction approach does not include any term to correct a global bias. As the overall negative bias is driven by the strong XCH₄ underestimation for low-albedo values (Fig. 3a), correcting for the albedo bias partly accounts for the overall bias.

Figure 8a shows the mean bias and the standard deviation for each of the stations ,and Fig. 8b shows the correlation plot. For a more strict collocation criterion of 100 km radius instead of 300 km, the number of points is reduced significantly, but the results of the validation do not change.

Figure 8 (a) Mean differences between TROPOMI and TCCON XCH₄ (ΔXCH₄), the standard deviation of the differences (${\mathit{\sigma }}_{{\mathrm{XCH}}_{\mathrm{4}}}$) and the number of collocations for each of the stations selected for the validation. (b) Correlation of daily average XCH₄ measured by TROPOMI and TCCON for all the stations.

Table 3 Overview of the validation results of TROPOMI XCH₄ with measurements from TCCON at selected stations. The table shows the number of collocations, mean bias and standard deviation for each station and the mean bias for all stations and the station-to-station variability (in ppb and in percentages between parenthesis). Results are shown for TROPOMI XCH₄ with and without the albedo bias correction applied.

4.2 High-latitude stations

Measurements at high-latitude stations such as East Trout Lake (54.36^∘ N) and Sodankylä (67.37^∘ N) show the highest variability and the highest bias in the validation before correction, which is partially reduced by the albedo correction (see validation results in Table 3). There is a seasonality in the bias which is positive during the February–April period and changes to a negative bias around May, that then increases to reach the highest (negative) values in autumn. This seasonality can be attributed to the fact that during the winter there is snow in these regions at high latitudes as a result of cold, dry air, influencing XCH₄ measurements by TROPOMI that affect the validation with TCCON measurements.

Figure 9 shows the time series of the bias between TROPOMI and TCCON XCH₄, together with the surface albedo retrieved in both the SWIR and NIR spectral range over East Trout Lake, Sodankylä and Lamont, the latter included as a mid-latitude reference. Low surface albedo in the SWIR together with high surface albedo in the NIR indicates the presence of snow, which is highly correlated with the seasonality in the bias in East Trout Lake and Sodankylä, seasonality that is more pronounced in 2019 than 2018. The seasonal bias is also correlated with high hydrogen fluoride (HF) and low H₂O concentrations (not shown). High HF concentrations are an indication of the influence of the vortex in a specific location, as HF is mostly found in the stratosphere; HF together with the contrast between surface albedo retrieved in the SWIR and NIR spectral ranges can be used as a proxy to identify the presence of snow and dry air from dynamic meteorological situations at high latitudes.

Figure 9 Daily mean relative differences (blue, left axis) between TROPOMI and TCCON XCH₄ (ΔCH₄) and surface albedo in the NIR (yellow, secondary axis, a, c, e) and surface albedo in the SWIR (yellow, secondary axis, b, d, f) at East Trout Lake (54.3^∘ N) (a, b), Sodankylä (67.4^∘ N) (c, d) and Lamont (36.6^∘ N) (e, f).

The presence of snow at high-latitude stations shifts the focus to retrieval errors as the most probable cause of the seasonal bias between TCCON and TROPOMI rather than errors due to collocation or influence of the different priors. Scenes covered by snow are characterized by low spectrum intensity in the SWIR, so signal-to-noise ratio is a limiting factor. On the other hand, the high TROPOMI signal in the NIR suggests that the weighting of each band might not be optimal in the inversion. As the optical properties are different in the NIR and SWIR bands, errors in the quantification of light path modifications over snow-covered scenes can lead to an overestimation of retrieved XCH₄. Furthermore, if H₂O may compensate for any radiometric offset in the strong CH₄ absorption bands, then in such dry conditions this would not be as effective in winter as in spring–autumn, causing the seasonality on the bias. A high bias in high latitudes correlated with H₂O columns was also found in H₂O∕HDO retrievals from TROPOMI by Schneider et al. (2020). Note that the seasonal bias is also present when XCH₄ is retrieved using the spectroscopic databases discussed in Sect. 3.2.

To filter for scenes covered with snow or ice, Wunch et al. (2011b) introduced the so-called “blended albedo”, which combines the surface albedo in the NIR and SWIR to be used as a filter. By applying it to Sodankylä and East Trout Lake, we found that a threshold value of 0.85 is optimal to remove these scenes that cause the seasonality on the bias. The influence of snow needs to be further investigated from the retrieval algorithm perspective, and it should be considered when interpreting the validation results and when analysing TROPOMI XCH₄ data over snow-covered scenes, most prevalent at high latitudes.

5 Comparison with GOSAT satellite

We compare XCH₄ retrieved from TROPOMI and GOSAT measurements for a period of 2 years (January 2018–December 2019). The comparison yields a mean bias of −10.3 ± 16.8 ppb (−0.6 ± 0.9 %) and a Pearson's correlation coefficient of 0.85. The overall comparison yields a mean bias of −12.5 ± 14.9 ppb if we use the GOSAT XCH₄ product retrieved with the full-physics approach. Figure 10 shows TROPOMI and GOSAT XCH₄ and their ratio averaged to a 2${}^{\circ }\times \mathrm{2}{}^{\circ }$ grid. Overall compared to GOSAT, TROPOMI underestimates XCH₄, specially in the regions around the tropics in South America (−0.6 ± 0.8 %) and in the African continent (−0.9 ± 0.8 %). In Asia there is higher variability (up to 1 %) compared to other regions, with areas of underestimation as well as overestimation. The overall underestimation is stronger by about 2 % in the non-corrected XCH₄, reflecting that the albedo correction improves the TROPOMI XCH₄ that is too low in areas where the surface albedo is low (e.g. forests around the Equator). For higher latitudes, the underestimation is less strong, and in some areas TROPOMI overestimates XCH₄ compared to GOSAT (e.g. Greenland and Antarctica), in agreement with the high bias in XCH₄ reported in the TCCON validation at East Trout Lake and Sodankylä.

Figure 10 Global distribution of XCH₄ measured by (a) TROPOMI and (b) GOSAT and (c) the ratio of GOSAT to TROPOMI XCH₄. Daily collocations are averaged to a 2${}^{\circ }\times \mathrm{2}{}^{\circ }$ grid for the period 1 January 2018–31 December 2019.

The latitudinal distribution of XCH₄ from TROPOMI, GOSAT and TROPOMI collocated with TCCON stations is shown in Fig. 11, summarizing the validation of TROPOMI XCH₄ and showing the good agreement between the three datasets. Similar to Fig. 10, it shows that TROPOMI underestimates GOSAT at most latitudes but both overlap within the XCH₄ variability. It also shows the shift to an overestimation at high latitudes where TROPOMI retrieves higher XCH₄. This agrees with the conclusion that over snow TROPOMI XCH₄ is too high, and although this distribution resembles the latitudinal distribution of XCH₄ shown in Fig. 1, it cannot be attributed to the selection of the spectroscopic database.

Figure 11 Latitudinal distribution of XCH₄ measured by TROPOMI and GOSAT and the TROPOMI and TCCON collocations over the selected stations for validation in Sect. 4. The shaded bands indicate the scatter (i.e. 1σ standard deviation) around the mean.

6 Conclusions

We have presented several improvements that have been implemented in the retrieval of XCH₄ from TROPOMI measurements in the NIR and SWIR spectral range. Now that TROPOMI has been measuring for more than 2 years, the amount of data allows for the implementation of a series of updates that were not previously possible without the use of any reference data (i.e. regularization scheme and an a posteriori correction derived using only TROPOMI XCH₄ data).

The regularization scheme with constant regularization parameters stabilizes the retrieval and yields less scatter in the TROPOMI XCH₄ data compared to the operational data product (version 1.2.0; Hu et al., 2016). We have investigated the effect of the horizontal resolution of the surface elevation database by replacing GMTED2010 S5P with the SRTM 15^′′ database, relevant in the XCH₄ retrieval for which accurate knowledge of surface pressure is necessary. The higher resolution database results in a more realistic representation of surface altitude, particularly for mountainous regions and places with rough surfaces, where differences in surface pressure above 5 hPa result in retrieved XCH₄ that varies up to 10 ppb for specific scenes.

We have tested three state-of-the-art spectroscopic databases (HITRAN 2008 with updates from Scheepmaker et al., 2013, HITRAN 2016 and SEOM-IAS). Using the SEOM-IAS database results in the best spectral fitting quality parameters in the retrieved XCH₄. Each of the different spectroscopic databases introduces a bias in the distribution of XCH₄ with respect to each other, but there is not any additional bias (e.g. latitudinal, albedo bias) that could point to the fitness for purpose of any of the databases. In view of the best fitting results, we decided to use the SEOM-IAS database, which was derived specifically for TROPOMI. However, there is a need for a thorough and detailed analysis of these databases, focusing on the different absorbers that are relevant in the CH₄ absorption bands to learn about the underlying processes that are driving the overall bias.

One of the most relevant updates is the implementation of an a posteriori correction that is fully independent of any reference data. We have derived a correction for the bias dependence on albedo using only TROPOMI XCH₄ data. This has been possible due to the high resolution of TROPOMI and its global coverage. We select regions around the globe which cover different albedo ranges and dependencies to estimate the albedo bias. The new correction is more accurate than the regular polynomial fit for the strong XCH₄ underestimation at low surface albedo scenes and also corrects for the positive bias in scenes with high surface albedo. After applying the correction, the albedo dependence in the TROPOMI-GOSAT and TROPOMI-TCCON comparison is removed, which is an independent verification of the correction scheme. The change in XCH₄ induced by the bias correction is in the range of 2 %, and although we attribute it mostly to unaccounted for light path modification due to scattering processes, Butz et al. (2012) predicted residual scattering errors to be mostly below 1 %, which suggests that other errors might exist, which needs to be further investigated.

The good agreement of TROPOMI XCH₄ with TCCON (−3.4 ± 5.6 ppb) and GOSAT (−10.3 ± 16.8 ppb) highlights the high quality of the TROPOMI measurements. Low- and high-albedo scenes are the most challenging for the XCH₄ retrieval algorithm, and although the a posteriori correction accounts for most of the bias, there is a need to further understand the underlying cause and whether it originates in the instrument or in the retrieval algorithm. Also the overestimation of XCH₄ over snow-covered scenes requires further investigation from the retrieval algorithm perspective. With respect to the validation, additional sites would be beneficial to cover the under-sampled regions and conditions. The network is currently limited to relatively low-albedo values, so there is a lack of reference data for high-albedo scenes, particularly around the Equator. Furthermore, there is a clear imbalance between the number of stations in the Northern and Southern Hemisphere, as well as a lack of stations below 45^∘ S. This is not only relevant for a complete validation of current and future satellite instruments, but also to have a complete global network to monitor concentrations of CH₄ in the atmosphere.

Appendix A: Filtering criteria

Table A1 Overview of the filters applied to ensure high-quality TROPOMI XCH₄ retrievals.

* Cloud fraction defined as the fraction of VIIRS pixels classified as confidently clear sky.

Appendix B: Regions selected for the a posteriori correction

The regions selected to estimate the a posteriori correction are shown in Fig. B1.

Figure B1 Black boxes over North Africa, Australia and Canada correspond to the different regions selected to estimate the a posteriori correction (see Sect. 3.4). Global distribution of XCH₄ averaged to a 0.1${}^{\circ }\times \mathrm{0.1}{}^{\circ }$ grid for the period 1 January 2018–31 December 2019.