The methane total column-averaged dry-air mole fraction (XCH4) is retrieved from TROPOMI measurements of sunlight backscattered by Earth's surface and atmosphere in the near-infrared (NIR) and shortwave-infrared (SWIR) spectral bands with the retrieval algorithm RemoTeC. This algorithm has been extensively used to retrieve both CO2 and CH4 from measurements of OCO-2 and GOSAT (e.g. ), and it is the Sentinel 5-P and Sentinel 5 operational algorithm for CH4 ().

The S5P RemoTeC algorithm uses the full-physics approach that simultaneously retrieves the amount of atmospheric CH4 and the physical scattering properties of the atmosphere. The algorithm aims at inferring the state vector x that contains all the parameters to be retrieved from the radiance measurements y in the SWIR (2305–2385 nm) and NIR (757–774 nm) spectral bands, where the forward model F simulates the TROPOMI measurements, 1y=F(x)+ϵy+ϵF.

Here, ϵy and ϵF are the measurement noise error and the forward model error respectively. The forward model employs the LINTRAN V2.0 radiative transfer model in its scalar approximation to simulate atmospheric light scattering and absorption in a plane parallel atmosphere (). Accurate modelling of absorption by molecules relies on spectroscopic databases, which provide the absorption cross-section of the target absorber CH4 as well as of the interfering gases CO, H2O and O2.

The inversion to estimate the state vector x requires the use of regularization methods, as measurements typically do not contain sufficient information to retrieve every state vector element independently. The RemoTeC retrieval algorithm uses the Philips–Tikhonov regularization scheme, which aims to find the state vector by solving the minimization problem 2x^=min(||Sy-1/2(F(x)-y)||2+γ||W(x-xa)||2), where ||⋅|| describes the Euclidian norm, Sy is the measurement error covariance matrix that contains the noise estimate, γ is the regularization parameter, W is a diagonal weighting matrix that renders the side constraint dimensionless and ensures that only the target absorber CH4 and the scattering parameters contribute to its norm and xa is the a priori state vector.

The retrieval state vector contains CH4 partial sub-column number densities at 12 equidistant pressure layers. The total columns of the interfering non-target absorbers CO and H2O are also retrieved, together with the effective aerosol total column, size and height parameter of the aerosol power law distribution. A Lambertian surface albedo in both the NIR and SWIR spectral range together with its first-order spectral dependence is also retrieved, as well as spectral shift and fluorescence in the NIR band.

The TROPOMI CH4 data product is given in the form of total column-averaged dry-air mole fraction, XCH4. It is calculated from the methane vertical sub-column elements xi and the dry-air column Vair,dry calculated with meteorology input from ECMWF (European Centre for Medium-Range Weather Forecasts) operational analysis product and surface topography from a high-resolution database: 3XCH4=∑i=0nxiVair,dry.

The precision σXCH4 available in the data product is defined as the standard deviation of the retrieval noise, which follows from the error covariance matrix Sx that describes the effect of the measurement noise on the retrieval : 4σXCH4=∑i,j=0nSx,i,jVair,dry.

The algorithm has been designed to provide accurate and precise retrievals for clear-sky scenes with minor scattering by aerosols and optically thin cirrus. To fulfil this criterion, a strict cloud filter is applied based on observations of the Visible Infrared Imaging Radiometer Suite (VIIRS) aboard the Suomi-NPP satellite that observes the same scene as TROPOMI approximately 5 min earlier. In fewer than 1 % of the cases when VIIRS data are not available, we use a back-up filter based on a non-scattering H2O and CH4 retrieval from the weak and strong absorption bands . These cases are flagged accordingly by the quality value indicator. Table  summarizes the filters applied in the retrieval process and in the TROPOMI data selection used in this study.

The CH4 total column-averaged dry-air mole fraction retrieved from TROPOMI with the operational retrieval algorithm (version 1.2.0) largely complies with the mission requirement of precision and accuracy below 1 %, with significantly improved data quality of the bias-corrected product . In Sect.  we present recent updates that further improve the quality of the data. The TROPOMI XCH4 scientific data product from SRON retrieved with the updated algorithm serves as a beta version of the operational processing. Another scientific retrieval algorithm using the weighting function modified differential optical absorption spectroscopy (WFM-DOAS) method to retrieve CO and CH4 from TROPOMI was presented by . Comparison of both retrieval approaches is foreseen as part of ongoing verification activities.

To validate XCH4 retrieved from TROPOMI we use independent ground-based XCH4 measurements from the Total Carbon Column Observing Network (TCCON) as a reference (data version GGG2014). Table  contains the information of the 13 different stations located in North America, East Asia, Europe and Oceania used for the validation. In regions where there are multiple TCCON stations, we have selected those located at flat terrain in relatively remote areas, which minimizes the errors due to assumptions on the vertical CH4 distribution used to correct for differences between the surface elevation of TROPOMI particular pixels and the ground altitude at the TCCON sites.

To evaluate the quality of the retrieved TROPOMI XCH4, we average TROPOMI XCH4 data within a collocation radius around each station of 300 km. The average retrieved TROPOMI XCH4 within the specific radius is compared with measurements of the matching TCCON station within ±2 h of the TROPOMI overpass (XCH4,TROPOMI-XCH4,TCCON). For all paired collocations at each station, we compute the mean bias defined as the mean of the difference of individual collocations (ΔCH4) and its standard deviation (σ) as a measure of the spread in the data. We then compute the average of the station biases (b‾) and its standard deviation (σ(b‾)) as a measure of the station-to-station variability. The station-to-station variability is an important diagnostic parameter as it indicates regional biases in our data, and it might be used as an overall uncertainty estimate.

XCH4 measurements by the Thermal and Near Infrared Sensor for Carbon Observation Fourier transform spectrometer (TANSO‐FTS) on board the Greenhouse gases Observing SATellite (GOSAT) satellite are used for the validation of the TROPOMI XCH4 data. GOSAT was launched in 2009, and it performs three-point observations in a cross-track swath of 790 km with 10.5 km resolution on the ground at nadir, which results in global coverage approximately every 3 d.

We use the GOSAT proxy XCH4 data product produced at SRON in the context of the ESA GreenHouse Gas Climate Change Initiative (GHG CCI) project . This XCH4 product is retrieved using the RemoTeC/proxy retrieval algorithm. The proxy approach infers a CO2 and CH4 total column from observations at 1.6 µm, ignoring any atmospheric scattering in the retrieval. Substantially, the XCH4 product is derived by 5XCH4proxy=VCH4VCO2⋅XCO2mod, where the column-averaged dry-air mole fraction XCO2mod is taken from the Carbon Tracker data assimilation system, and VCH4 and VCO2 are the vertical column densities. This approach assumes that light path modifications due to scattering in the atmosphere are the same for the target absorber (i.e. CH4) and the proxy absorber CO2, whose prior is assumed to be known with high accuracy.

In the validation in Sect.  we found that there is no bias between the GOSAT proxy and full-physics products. However, for the comparison we have selected the GOSAT proxy product over the full-physics because of its higher data yield. The proxy approach cannot be applied to retrieve XCH4 from TROPOMI since it does not cover the 1.6 µm CH4 and CO2 absorption bands. compared both the physics and proxy retrievals applied to GOSAT measurements to retrieve XCH4 and concluded that both retrievals performed similarly with respect to bias variability and precision when validating the retrieved XCH4 with ground-based TCCON measurements. This study also concluded that both methods can retrieve XCH4 in aerosol-loaded scenes with retrieval errors of less than 1 %.

determined the regularization parameter γ in the inversion (Eq. ) using the L-curve criterion () in each iteration of the TROPOMI measurement inversion. As TROPOMI has been measuring for more than 2 years, it is possible to select a constant regularization optimized for real observations. This includes a dedicated regularization parameter for the target absorber CH4 and one for each of the scattering parameters (effective aerosol distribution height and size parameter, and effective aerosol column). The advantage of the constant regularization is a more stable performance compared to the L-curve method in which the regularization strength changes at each iteration for every scene. The regularization parameters are selected such that the degrees of freedom for CH4 are between 1 and 1.5 and that retrieved scattering parameters follow a distribution that we would expect.

The main improvement of the constant regularization is that the dispersion in the retrieved XCH4 is significantly reduced. This is noticeable in the XCH4 distribution over small regions where we do not expect large gradients of XCH4. At regions with relatively low albedo, the decrease in the spread of the data can reach 10 %–20 % (e.g. from 18 to 14 ppb over Canada and 11 to 9 over Australia). Furthermore, the validation with TCCON shows a decrease in the station-to-station variability of 4 ppb (25 % decrease of the 15 ppb station-to-station variability using the L-curve approach) when analysing 1 year of data. The comparison with GOSAT shows that the new regularization scheme reduces the standard deviation of the difference between collocated GOSAT and TROPOMI XCH4 observations by 9 % (from 21.5 to 19.7 ppb).

The TROPOMI CH4 retrieval uses external spectroscopic information to simulate the molecular absorption lines of the target absorber CH4 as well as of CO and H2O. The baseline retrieval algorithm employs the HITRAN 2008 spectroscopic database with updated spectroscopy parameters for H2O from . In preparation for the Sentinel 5 Precursor mission, established an improved spectroscopic database, the so-called Scientific Exploitation of Operational Missions – Improved Atmospheric Spectroscopy Databases (SEOM-IAS hereafter) for the interpretation of TROPOMI observations. The release of the HITRAN 2016 database already included some of the updates from the SEOM-IAS project regarding H2O . We have tested the effect of the three spectroscopic databases on the retrieved XCH4 using 1 year of TROPOMI data (September 2018–September 2019).

The TCCON validation shows that after substituting HITRAN 2008 by HITRAN 2016 and SEOM-IAS for all the molecules in the CH4 retrieval, the station-to-station variability does not change significantly (less than 1 ppb; see Table ). The change in the mean bias shows that the different spectroscopy databases introduce an overall bias in the retrieved XCH4 with respect to HITRAN 2008 (+20.3 ppb for HITRAN 2016 and -14.7 ppb for SEOM-IAS), but the correlation of the bias with other retrieved parameters (surface albedo, H2O) does not improve or worsen with any of the spectroscopic database. The spectral fitting quality parameters (e.g. the root mean square of the spectral fit residuals, rms, and the corresponding χ2) show a slight improvement over TCCON stations when using the SEOM-IAS spectroscopic database, similar to what was found for the CO retrieval from TROPOMI . The comparison with XCH4 measured by GOSAT also shows that different spectroscopic databases introduce an overall bias, but the standard deviation of the bias does not change significantly (Table ).

On a global scale, we see that both the rms and χ2 improve significantly when using the SEOM-IAS database, with HITRAN 2008 giving the worst fitting results. Global mean χ2 improves by 19 % with SEOM-IAS cross-section and by 7 % with HITRAN 2016 with respect to HITRAN 2008. Figure  shows the latitudinal distribution of XCH4 retrieved with HITRAN 2008, SEOM-IAS and HITRAN 2016, referenced to the value at 0∘ latitude. XCH4 retrieved with HITRAN 2016 shows the least latitudinal variation at latitudes higher than 55∘ where differences between the datasets are largest; however the global distribution does not point to a better performance of any of the spectroscopic databases. The validation with TCCON observations including Eureka (80.05∘ N) and Lauder (45.04∘ S) just reflects the overall bias but does not point to any latitudinal bias of XCH4 retrieved with any of the spectroscopic databases (not shown).

The results of the sensitivity tests do not point to an improved data quality when the HITRAN 2016, SEOM-IAS or HITRAN 2008 spectroscopic database is used. Each of them introduces an overall bias that cannot be used as an independent argument to favour a specific database. In view of the better spectral fitting results in the retrieved XCH4, we have decided to use the SEOM-IAS spectroscopy database.

Satellite remote sensing of XCH4 requires accurate knowledge of surface pressure and thus of surface elevation, which is specially relevant for the spatially highly resolved measurements of TROPOMI. The effect is twofold: (1) through the pressure dependence of the absorption cross-sections and (2) through the dry-air column used to calculate the dry-air mixing ratio from the retrieved column (Eq. ).

In a first preprocessing step of the retrieval, the elevation data from a digital elevation map (DEM) are interpolated in space to the ground pixel. Then a correction is applied to the atmospheric variables (i.e. surface pressure and model pressure levels) based on the difference between the coarse resolution ECMWF altitude and the surface elevation from the DEM. To minimize errors, a filter is applied on terrain roughness, which excludes scenes with a standard deviation of the surface elevation higher than 80 m within the observed area. The default source for surface elevation information for all TROPOMI products is the Global multi-resolution terrain elevation data 2010 DEM (GMTED2010), with an aggregation radius of 5 km and a sampling of around 2 km, which results in a resolution of approximately 2 km (S5P-DEM hereafter).

The updated retrieval scheme uses the Shuttle Radar Topography Mission (SRTM) (Farr et al., 2007) digital elevation map with a resolution of 15 arcsec, approximately 400 m. To match the DEM surface elevation with the ground pixel, we perform a spatial sampling of 0.5 km and compute the mean altitude and its standard deviation for each scene. Figure a shows altitude differences between S5P-DEM and SRTM collocated to TROPOMI pixels (before altitude correction) on 5 May 2019 over the United States. In this specific area, 5 % of the pixels have differences in altitude greater than 45 m, with the highest differences over mountain regions. For these scenes the differences in retrieved XCH4 are up to 7 ppb. On a yearly average (and after correction and quality filtering), 1 % of the retrievals present altitude differences greater than 50 m, which result in surface pressure differences above 5 hPa and XCH4 differences above 10 ppb (Fig. b–d). The terrain roughness within TROPOMI pixels has not changed significantly with the SRTM DEM, so it does not affect the data yield due to the 80 m threshold. Although globally the average altitude difference is small, the analysis of small-scale XCH4 enhancements will benefit from this update. Due to its higher resolution, the SRTM DEM is a better representation of elevation not only over mountains, but also close to coastlines and over rough terrain (e.g. Greenland, Sahara).

Greenhouse gas concentrations retrieved from satellite instruments like TROPOMI generally show systematic biases with different instrumental or geophysical parameters. Retrieved CO2 and CH4 from GOSAT and OCO-2 are typically corrected for dependencies on goodness of fit, surface albedo or aerosol parameters (e.g. ; ; ). In the approach that derived for OCO-2 CO2 retrievals, such parametric bias is part of a more complex correction that also accounts for footprint-level and global biases using a set of four “truth proxies” as a reference.

The comparison of TROPOMI and TCCON XCH4 measurements shows a dependence of the bias (i.e. difference between TROPOMI and TCCON) on surface albedo retrieved in the SWIR spectral range, while for the other retrieved parameters the dependence is negligible (compared to that of the surface albedo; see Fig.  with solar zenith angle and aerosol optical depth as an example). Figure a shows that for low-albedo values, TROPOMI XCH4 strongly underestimates TCCON measurements, while for relatively high-albedo values TROPOMI overestimates TCCON measurements. The comparison of TROPOMI XCH4 with XCH4 retrieved from measurements of GOSAT shows the same dependence of the bias with the retrieved surface albedo. For scenes with low-albedo values, generally the retrieval's sensitivity is low due to the large measurement noise, therefore errors from unaccounted light path modification due to scattering processes can be more significant than for scenes with a relatively higher albedo. For low-albedo scenes, this effect leads to an underestimation in the retrieved trace gas (), resembling the TROPOMI XCH4 underestimation in Fig. a.

To account for the albedo dependence, we apply an a posteriori bias correction to the retrieved XCH4. In the baseline operational algorithm a few months after TROPOMI was operational, we applied a correction based on the comparison of TROPOMI XCH4 with GOSAT retrievals . After more than 2 years of measurements, we have sufficient data to derive a new correction using only TROPOMI XCH4 measurements. We use a similar approach to the “small area approximation” applied to OCO-2 , assuming a uniform XCH4 distribution as a function of albedo in several regions. This approach makes the correction completely independent of any reference data (e.g. GOSAT, TCCON) that could introduce additional biases when applying the correction and does not allow for an independent verification of the correction.

The new correction is derived as follows:

We select areas at several latitudes and longitudes throughout the globe, small enough so we can assume that XCH4 does not vary but large enough to cover scenes with a wide range of albedo values. Figure  shows the different regions.

The B-spline method fits piece-wise polynomials that are continuous at the preselected knots. The knots and the order of the polynomials are chosen such that the residual rms of fit residuals is minimum and that the shape of the fit at the edges of the surface albedo range does not vary sharply.

Figure shows the distribution of the reference to TROPOMI XCH4 ratio for all the areas and the result of the B-spline fit. We observe two distinct features: (1) the strong underestimation for low-albedo values (already shown in the TCCON comparison in Fig. a), for which the B-spline fit corrects more strongly than the regular polynomial fit that was previously derived for the operational product and (2) an overestimation for high-albedo values, not captured by TCCON due to the limited albedo range values but reported in the TROPOMI and GOSAT comparisons.

The correction applied to the retrieved XCH4 can be expressed as 6XCH4icorr=XCH4i⋅f(Asi).

The correction function f depends on the retrieved surface albedo As in the SWIR spectral range at each pixel i.

Figure shows the global distribution of XCH4 before and after applying the correction. Distinctive features that correspond with low and high surface albedo areas are visible in the difference map. After correction, for example, the XCH4 underestimation for low-albedo values (e.g. over high latitudes over Canada and Russia) is corrected. Similarly, the XCH4 overestimation for high-albedo values over desert areas like Sahara is accounted for in the correction. The change in XCH4 induced by the bias correction is in the range of 2 %, in agreement with the errors observed in the TCCON comparison.

As the correction is derived using only TROPOMI XCH4 data, the comparison with TCCON and GOSAT is an independent verification of the approach. The validation with TCCON shows a reduction of 5.9 ppb (50 %) in the station-to-station variability and of 13.6 ppb in the bias due to the albedo correction. The dependence of the bias on surface albedo is removed (Fig. a vs. Fig. a), and the dependence on other parameters remains negligible (not shown). The comparison with GOSAT measurements shows that bias dependence on albedo is removed after applying the correction (Fig. b), which reduces by 4 ppb the scatter of the differences in XCH4 measured by the two satellites. In the remainder of the paper, the corrected XCH4 product will be used.

We perform a detailed comparison of the TROPOMI XCH4 corrected with XCH4 measured at 13 TCCON stations selected for the validation (Table ). TROPOMI is able to capture the temporal XCH4 variability, both the seasonal cycle and the year-to-year increase. This is clearly visible in the time series (e.g. Pasadena or Lamont) in Fig. , which shows the time series of daily average XCH4 measured at each TCCON station and by TROPOMI for the period 1 December 2018–31 December 2019, with a collocation radius of 300 km.

The mean bias is below 1 % for all stations; the validation results are summarized in Table . The average bias for all stations is -0.2 % (-3.4 ppb), and the station-to-station variability is 0.3 % (5.6 ppb), both parameters below the mission requirements for TROPOMI XCH4 retrievals. Compared to the uncorrected TROPOMI XCH4, the mean bias is reduced significantly (from -3.4 % to 0.2 %), even though the correction approach does not include any term to correct a global bias. As the overall negative bias is driven by the strong XCH4 underestimation for low-albedo values (Fig. a), correcting for the albedo bias partly accounts for the overall bias.

Figure a shows the mean bias and the standard deviation for each of the stations ,and Fig. b shows the correlation plot. For a more strict collocation criterion of 100 km radius instead of 300 km, the number of points is reduced significantly, but the results of the validation do not change.

Measurements at high-latitude stations such as East Trout Lake (54.36∘ N) and Sodankylä (67.37∘ N) show the highest variability and the highest bias in the validation before correction, which is partially reduced by the albedo correction (see validation results in Table ). There is a seasonality in the bias which is positive during the February–April period and changes to a negative bias around May, that then increases to reach the highest (negative) values in autumn. This seasonality can be attributed to the fact that during the winter there is snow in these regions at high latitudes as a result of cold, dry air, influencing XCH4 measurements by TROPOMI that affect the validation with TCCON measurements.

Figure shows the time series of the bias between TROPOMI and TCCON XCH4, together with the surface albedo retrieved in both the SWIR and NIR spectral range over East Trout Lake, Sodankylä and Lamont, the latter included as a mid-latitude reference. Low surface albedo in the SWIR together with high surface albedo in the NIR indicates the presence of snow, which is highly correlated with the seasonality in the bias in East Trout Lake and Sodankylä, seasonality that is more pronounced in 2019 than 2018. The seasonal bias is also correlated with high hydrogen fluoride (HF) and low H2O concentrations (not shown). High HF concentrations are an indication of the influence of the vortex in a specific location, as HF is mostly found in the stratosphere; HF together with the contrast between surface albedo retrieved in the SWIR and NIR spectral ranges can be used as a proxy to identify the presence of snow and dry air from dynamic meteorological situations at high latitudes.

The presence of snow at high-latitude stations shifts the focus to retrieval errors as the most probable cause of the seasonal bias between TCCON and TROPOMI rather than errors due to collocation or influence of the different priors. Scenes covered by snow are characterized by low spectrum intensity in the SWIR, so signal-to-noise ratio is a limiting factor. On the other hand, the high TROPOMI signal in the NIR suggests that the weighting of each band might not be optimal in the inversion. As the optical properties are different in the NIR and SWIR bands, errors in the quantification of light path modifications over snow-covered scenes can lead to an overestimation of retrieved XCH4. Furthermore, if H2O may compensate for any radiometric offset in the strong CH4 absorption bands, then in such dry conditions this would not be as effective in winter as in spring–autumn, causing the seasonality on the bias. A high bias in high latitudes correlated with H2O columns was also found in H2O/HDO retrievals from TROPOMI by . Note that the seasonal bias is also present when XCH4 is retrieved using the spectroscopic databases discussed in Sect. .

To filter for scenes covered with snow or ice, introduced the so-called “blended albedo”, which combines the surface albedo in the NIR and SWIR to be used as a filter. By applying it to Sodankylä and East Trout Lake, we found that a threshold value of 0.85 is optimal to remove these scenes that cause the seasonality on the bias. The influence of snow needs to be further investigated from the retrieval algorithm perspective, and it should be considered when interpreting the validation results and when analysing TROPOMI XCH4 data over snow-covered scenes, most prevalent at high latitudes.