On the consistency of methane retrievals using the Total Carbon Column Observing Network (TCCON) and multiple spectroscopic databases

The next and current generations of methane-retrieving satellite instruments are reliant on the Total Carbon Column Observing Network (TCCON) for validation. Understanding the biases inherent in TCCON and satellite methane retrievals is as important now as when TCCON started in 2004. In this study we highlight possible biases between different methane products by assessing the retrievals of the main methane isotopologue ¹²CH₄. Using the TCCON GGG2014 retrieval environment, retrievals are performed using five separate spectroscopic databases from four separate TCCON sites (namely, Ascension Island, Ny-Ålesund, Darwin and Tsukuba) over the course of a year. The spectroscopic databases include those native to TCCON, GGG2014 and GGG2020; the high-resolution transmission molecular absorption database 2016 (HITRAN2016); the Gestion et Etude des Informations Spectroscopiques Atmosphériques 2020 (GEISA2020) database; and the ESA Scientific Exploitation of Operational Missions – Improved Atmospheric Spectroscopy Databases (SEOM-IAS). We assess the biases in retrieving methane using the standard TCCON windows and the methane window used by the Sentinel-5 Precursor (S5P) TROPOspheric Ozone Monitoring Instrument (TROPOMI) for each of the different spectroscopic databases.

By assessing the retrieved ¹²CH₄ values from individual windows against the standard TCCON retrievals, we find bias values of between 0.05 and 2.5 times the retrieval noise limit. These values vary depending on the window and TCCON site, with Ascension Island showing the lowest biases (typically <0.5) and Ny-Ålesund or Tsukuba showing the largest. For the spectroscopic databases, GEISA2020 shows the largest biases, often greater than 1.5 across the TCCON sites and considered windows. The TROPOMI spectral window (4190–4340 cm⁻¹) shows the largest biases of all the spectral windows, typically >1, for all spectroscopic databases, suggesting that further improvements in spectroscopic parameters are necessary. We further assess the sensitivity of these biases to locally changing atmospheric conditions such as the solar zenith angle (SZA), water vapour and temperature. We find evidence of significant non-linear relationships between the variation in local conditions and the retrieval biases based on regression analysis. In general, each site, database and window combination indicates different degrees of sensitivity, with GEISA2020 often showing the most sensitivity for all TCCON sites. Ny-Ålesund and Tsukuba show the most sensitivity to variations in local condition, while Ascension Island indicates limited sensitivity.

Finally, we investigate the biases associated with retrieving ¹³CH₄ from each TCCON site and spectroscopic database, through the calculation of δ¹³C values. We find high levels of inconsistency, in some cases >100 ‰ between databases, suggesting more work is required to refine the spectroscopic parameters of ¹³CH₄.

1 Introduction

Methane is widely acknowledged to have a significant impact on the global climate (IPCC, 2014), but the processes via which it enters and is removed from the atmosphere are still poorly understood, with bottom-up estimations (scaled-up in situ measurements) of the global methane budget not agreeing with top-down estimations (models) (Kirschke et al., 2013; Saunois et al., 2020). This disconnect is one of many reasons that has led to the development of multiple satellite missions with the aim of improving the knowledge of the global methane budget. The remote sensing of methane is fundamentally dependent on inferring atmospheric concentrations from the absorption of light at wavelengths unique to methane, otherwise known as spectral lines. Methane, like all gases, is composed of a number of isotopologues, for example ¹²CH₄ and ¹³CH₄, forming the main constituents. The position and intensity of the spectral lines of these isotopologues are stored in large databases known as spectroscopic databases (Gordon et al., 2017). These databases are a considerable source of error in the retrieval of atmospheric methane abundances due to the uncertainty in the position and the magnitude of these spectral lines. The uncertainty is less with more abundant isotopologues (for example ¹²CH₄); however rarer isotopologues (e.g. ¹³CH₄) can have far more uncertainty. Differences in the various available spectroscopic databases could lead to significant differences between satellite estimates of methane (Galli et al., 2012; Scheepmaker et al., 2016). Understanding the spectroscopic differences in methane isotopologues is an important step towards reducing these uncertainties in future satellite measurements and further refining the databases.

The launch of the Sentinel-5 Precursor (S5P) satellite, with the TROPOspheric Monitoring Instrument (TROPOMI) instrument (Veefkind et al., 2012) and the future Sentinel 5 (S5) mission with its Ultra-violet Visible Near-infrared Shortwave-infrared (UVNS) instrument (Ingmann et al., 2012) represent a significant advancement in space-based greenhouse gas (GHG) remote sensing, building on a decade of progress from the Greenhouse Gases Observing Satellite (GOSAT) (Yoshida et al., 2013). Unlike GOSAT, TROPOMI and UVNS exploit the 4190–4340 cm⁻¹ spectral range, which has not been explored in detail from previous space-based instruments for methane retrievals. The SCanning Imaging Absorption spectroMeter for Atmospheric CHartographY (SCIAMACHY) (Bovensmann et al., 1999) on board Envisat was sensitive to this spectral range but was plagued with detector issues (ice build-up). The Measurements of Pollution in the Troposphere (MOPITT) instrument (Drummond and Mand, 1996) is also sensitive to this spectral range but is also affected by technical issues and has never successfully retrieved methane in this spectral window. The follow-on to GOSAT (GOSAT-2) also uses this spectral range; processing for GOSAT-2 is currently ongoing. In addition, the wide spectral sensitivity of the limb-viewing Canadian Atmospheric Chemistry Experiment Fourier Transform Spectrometer (ACE-FTS) (Bernath et al., 2005) includes this spectral window, but again the methane products of ACE-FTS do not include retrievals in this window. S5P–TROPOMI and S5–UVNS therefore rely on spectroscopic parameters for which only limited experience is available in their application to space-based methane retrieval instruments (Checa-Garcia et al., 2015; Galli et al., 2012). The Total Carbon Column Observing Network (TCCON), although sensitive to this spectral range, has primarily provided its methane abundances retrieved from the 6000 cm⁻¹ spectral region, allowing for direct comparisons with SCIAMACHY and GOSAT.

TCCON is a global network of 27 ground-based Fourier transform spectrometers (FTSs) (Wunch et al., 2010), with the primary aim of providing reference total column (a weighted average value for a nadir-viewing profile) abundances of numerous atmospheric species calibrated against aircraft profiles (Wunch et al., 2010, 2011), including methane, for validation and cross-calibration purposes. TCCON operates in a wide spectral range (4000–11000 cm⁻¹) and records direct solar spectra. TCCON is currently one of the key sources of reference data for the validation of satellite-based GHG retrievals, e.g. the Orbiting Carbon Observatory 2 (OCO-2), GOSAT and TROPOMI (Yoshida et al., 2011; Crisp et al., 2012; Lorente et al., 2021). TCCON instruments have both high spectral resolution (0.02 cm⁻¹), and high signal-to-noise ratios (SNRs), and insensitivity to atmospheric scattering due to direct solar-viewing geometry, thus making TCCON measurements higher quality than satellite measurements and excellent comparison datasets for satellite retrievals. TROPOMI and UVNS both have overlapping spectral windows with the wide spectral range of TCCON within the short-wave infrared (SWIR) methane absorption regions, 5970–6289 cm⁻¹ for UVNS and 4190–4340 cm⁻¹ for UVNS and TROPOMI.

When validating methane products from TROPOMI and UVNS, retrievals using the 4190–4340 cm⁻¹ window will be compared with TCCON methane products generated using the standard TCCON windows (1) 5880–5996 cm⁻¹, (2) 5996.45–6007.55 cm⁻¹ and (3) 6007–6145 cm⁻¹. Therefore potential biases associated with the choice of fit windows should be quantified and understood. Indeed, if the 4190–4340 cm⁻¹ window proves to be as accurate as the standard TCCON windows, then there is justification to integrate TCCON retrievals from this window into future TCCON retrieval products. In addition, numerous algorithms will be used to provide methane data products from TROPOMI–UVNS (Hu et al., 2016; Schneising et al., 2019), which may use differing spectroscopic databases and are therefore subject to differing biases. Building on examples of similar studies in the past (Checa-Garcia et al., 2015; Galli et al., 2012), the high SNR and high spectral resolution make TCCON data an excellent resource to assess any potential variations due to differences in the spectroscopic databases. By investigating the biases present in TCCON observations made at several sites over several seasons, we can infer some of the potential spectroscopy-related biases in satellite retrievals and their dependencies on local conditions such as water vapour that are relevant to ongoing TROPOMI and future S5–UVNS validation. Note that the spectral resolution of TCCON is typically significantly higher than that of TROPOMI and other satellite instruments, which are unlikely to be affected to the same degree as TCCON.

In addition to assessing the window and spectroscopic source biases for the main methane isotopologue ¹²CH₄, the opportunity is taken to retrieve the second-most-abundant isotopologue ¹³CH₄ and from this calculate the δ¹³C value, which is defined as

$$\begin{array}{}\text{(1)}& {\mathit{\delta }}^{\mathrm{13}}\mathrm{C}=\left({\displaystyle \frac{{\left({}^{\mathrm{13}}{\mathrm{CH}}_{\mathrm{4}}{/}^{\mathrm{12}}{\mathrm{CH}}_{\mathrm{4}}\right)}_{\mathrm{sample}}}{{(}^{\mathrm{13}}{\mathrm{CH}}_{\mathrm{4}}{/}^{\mathrm{12}}{\mathrm{CH}}_{\mathrm{4}}{)}_{\mathrm{VPDB}}}}-\mathrm{1}\right)\times \mathrm{1000}\mathrm{‰}.\end{array}$$

VPDB refers to Vienna Pee Dee Belemnite, an international reference standard for ¹³C assessment. δ¹³C requires the concentration of ¹²CH₄ and ¹³CH₄, which make up roughly 99 % and 1 % of global atmospheric methane respectively. Almost all measurements of this value are limited to in situ studies or airborne flask measurements, which although highly accurate, by their nature are spatially limited. Some effort has gone into satellite-based retrievals of this value (Buzan et al., 2016; Weidmann et al., 2017; Malina et al., 2018, 2019), but the results of these studies show this to be a challenging task. Therefore the calculation of the δ¹³C value is a target of secondary importance in this study.

δ¹³C has been used in numerous studies to differentiate methane source types (Fisher et al., 2017; Nisbet et al., 2016; Rigby et al., 2017; Rella et al., 2015), e.g. fossil fuel burning or wetlands. Tropospheric methane typically exhibits a δ¹³C value of roughly −47 ‰ (Rigby et al., 2017), and total column measurements from TCCON should not deviate from this value to a significant degree. Therefore this tropospheric δ¹³C value acts as a useful proxy to determine the stability and variability associated with retrievals of methane isotopologues from different spectral windows, spectroscopic databases, locations and times. In terms of ¹³CH₄, there are no published precision and accuracy requirements or statistics with TCCON. Calculating total column values would be highly beneficial for understanding the global methane budget but is unlikely to be achievable with TCCON with an accuracy that would be sufficient for that purpose. However, calculation of δ¹³C with TCCON will allow for an assessment of how far current technology is from making a useful total column assessment. In this study we use the TCCON GGG2014 (Toon, 2015) environment as the main tool for retrievals. Spectra are taken from four different TCCON sites in order to assess the impact of varying atmospheric conditions at different global locations. We assess the differences in abundances of the isotopologues and the quality of the fits when retrieved from standard TCCON spectral windows and methane spectral windows in the TROPOMI–UVNS spectral range. We also quantify the variations in retrieval abundances when using five separate spectroscopic databases. Building on this assessment, the sensitivity of the retrievals to variations in water vapour concentration and path length is studied. This allows for the assessment of how differing windows and spectroscopic databases are sensitive to variations in local conditions.

This paper is structured as follows: Sect. 2 outlines the methods used in this study, including details about the TCCON sites and spectra used, as well as the retrieval method. Information about the spectroscopic databases used in this study are also given. The results of this study are shown in Sect. 3, outlining the biases between sites and databases, including an assessment of the sensitivity of the retrievals to variability in local conditions. Section 4 discusses the results shown in Sect. 3, and conclusions are drawn in Sect. 5.

2 Methods, tools, datasets and requirements

2.1 TCCON sites used in study

We use TCCON spectra from four different sites identified in Table 1. Datasets over a single year were chosen in order to represent a wide range of seasonal conditions. The years chosen represent the years with maximum data coverage for each site respectively.

Table 1 TCCON sites used in this study.

The TCCON sites used in this study were picked to have a wide range of conditions, with Ny-Ålesund capturing spectra in largely unvarying conditions with high SZAs and low water vapour and Ascension Island being similar in having unvarying conditions although with higher background water vapour conditions and lower SZAs. This is contrasted by Darwin and Tsukuba, which capture spectra under a wide range of SZAs and highly variable water vapour conditions. The mean background conditions for each site, as well as the variations (standard deviations) over the dataset periods shown in Table 1, are indicated in Table 2. Significant variations in conditions and SZA are apparent between the TCCON sites, suggesting a wide range of capture conditions. We note the distributions of the conditions shown in Table 2 may not be normally distributed, but these statistics serve as a useful baseline to show the condition variations between the sites.

Table 2 TCCON site water vapour, temperature, and SZA average and variation.

2.2 GFIT retrieval algorithm

In this study we use the GGG2014 environment, which includes the GFIT retrieval algorithm (Wunch et al., 2010) summarised briefly here. GFIT employs a non-linear least-squares fitting scheme: a forward model (radiative transfer model which simulates radiation transfer through an atmosphere or a body of gas) is used to calculate synthetic irradiance spectra based on a set of parameters known as state vector elements (typically trace gas concentrations) and model parameters (e.g. temperature and pressure profiles). These synthetic irradiance spectra are then fitted to the measured irradiance spectra by adjusting the state vector elements to provide a final result, normally a trace gas abundance. In the case of GFIT, the state vector can include the following:

• first target gas scaling factor (desired output),

• interfering gas scaling factor,

• continuum level of the irradiance spectrum,

• continuum tilt,

• continuum curvature,

• frequency shift,

• zero level offset,

• solar scaling (differences in shifts of atmospheric and solar lines),

• fit channel fringes.

Note that not all of the above are routinely included in the state vector, especially the continuum curvature, which is not commonly included. This option is designed to remove instrument features but may also remove other effects due to the spectroscopic database, as noted in the TCCON wiki (TCCON, 2020). GFIT assumes a fixed profile shape for each trace gas, and the sub-column amounts for each altitude/pressure level are not independently scaled. Unlike in most satellite retrieval algorithms, aerosol and albedo terms are not included in the state vector because TCCON operates by direct solar viewing, where scattering is considered unimportant and surface terms are not necessary. The retrieved trace gas columns are calculated by multiplying scaling factors from the retrieved state vector by the a priori vertical column abundances. The TCCON a priori profiles are obtained from the National Centers for Environmental Prediction National Center for Atmospheric Research. These are adjusted with empirical models for CO₂, CO, CH₄ and N₂O developed from FTS balloon flights and data from the ACE-FTS instrument (Wunch et al., 2011). Dry-air mole fractions (DMFs) are calculated by dividing the scaled trace gas column with the total column O₂, retrieved from a wide window in the 7885 cm⁻¹ spectral region multiplied by the volume mixing ratio of O₂, 0.2095. We use O₂ from GGG2014 only to provide a point of consistency between the spectroscopic databases and because the ESA Scientific Exploitation of Operational Missions – Improved Atmospheric Spectroscopy Databases (SEOM-IAS) do not include spectral lines in this region. DMF gas volumes identify retrieved abundances as mole fractions, as opposed to absolute concentrations; all retrieved ¹²CH₄ abundances are referred to as DMF values.

Because of the high spectral resolution of the TCCON instruments (0.02 cm⁻¹), most spectral lines are resolved; therefore radiative transfer calculations are performed on a line-by-line basis. GGG includes a spectroscopic database in its environment, which is similar to other more widely adopted databases. TCCON has a standard set of spectral windows for methane retrievals, all of which are in the 6000 cm⁻¹ methane absorption window range. In this study we include the TROPOMI–UVNS SWIR spectral windows. These windows, along with a description of all of the windows considered in this study, are described in Table 3.

Table 3 Spectral windows used in study.

Windows 2–4 are standard TCCON methane retrieval windows which in this study are used for ¹²CH₄, and window 1 is based on the TROPOMI spectral window (Galli et al., 2012; Hu et al., 2016), given that no standard windows exist in this spectral window for TCCON. TCCON methane products are the result of a standardised process, where the final reported values are calculated from a weighted average of three retrieved values from windows 2, 3 and 4 as described in Table 3 (https://tccon-wiki.caltech.edu/Main/RunningGGG2020#Run_post_45Processing, last access: 12 April 2020).

For ¹³CH₄ retrievals, windows 1 and 4 are used.

2.3 Spectroscopic databases

We use parameters from five separate spectroscopic databases, which are as follows: (1) the first is the database included with GGG2014 (Toon, 2015), which currently assumes a Voigt line shape for all lines. (2) The second is the database included with the updated GGG2020 software, which includes numerous updates to the GGG2014 spectroscopic parameters and is referred to as GGG2020 in this study. Some non-Voigt parameters are included in GGG2020 but are not exploited in this study because GGG2014 was not modified to take advantage of them. (3) The third is the high-resolution transmission molecular absorption database (HITRAN), which is a well-established series of spectroscopic databases that have been used in numerous satellite-based studies previously (Galli et al., 2012). This study uses the 2016 version (Gordon et al., 2017), which builds on the prior release (HITRAN2012) with new methane lines and parameters included for both of the main isotopologues. HITRAN2016 includes the additional parameters required to model non-Voigt line shapes; however the current version does not include these parameters for methane (at the time of writing). (4) The fourth is the Gestion et Etude des Informations Spectroscopiques Atmosphériques 2020 (GEISA2020) database (Delahaye et al., 2021), another spectroscopic database, similar in design and goals to the HITRAN databases. The GEISA database does not currently include non-Voigt line shape parameters. (5) The fifth is SEOM-IAS (Birk et al., 2017), specifically developed for the TROPOMI spectral window and designed around non-Voigt atmospheric line shape profiles. These databases only have data within the 4190–4340 cm⁻¹ spectral range and can therefore only contribute to window 1 of this study.

For clarification purposes, there are no official releases of the spectroscopic parameters used in the GGG TCCON retrievals. We refer to the databases used in this study as GGG2014 and GGG2020 in order to differentiate between them, based on the GGG retrieval environment releases, with GGG2020 due for release in the near future (Laughner et al., 2021).

Some work on comparing spectroscopic databases has been performed previously (e.g. Jacquinet-Husson et al., 2016; Armante et al., 2016), generally indicating that the need to resolve differences between spectroscopic databases remains. Yet none have specifically targeted the TROPOMI SWIR spectral region; therefore this study is the first case with respect to the TROPOMI spectral window with TCCON.

Beyond exploring the impact of differing spectroscopic database parameters, we investigate the use of non-Voigt broadening parameters. Ngo et al. (2013) find the standard Voigt profiles used for spectral line broadening may be inadequate for trace gas retrievals (based on laboratory studies), which can lead to errors larger than instrument precision requirements. In order to calculate more accurate line shapes for remote sensing purposes, numerous models have been proposed. In this paper we use the quadratic speed-dependent hard-collision (qSDHC) model (Ngo et al., 2013; Tran et al., 2013). This model includes additional parameters based on speed dependence of collisional broadening and velocity changes of molecules due to collisions, on top of the standard parameters of pressure-induced air broadening and pressure-induced line shift. Note that only SEOM-IAS use these additional parameters; the remaining spectroscopic databases do not include these parameters for methane at the time of writing this paper. We use the Fortran routines provided in Ngo et al. (2013) to implement the qSDHC model into the GFIT algorithm, modified to include first-order Rosenkranz line-mixing effects. Mendonca et al. (2017) report that incorporating speed dependence and line mixing has a significant effect on calculated methane columns when compared against assuming Voigt dependency. They find a +1.1 % difference in total methane column abundances from 131 124 spectra (albeit in the 5880–6145 cm⁻¹ spectral region). The implication is that it is important to account for the additional physical parameters included in non-Voigt models when retrieving methane.

We note that the introduction of ¹³CH₄ into spectroscopic databases in the TROPOMI spectral region is relatively recent and, in the case of HITRAN, was only introduced in the 2012 release (Brown et al., 2013), thus suggesting that ¹³CH₄ spectroscopic parameters may retain high levels of uncertainty.

2.4 Analysis structure and metrics

The following section describes the assessment metrics used in this study. Firstly we assess the quality of the fit of the measured and modelled spectra for each window indicated in Table 3 for each spectroscopic database at each TCCON site. The quality of the fit is expressed through root mean square error (RMSE) of the residual between the calculated transmission spectra and the TCCON measurement transmission spectra and through the χ² test, quantitatively defined as

$$\begin{array}{}\text{(2)}& {\mathit{\chi }}^{\mathrm{2}}=\sum _i[y_{\mathrm{measured}}-y_{\mathrm{calculated}}{]}^{\mathrm{2}},\end{array}$$

where y_measured refers to the measured TCCON spectrum and y_calculated is the synthetic spectrum calculated by the forward model. Secondly, we assess the variance of the calculated DMFs of ¹²CH₄ for each window, spectroscopic database and TCCON site with respect to the standard methane product used in TCCON retrievals currently. This variance is described through the RMSE in Eq. (3):

$$\begin{array}{}\text{(3)}& \mathrm{NRMSE}=\sqrt{\sum ^n{\displaystyle \frac{{\left(\frac{{\mathrm{X}}^{\mathrm{12}}{\mathrm{CH}}_{\mathrm{4}}{}_{\mathrm{window}}-{\mathrm{X}}^{\mathrm{12}}{\mathrm{CH}}_{\mathrm{4}}{}_{\mathrm{standard}}}{{\mathit{\sigma }}_{\mathrm{standard}}}\right)}^{\mathrm{2}}}{n}}},\end{array}$$

where NRMSE refers to the normalised RMSE, X¹²CH_4 window is the retrieved DMF from a specific window, X¹²CH_4 standard is the retrieved DMF from the TCCON standard product and σ_standard is the retrieval error from the standard methane product. The variance is also given by the absolute mean residual described in Eq. (4):

$$\begin{array}{}\text{(4)}& \mathrm{NAM}=\left|\sum ^n{\displaystyle \frac{\left(\frac{{\mathrm{X}}^{\mathrm{12}}{\mathrm{CH}}_{\mathrm{4}\mathrm{window}}-{\mathrm{X}}^{\mathrm{12}}{\mathrm{CH}}_{\mathrm{4}\mathrm{standard}}}{{\mathit{\sigma }}_{\mathrm{standard}}}\right)}{n}}\right|,\end{array}$$

where NAM is the normalised absolute mean residual and all other terms are as identified previously. Variations in the retrieval conditions throughout the course of a day of measurements are included in TCCON error budgets; for example artefacts can appear in TCCON retrievals at extreme SZA values (Wunch et al., 2011). We therefore investigate if the methane retrieval biases vary with respect to the following local parameters: (1) the first parameter is the SZA, where extreme angles can cause errors in the air-mass assumptions and affect characteristics of the instrument line shape function (ILS) (Wunch et al., 2011). (2) The second parameter is water vapour (retrieved by TCCON in two standard narrow windows) through the whole available data range at the respective TCCON sites. The GFIT retrieval algorithm is a scaling retrieval algorithm, meaning that an incorrect a priori trace gas profile shape will yield errors in the retrieval. The GFIT water vapour a priori is based on a profile taken at midday (local time) for each specific retrieval, meaning that any significant variations from this daily profile will yield errors in the retrieval that will vary depending on the impact of water vapour on a specific spectral window. (3) The third parameter is temperature, which is not included in the retrieval state vector, and dependencies on temperature will not be removed in the retrieval process. Temperature errors are introduced through the spectroscopic line strengths (An et al., 2011); therefore poor knowledge of spectroscopic parameters will potentially lead to temperature-based errors.

These dependencies are quantified by non-linear regression analysis, consisting of fitting the variations in water vapour, SZA and measured temperature against the normalised differences between each methane isotopologue DMF case and the DMFs from the standard TCCON methane retrieval window. Here the normalisation factor is the uncertainty from the standard TCCON methane retrieval.

The magnitude of the metrics defined above can be put into context by comparisons with the TCCON error budget. TCCON typically aims for precision of <0.3 % on methane retrievals and has a rough estimate of 1 % systematic uncertainties (dominated by in situ calibration, which can affect sites differently; Wunch et al., 2015). Therefore it is possible to judge the variation in ¹²CH₄ DMFs between windows and databases based on these bias and precision values.

Finally, although the quality of the ¹³CH₄ fit metrics in this study is not covered in detail, we calculate δ¹³C in order to understand the plausibility of and variation in retrieving ¹³CH₄ from TCCON. The final aim of retrieving ¹³CH₄ is to calculate δ¹³C. How much δ¹³C varies in the total column is a complex issue (Weidmann et al., 2017; Malina et al., 2018, 2019). In situ studies (Nisbet et al., 2016; Rigby et al., 2017; Fisher et al., 2017) all show that an uncertainty of ≪1 ‰ in δ¹³C is required in order to determine natural annual variability at the surface. However, variability in δ¹³C can be higher in the troposphere and stratosphere due to variability in the OH sink and the fractionation caused by OH (Röckmann et al., 2011; Buzan et al., 2016), with evidence that δ¹³C can vary by up to 10 ‰ in different air parcels (Röckmann et al., 2011). Based on these factors, we assume a rough total column δ¹³C variability of 1 ‰, which equates to a total uncertainty of <0.02 ppb in ¹³CH₄ retrievals, or roughly 0.1 % of the total column, calculated using Eq. (1) (Malina et al., 2018). This is clearly an unrealistic target for individual retrievals given the uncertainty requirements for ¹²CH₄ described above. However, TCCON currently represents the best chance of remotely measuring δ¹³C since precision errors are low and the SNR is high (Wunch et al., 2011). In addition, because TCCON sites are situated in fixed positions, long-term averaging is possible, which further reduces precision-based errors. Therefore one of the minor aims of this study is to identify how far away TCCON uncertainty (including systematic errors) is from the desired uncertainty of <1 ‰ δ¹³C.

3 Results

3.1 Quality of spectral fitting

An example of residual transmission spectra from the Ny-Ålesund site is shown in Fig. 1, with the standard deviation of a selection of retrievals within the same time period indicated by the red lines. Examples of spectral fits from the other TCCON sites considered in the study are shown in the Appendix. Qualitatively we note clear differences in the quality of the fits between windows and databases; for example there are clear deviations apparent, especially in window 1 for HITRAN and GEISA.

Figure 1 Example residual transmission spectra calculated from measured and fitted spectra from the Ny-Ålesund in 2019. The blue line indicates an example of the fit residual between the calculated transmission and the measured transmission. The red line indicates the standard deviation of the residual, based on all spectra taken over the entire dataset. The columns of this figure identify the residuals of a specific window and the rows a specific database, as identified in the axis labels.

The analysis statistics for the residual transmission spectra (as discussed in Sect. 2.4) shown in Fig. 1 are presented in Fig. 2, as well as the associated statistics for the other TCCON sites considered in this study. What is clear from Fig. 2 is that the fit statistics for each spectroscopic database, irrespective of the TCCON site and window, generally have the same pattern in terms of quality. For window 1 SEOM-IAS and GGG2020 are more or less equivalent in quality, followed by GGG2014, HITRAN and then GEISA. In windows 2–4 where SEOM-IAS have no data, GGG2020 typically shows the highest-quality fits, suggesting the latest iteration of the GGG2020 spectroscopic parameters has superior performance to the older version.

Window 1 typically shows the poorest fit metrics of all windows, possibly because it is the largest window but also because it covers a more complex region in terms of absorption (Brown et al., 2013) than the other windows. Window 4 for example is also wide but typically shows higher-quality fits than any of the other windows in this study, the implication being that the knowledge of spectroscopic parameters in window 1 is still lacking in comparison to the traditional TCCON windows.

There are differences in the metrics between TCCON sites, with Ascension Island showing poorer RMSE values than any of the other sites, similarly to Darwin. This is to be expected however since these instruments are not identical and capture spectra under differing conditions. We note that all instruments are run according to TCCON specifications but their respective configurations are not exactly the same. This is normal and necessary as different sites need local adjustments to account for different local conditions such as altitude, humidity or cloud conditions. Most of the effects caused by such individual configurations are removed by the differential CO₂ and CH₄ DMF retrievals but will affect individual spectra. For example, in the case of Tsukuba and Ascension, the configuration effects cannot be compared directly except for detector noise, which turned out to be comparable. However, the signal on the detector of the Ascension Island instrument is at least 50 % lower than that of the Tsukuba instrument. Likely reasons for this difference are as follows: (1) the Ascension FTS runs on a higher spectral resolution (0.014 vs. 0.02 cm⁻¹) and a faster scanner speed (10 vs. 7.5 kHz). Both reduce integration time per spectral pixel. (2) The illumination of the InGaAs detector on Ascension is kept low on purpose to avoid saturation. This setting cannot be readjusted in between site visits and has to last for months. Other sites may use similar techniques and may vary depending upon need. (3) The solar tracker has known issues with pointing at the centre of the sun at low SZAs but cannot be replaced easily. In addition, dust build-up on the solar tracker mirrors reduces the reflectively of the mirrors quickly. They are cleaned weekly, but a signal loss on the order of 20 % over a few days is not uncommon.

The results from Tsukuba are different from the other showcased results. This is likely because of the smaller number of spectra available for plotting (owing to data transfer and storage limits). There was no limit in the actual retrievals identified in the following paper sections.

Example transmission spectra for Darwin, Tsukuba and Ascension Island are identified in Appendix A.

Figure 2 Bar charts indicating the fit statistics for a selection of retrievals from each of the TCCON sites. Each row of the figure refers to results from one of the TCCON sites, indicated by the row title. Each column shows the results from one window, indicated by the title of each column. Each subplot shows the RMSE and χ² values for each spectroscopic database indicated on the x axis, with the blue bars referring to the RMSE values, with magnitudes shown on the left-hand y axis. The black bars refer to the χ² values, with the magnitudes indicated on the right-hand y axis. Note the scale on the Tsukuba row is slightly different to account for the lower-magnitude results. Cross sections of 500 spectra for each TCCON site are used to generate the statistics in this figure.

Since all trace gases are fitted simultaneously in all of the windows, there are no specific metrics associated with ¹³CH₄. The ¹³CH₄ in this study is fitted in windows 1 and 4.

Building on the residuals indicated in Fig. 1, we investigate the differences observed in transmission residuals. We calculate the percentage difference for each spectroscopic database, with respect to the transmissions calculated by the GGG2014 database, for ¹²CH₄ and H₂O species.

Figure 3 Percent difference in calculated transmission for ¹²CH₄ and H₂O from each spectroscopic database for each window with respect to GGG2014. Rows indicate the spectroscopic database, and columns indicate the windows. The y-axis values have been limited to 20 % to avoid the plots being dominated by large excessive-noise values; note however, especially in window 1, the values are sometimes in excess of 20 %.

The results shown in Fig. 3 suggest a number of different conclusions. The impact of differences in the spectroscopic parameters of water vapour are highly significant in window 1, more so than differences in ¹²CH₄, with each spectroscopic database showing differences >20 % at numerous wavelengths. Each spectroscopic database shows significant disagreement as to where the differences occur, suggesting large differences in the treatment of water vapour parameters in window 1 for each spectroscopic database. Therefore the poorer fit quality shown in Fig. 1 for window 1 is likely driven by water vapour uncertainty as opposed to methane. For window 2, we note again that water vapour seems to have the largest uncertainties (not to the degree of window 1). The main points of disagreement in general line up with the largest deviations in Fig. 1. For windows 3 and 4, water vapour differences have less impact with the majority of the differences attributed to ¹²CH₄, which still have a lower magnitude than window 1. The large uncertainty in window 3 at 6001 cm⁻¹ (characteristic bump shape) seems to be caused by water vapour uncertainty in HITRAN and GEISA2020. The main conclusion from this assessment is that when considering uncertainty in ¹²CH₄ between windows and spectroscopic databases, uncertainty in water vapour should be considered at the same time.

3.2 Quantification of variance between windows and databases

All the time series available for this study for each TCCON site are shown in Figs. 4, 5, 6 and 7. XAir, a quantity normally retrieved with TCCON, is shown as an additional quality indicator for GGG2014, GGG2020, HITRAN and GEISA2020 (SEOM-IAS do not have spectral coverage in the TCCON XAir retrieval window) with variations between 0.96 and 1.04 assumed as good quality. Qualitative inspection of these figures shows scatter between all windows for each database; further the HITRAN, GEISA and SEOM-IAS databases show significant positive bias with respect to the standard deviation of the reference TCCON retrieval, indicated by the dashed black lines. Quantitative metrics for these figures are shown in Fig. 8. We also find the retrieved XAir values indicate good-quality retrievals, with only a small number falling outside the acceptable range.

Figure 4 Retrieval time series for ¹²CH₄ DMFs from the Ny-Ålesund site. Each panel indicates retrievals from each spectral window (indicated in the legend) from a specific spectroscopic database, indicated in the panel title. Blue stars show retrievals from band 1; yellow pluses are band 2; green triangles are band 3; red circles are band 4. The standard deviations about the reference TCCON retrievals are indicated by the horizontal dashed lines. The bottom panel indicates the retrieved XAir DMF as a quality indicator for the retrievals, with the dashed lines indicating the standard range of acceptable XAir values. The date format is year-month.

Figure 5 As Fig. 4 but for retrievals from the Darwin TCCON.

Figure 6 As Fig. 4 but for retrievals from the Tsukuba TCCON.

Figure 7 As Fig. 4 but for retrievals from the Ascension Island TCCON.

The metrics used in Fig. 8 indicate the bias for ¹²CH₄ retrievals for each database and window with respect to the reference retrieval (NAM) and the presence of any large deviations (RMSE). These metrics are normalised by the retrieval uncertainty in the reference retrievals; thus we assume any biases with values greater than 1 cannot be attributed to uncertainty and are therefore real.

Firstly we analyse the results from Ny-Ålesund, which due to the constant nature of the atmospheric conditions can be considered a baseline. For window 1, both the NRMSE and the NAM values for all of the databases indicate values greater than 1, thus suggesting there are still significant variations in the treatment of spectroscopic parameters in window 1. The HITRAN and GEISA databases show bias deviations twice that of GGG2014; however these values do not indicate any one database is more accurate than the other but either large differences in spectroscopic parameters or differences in sensitivity to local conditions. Windows 2 and 3 do not show any notable biases apart from the GEISA database, which generally shows the largest deviations across all of the windows (except window 1). In window 4, both HITRAN and GEISA show notable deviation from the reference retrievals, which is a surprising result given this window is popular in satellite retrievals of methane (Yoshida et al., 2013). We note the NRMSE and NAM values are similar in the majority of cases, indicating that there is an underlying bias between the database retrievals as opposed to large spikes in differences. Considering the bias deviations across the windows, GEISA is the only example to exceed values of 1 across all windows, with window 3 showing the largest deviation from the reference value.

Secondly considering the dataset from Darwin, the magnitudes of the NRMSE and NAM values are typically lower than the equivalents in the Ny-Ålesund dataset. The relative differences between the NRMSE and NAM values between the databases are the same as those shown in the Ny-Ålesund dataset; i.e. GGG2014 shows the lowest differences and GEISA shows the largest, apart from window 1, in which case it is HITRAN. Investigating each window in turn, only HITRAN shows a notable deviation from the standard retrieval in window 1 with GGG2020 and GEISA not indicating a significant deviation above the standard noise level (only 0.02 and 0.07 above 1 respectively). For windows 2 and 3, only the GEISA database shows a significant bias with respect to the standard, as with the Ny-Ålesund site. Again in window 4 only the HITRAN and GEISA databases show notable deviation from the standard, again suggesting the spectroscopic parameters in window 4 still have significant uncertainty with respect to windows 2 and 3. The implication of the Darwin results with respect to those from Ny-Ålesund is that either one or both of the differences in the instrument setup and the local conditions impact inter-window and/or spectroscopic database biases. Note for the Darwin retrievals there is an inconsistency at the beginning of March 2020, when there is a small “bump” in the magnitudes. This effect only appears at the start of March, and the typical retrieval variability is quickly restored. Quality control indicators (such as XAir) do not indicate any problems with the results in this period, and the reason for this inconsistency remains unclear.

The results for the Tsukuba retrievals are very similar to those shown for Ny-Ålesund, with GGG2014 not showing any significant differences except in window 1; as with the other sites HITRAN shows deviation in windows 1 and 4, and GEISA shows the largest differences apart from in window 1. However, the main difference is with the GGG2020 database: as with the other TCCON sites the NAM shows deviation in windows 1 and 4. The NRMSE indicates significant differences in all windows, suggesting there are a small number of retrieval cases that have large biases with respect to the standard values. This behaviour is not replicated in the other TCCON sites.

Finally, all results from the Ascension Island measurements indicate no deviations of any significance, contrasting with the results from all other sites. We note the standard deviation about the reference TCCON retrievals in Fig. 7 is smaller than for any of the other TCCON sites. This suggests constant retrievals in methane over the course of the year at Ascension Island and therefore limited opportunity for biases to form.

The results in Fig. 8 clearly indicate that in the cases where deviations exist, they are reflected in all of the TCCON sites (when significant), implying that despite the fit differences shown in Fig. 2, these biases cannot (purely) be attributed to errors in the TCCON instruments, but given the consistency of the deviations we can attribute these differences to spectroscopic parameters. Figure 8 indicates that there are significant differences between SEOM-IAS, GEISA and HITRAN databases with respect to the GGG databases, which shows less deviation. This is not surprising since the reference values are based on GGG2014, and GGG2020 is built upon GGG2014; however this is not the case in window 1, where larger deviations are observed. This suggests that knowledge of spectroscopic parameters in window 1 is not as good as in the other windows which have been routinely used in TCCON. It is difficult to assess all of the differences between the databases due to the range of parameters used; there are some papers which describe the sources of the spectral lines for each of the databases (Brown et al., 2013; Jacquinet-Husson et al., 2016), but specifics are limited due to the size of the databases. Complexity is added by the fact that several of these databases state that data are drawn from the same sources (Albert et al., 2009; Nikitin et al., 2015, 2017). However, these papers go on to say that not all of the lines from these studies are implemented based on in-house assessments of fit quality. This implies that it is challenging to specifically identify where spectroscopic parameter differences occur between the databases.

Figure 8 Bar plots indicating NRMSE and normalised absolute mean residual difference values for ¹²CH₄ retrievals from each TCCON site, each window and each spectroscopic database under consideration in this study with respect to the original TCCON methane retrieval window. Each row shows data from each TCCON site, as indicated by the y-axis titles, and each column shows results from each window, as indicated by the column title. Each subplot shows the NRMSE (the blue bars, magnitude shown by the left-hand y axis) and NAM (the black bars, indicated by the right-hand y axis) values for each spectroscopic database as indicated by the x axis. The horizontal dashed red lines indicated the magnitude of 1, the value where we assume the bias values to be significant.

For ¹³CH₄ DMFs there is no obvious reference value available since ¹³CH₄ is not typically retrieved from TCCON. We therefore chose to use GGG2014 window 1 as a reference in order to investigate window deviations. We found that apart from SEOM-IAS window 1, which showed deviation below the noise level from every TCCON site, every other case showed notable levels of deviation ranging from 1.5–5. Here we cannot attribute these disagreements purely to spectroscopic differences since ¹³CH₄ retrievals will be subject to high noise levels.

3.3 Impact of local condition changes on variance between windows and databases

It has been shown (Wunch et al., 2011) that the variability in local conditions can have an impact on the accuracy of TCCON retrievals (through the a priori data). We therefore investigate in this section if varying local conditions (specifically, water vapour, SZA and temperature) affect each window in each spectroscopic database differently. Wunch et al. (2011) identified a non-linear relationship between the SZA and retrieval anomalies. To account for non-linearity, we fit the normalised residual DMF values with a second-order model.

Figure 9 The sensitivity of water vapour, SZA and temperature variations in retrieved ¹²CH₄ DMFs from each TCCON site, spectroscopic database and window. Each subplot shows the second-order regression fit of the normalised residual between the retrieved window and spectroscopic database DMFs and the TCCON reference DMFs, with the fit window and spectroscopic database indicated by the legend. Each row of the figure shows data from each TCCON site as indicated in the y axis, and each column shows the sensitivity to a specific condition as shown in the title.

Figure 9 qualitatively describes the sensitivity of each TCCON site, database and window to variations in local conditions. For Ny-Ålesund (row 1), there are a mixture of non-linear and linear sensitivities to variations in water vapour and SZA. Windows 2, 3 and 4 for GEISA2020 indicate particularly significant non-linear sensitivities to SZA variations. Sensitivities to temperature variation are generally linear, although some indications of slight non-linear behaviour are apparent (GEISA2020). There are some cases where little sensitivity is observed, e.g. HITRAN, suggesting a wide range of responses in the databases and windows. In contrast to Ny-Ålesund, Darwin (row 2) shows limited sensitivity to local condition variations, with low-magnitude linear gradients observed for most cases. There are some exceptions, notably HITRAN window 3 and GEISA2020 windows 3 and 4 in relation to SZA variations, where large-magnitude non-linear behaviour is observed. Tsukuba (row 3) again shows different behaviour, with almost all databases and windows showing significant linear or non-linear sensitivity. Window 1 for SEOM-IAS, GGG2020, HITRAN and GEISA indicates high-magnitude negative linear relationships, with all other cases showing a range of sensitivity. For variations in SZA, as with Ny-Ålesund and Darwin, HITRAN window 3 and GEISA2020 windows 3 and 4 suggest strong non-linear sensitivity to variations in SZA. Most of the other windows and databases indicate some linear/non-linear sensitivity but not to the same degree as HITRAN window 3 and GEISA2020 windows 3 and 4. Temperature variations for Tsukuba indicate significant non-linear sensitivity for window 1 in most cases (except GGG2014) and in general show different results from those shown in Ny-Ålesund and Darwin. Finally for Ascension Island, we note almost no sensitivity to any local condition variation, except for HITRAN window 3 and GEISA2020 windows 3 and 4 with SZA variations, which have shown sensitivity in all cases.

The qualitative assessment above is explored quantitatively in more detail in Figs. 10, 11 and 12, where Pearson's correlation coefficients and the regression coefficients for a second-order fit are shown. Note that the y-axis scales are unified for all sites and windows for each variable considered to allow for direct comparison. In the following analysis, we assume the presence of a substantial linear correlation when r values >0.5 are shown. Further, we assume any deviation from zero of the regression statistics to indicate the presence of at least minor sensitivity. We first consider the impact of SZA variations in window 1 across all sites. There is no indication of a substantial linear correlation, with no database at any of the sites showing an r value >0.5. The regression statistics for Darwin and Ascension Island do not suggest any non-linear or linear relationship, while Tsukuba indicates the presence of a non-linear relationship for SEOM-IAS and HITRAN. All three sites do indicate the presence of a constant bias (although minor in the case of Ascension Island). The results from Ny-Ålesund indicate more significant sensitivity to SZA variations. SEOM-IAS show a slight linear relationship, while GGG2014, GGG2020 and GEISA2020 show second- and first-order sensitivity; in contrast HITRAN shows only minor sensitivity. For window 2 we see weak to strong linear correlation in all databases across all sites, except for a few cases (e.g. HITRAN at Darwin and most cases at Ascension Island). The regression statistics indicate minor sensitivity for GGG2014, GGG2020 and HITRAN at most sites, except for Tsukuba where HITRAN is significant. Ny-Ålesund shows notable non-linear regression statistics for all databases, indicating high sensitivity to SZA variations, and given that Ny-Ålesund operates at the highest SZAs of all of the TCCON sites under consideration, this is logical. Window 3 shows notable levels of correlation to SZA variations with respect to all sites, with Ny-Ålesund, Tsukuba and Ascension Island showing particularly large correlations (>0.5), especially in the GGG2014 and GEISA2020 databases. However, Darwin and Ascension Island show only very minor sensitivity to SZA variations, except for GEISA2020, at these sites. Tsukuba and again Ny-Ålesund show much higher sensitivity for all databases (except GGG2020 and HITRAN for Tsukuba), with the regression statistics showing second- and first-order coefficient values much larger than in any other case. Window 4 shows large linear correlations for almost all of the databases, especially in the Ny-Ålesund retrievals. Generally for Darwin, Tsukuba and Ascension Island, little to no sensitivity to SZA variations is observed for GGG2014 and GGG2020, while HITRAN and GEISA2020 both indicate more significant sensitivity, comparable to Ny-Ålesund in window 1. In contrast, Ny-Ålesund shows a much greater sensitivity to SZA variations for all spectroscopic databases. Overall the results in Fig. 10 indicate limited to no sensitivity to SZA variations for the Darwin, Tsukuba and Ascension Island sites for the GGG2014 and GGG2020 databases, while HITRAN and GEISA2020 do indicate some sensitivity, especially in windows 3 and 4. The Ny-Ålesund site, however, shows significant sensitivity for all of the spectroscopic databases, across all of the windows. The windows and spectroscopic databases all show similar results, with no clear “winner” or “loser”. These results could be explained by the fact that Ny-Ålesund operates at higher SZA angles than any of the other TCCON sites, meaning the retrieval path length will be longer, potentially allowing for more errors to propagate into the retrievals. The results from Ascension Island, which operates at lower SZAs than any of the other TCCON sites considered in this study, indicates the lowest-magnitude sensitivities, adding weight to this argument.

The sensitivity of each window to water vapour is explored in Fig. 11. Beginning with window 1, only the SEOM-IAS, HITRAN and GEISA2020 databases at the Tsukuba site indicate the presence of a significant linear correlation. For these sites we note that the second-order regression statistics indicate only very minor sensitivity, and although there is more evidence of stronger first-order sensitivity, the lack of a linear correlation suggests water vapour variation has only a minor impact. Given both Tsukuba and Darwin have large variations in background water vapour, this is an interesting result. Ny-Ålesund, however, indicates the presence of strong non-linear relationships for GGG2020, HITRAN and GEISA2020, contrasting the results from the other sites. Window 2 shows little linear correlation for most of the cases, except for GGG2020 at Ny-Ålesund and all sites at Tsukuba. Ny-Ålesund indicates the presence of significant non-linear correlations for all spectroscopic databases in contrast to Darwin, which shows only very minor sensitivity to water vapour variations. GGG2014 at the Tsukuba site presents a very minor linear sensitivity, while HITRAN and GEISA2020 both show the presence of a significant non-linear regression. Finally for window 2, Ascension Island shows similar results for GGG2014, GGG2020 and HITRAN, namely a minor non-linear relationship, while GEISA2020 shows the presence of a more significant non-linear relationship. With window 3, there are no cases of significant linear correlation; we note no or very slight (HITRAN and GEISA2020 – Darwin) non-linear relationships at Darwin, Tsukuba and Ascension Island, except for GEISA2020 at Ascension Island. The results from Ny-Ålesund contrast the other sites, by indicating strong non-linear relationships for all spectroscopic databases, except GGG2020. Window 4 shows similar results to window 3, with almost no cases indicating significant linear correlation (except GGG2014 at Tsukuba). Darwin and Tsukuba show only very slight non-linear sensitivity to water vapour variation, while Ny-Ålesund and Ascension Island both indicate significant sensitivity across all databases (aside from GGG2014 at Ny-Ålesund). The conclusions from this analysis suggest locally varying conditions are key in determining the impact of water vapour variations. It is interesting that the site with the lowest-magnitude background water vapour (Ny-Ålesund) is the most affected by water vapour variation, as opposed to Darwin which has the highest background and variation, showing little sensitivity to water vapour variation. The key difference between Ny-Ålesund and the other sites is the SZA at which measurements are taken, meaning TCCON measurements taken at a high SZA will be more sensitive to other varying conditions.

The results for retrieval bias sensitivity to variations in temperature are shown in Fig. 12. For window 1, only the Tsukuba site shows significant linear correlation with temperature variation, for the SEOM-IAS, HITRAN and GEISA2020 databases. Ny-Ålesund shows limited sensitivity across all spectroscopic databases, contrasting with the other TCCON sites, all of which indicate some sensitivity. Darwin and Tsukuba show the presence of a non-linear relationship across all spectroscopic databases, with Ascension Island showing a non-linear relationship in most cases. For window 2, Ny-Ålesund shows very limited non-linearity, and although strong linear correlations are observed, only linear coefficients of minor magnitude are observed, indicating low sensitivity. Darwin shows little to no sensitivity across all cases, while Tsukuba suggests a strong linear correlation for GGG2014 and almost no sensitivity, but both HITRAN and GEISA2020 suggest strong non-linear relationships. Contrasting the other sites, Ascension Island shows almost no linear correlation but the presence of significant non-linear relationships for all spectroscopic databases. For window 3, Ny-Ålesund shows only minor sensitivity to temperature variations, while Darwin indicates the presence of non-linear relationships similarly to window 1. Tsukuba shows similar results to window 2, such that GGG2014 does not indicate any sensitivity, while HITRAN and GEISA2020 show stronger non-linear relationships. Again Ascension Island is the outlier, showing strong non-linear sensitivity for all spectroscopic databases. Window 4 is also different from the other windows, with the results from Ny-Ålesund generally showing more sensitivity to temperature variations than the other windows, especially GEISA2020. For Darwin, GGG2014 and GGG2020 show very little sensitivity, while HITRAN and GEISA2020 show significant non-linear relationships. Tsukuba shows similar results to windows 2 and 3, where HITRAN and GEISA2020 indicate strong non-linear sensitivities, while GGG2014 is largely invariant to temperature variations. Ascension Island shows similar results to windows 2 and 3, where strong non-linear relationships are observed for all spectroscopic databases. In summary, variations in temperature will impact inter-window and inter-database biases. The impact depends significantly on the local conditions as well as the window and database in question. We note that Ascension Island shows the most significant impact in these results. However, this is possibly a biased result due to the fact that Ascension Island shows very little temperature variation over the measurement dataset. The other sites which capture measurements over a much wider range of temperatures are therefore more reliable.

In general, there is no obvious case of one window or database showing increased sensitivity over and above any of the others (although Ascension Island is typically less sensitive). However there are clear indications of sensitivity to variations in the local conditions which vary between window, database and TCCON site, in some cases with very strong correlations and sensitivities. For example, Ascension Island has some of the least varying conditions of all of the sites, and the results from this study indicate the least bias sensitivity. Ny-Ålesund has less variability than Darwin or Tsukuba but takes measurements under more challenging conditions and also shows more sensitivity than either of these sites.

We note that when calculating Pearson's correlation coefficient for GGG2020 values for window 1 at the Tsukuba site, large p values were found, indicating that these results are not statistically significant and therefore should be ignored. The p test was applied to all other window and spectroscopic database combinations, all of which showed significance with respect to the p test, i.e. <0.05.

Figure 10 Bar and scatter charts indicating the statistics behind the sensitivities to SZA variations shown in Fig. 9. Each row indicates the results from a TCCON site, as indicated by the y-axis label; each column shows the results from a particular window as shown by the column title. The blue bar plots show Pearson's correlation coefficients, with the left-hand blue y-axis values giving the appropriate scale. The black dots show the second-order coefficients from the regression from the fits shown in Fig. 9, with the right-hand black y-axis values as the scale. The red dots show the first-order coefficients, corresponding to the right-hand red y axis, and the green dots show the constant values corresponding to the right-hand green y axis. Note all values for GGG2020 at the Tsukuba site have been removed due to p-test failure.

Figure 11 As Fig. 10 but showing the sensitivities to water vapour variations.

Figure 12 As Fig. 10 but showing the sensitivities to temperature variations.

A similar analysis for retrieval bias sensitivities for ¹³CH₄ indicated high levels of sensitivity to SZA variation, especially those retrievals from Ny-Ålesund where SZAs are high. There are some windows that indicate no correlation, but the majority had values greater than 0.3. With respect to water vapour and temperature variations, these results are mixed with different windows and databases at different sites indicating different results. However, there is a general trend of sensitivity to water vapour and temperature variations, with only a small number of cases indicating no correlation. These results suggest ¹³CH₄ retrievals are more sensitive to changing conditions than ¹²CH₄, which is expected.

3.4 Calculation of δ¹³C values

The calculation of the δ¹³C values (Eq. 1) can give some insight into the accuracy of ¹³CH₄ retrievals from TCCON, as well as into the impact of local condition variations on these retrievals. δ¹³C is calculated for all TCCON sites using all combinations of windows from all databases, using averaged ¹²CH₄ and ¹³CH₄ for the whole time series available for each TCCON site. There are two factors to look for in the analysis of δ¹³C, firstly the bias with respect to the accepted atmospheric average of −47 ‰ and secondly the consistency of the calculated values across databases and windows.

Table 4 Averaged values of δ¹³C from all TCCON sites for all possible combinations of ¹²CH₄ with window 1 of ¹³CH₄ for each spectral database.

Table 5 Averaged values of δ¹³C from all TCCON sites for all possible combinations of ¹²CH₄ with window 4 of ¹³CH₄ for each spectral database.

Tables 4 and 5 indicate a wide range of results, suggesting either significant differences in spectroscopic parameters or large retrieval uncertainty. GGG yields a mean uncertainty in ¹³CH₄ retrievals of between 0.5–2 ppb (∼2.5 %–10 %) depending on the database and TCCON site. However, given that these uncertainties can be averaged over a long period of time, they should reduce significantly (by about 200 times in the case of Darwin), meaning that the precision of ¹³CH₄ retrievals should be very high (e.g. <0.006 ppb). Therefore precision errors cannot explain the differences in δ¹³C values shown in Tables 4 and 5, meaning differences in the spectroscopic databases are the key sources of errors in ¹³CH₄ retrievals. This therefore suggests that knowledge of ¹³CH₄ spectroscopic parameters must be improved before serious attempts of remote sensing of ¹³CH₄ can be made.

Looking at these results in more detail and considering the retrievals that use window 1 for ¹³CH₄, these combinations yield surprisingly consistent results across all sites and windows. An exception is the HITRAN results, which show significant bias at the Ny-Ålesund site, although this is contrasted by the HITRAN results at Darwin and Tsukuba, which indicate values very close to what might be expected. Indeed, the results from HITRAN at Ny-Ålesund are significantly different from those at any of the other sites, which can be explained by ¹³CH₄ retrievals showing significantly larger biases (at least twice those of any other database), and very high Pearson's correlation (0.7). The results from GEISA are the most consistent across the window combinations for all sites, showing a maximum of ∼13 ‰ variation across all window combinations, which is a remarkable result. For comparison purposes, HITRAN shows ∼25 ‰ variation, GGG2020 ∼18 ‰ variation and GGG2014 ∼25 ‰–60 ‰ variation. The variations between databases in the same window combinations are larger than those in between windows, suggesting variable dependence on local conditions and thus differences in spectroscopic parameters. Except for HITRAN, all of the databases and windows seem to underestimate the accepted δ¹³C values.

The results for the ¹³CH₄ window-4 combination are highly varied, more so than those shown for the window-1 combinations. For Ny-Ålesund, both the GGG2014 and the GGG2020 results show high levels of bias and significant variation, while the HITRAN and GEISA results show much lower bias levels and generally consistent results, with the HITRAN window-1-and-4 combination showing a realistic result. This is contrasted by the results from Darwin, where large biases are observed from all of the databases, but there are similar levels of consistency between the window combinations. Tsukuba again shows large bias levels between databases and windows, with GEISA showing high levels of consistency but large bias. Ascension Island shows similar results to Darwin (except for the HITRAN calculations), indicating similar sensitivity to background conditions. Pearson's correlation coefficients for window 4 generally indicated lower levels of sensitivity to variations in local conditions than window 1, suggesting the spectroscopic parameters for ¹³CH₄ in window 4 have significant uncertainty. Furthermore, we found that the retrieval errors generated from ¹³CH₄ in window 4 were at least double those from ¹³CH₄ in window 1. This lower uncertainty is key in explaining the lower variation in the δ¹³C metric calculated using window 1.

Overall the results in Tables 4 and 5 suggest GEISA2020 has the most consistent ¹³CH₄ retrievals across all windows and sites and relatively low bias levels. This consistency is surprising and is worth further investigation. Window 4, however, for all spectroscopic databases yields far less accurate results, suggesting more work must be done for spectroscopic parameters in this window for ¹³CH₄.

4 Discussion

We have shown the presence of correlations between variations in specific local conditions and retrieval biases. However it should be noted that other local conditions do vary in parallel with those indicated in Sect. 3.3. It is therefore likely that each window and spectroscopic database show bias variability due to the variation in a number of conditions, which is why each TCCON site shows different results. The key message remains true, however, that different windows in different spectroscopic databases are sensitive to varying degrees to local changing conditions. Further analysis in this topic should be assessed; for example the impact of the air-mass factor changes or variations in the O₂ retrievals may be important. We note Cygan et al. (2012) and Ngo et al. (2013) identify Voigt broadening parameters for O₂ as insufficient. The release of the GGG2020 environment may allow for the testing of the impact of non-Voigt parameters on O₂ retrievals. Currently, the bias present in TCCON O₂ retrievals is removed by air-mass correction factors, based on results from the O₂ parameters in the GGG2014 database, and modified for each TCCON site. This means the use of DMFs for the comparisons in this study is likely to unfairly favour the results from the GGG2014 database. Therefore a potential option for comparison purposes would be to calculate volume mixing ratios (VMRs) based on dividing the retrieved methane quantity by the column of dry air calculated using the surface pressure and water vapour column, as opposed to the O₂ column. While this method would remove the biases associated with the GGG2014 O₂, it would introduce biases associated with the measured surface pressure and the water vapour column which are more significant than the biases associated with O₂ (Wunch et al., 2011). Further, one of the key reasons for using DMFs as opposed to VMRs is that O₂ is well known and a constant and can be used as a standard between all of the sites. Therefore, while the use of DMFs introduces biases, the use of VMRs would make the different sites less comparable.

We have also not considered errors in the instruments themselves; for example variations in the instrument line shape function between different TCCON instruments could cause additional biases.

We note that advancements are currently being tested on retrievals of methane from TCCON spectra, for example with the SFIT4 algorithm (Zhou et al., 2019), which allows for profile retrievals and would therefore be less subject to the methane profile errors that can occur in GGG retrievals (Wunch et al., 2011). In addition to profile retrievals, this study used the GGG2014 retrieval software, while the more recent version of this software GGG2020 has also recently been announced. This update includes an improved spectroscopic database (this database was used in this study, wrapped in the GGG2014 software), which includes non-Voigt line shapes for methane and possibly other gases. However, the GGG2014 software used in this study cannot leverage the non-Voigt parameters currently embedded in the GGG2020 spectroscopic database. Therefore further analysis using the GGG2020 software instead of GGG2014 and the use of other algorithms in this study could yield improved or different results. However, it is likely that the bias problems identified in this study would remain to some degree.

In addition to understanding the biases associated with retrieving ¹²CH₄ DMFs from TCCON spectra with differing spectroscopic databases, this study touches on a question that is of some interest to the community, namely whether it is possible to calculate realistic and constant δ¹³C values from TCCON. The results shown in Tables 4 and 5 suggest this is not yet possible since they are often significantly different from the tropospheric average δ¹³C value which is assumed to be −47 ‰ (Sherwood et al., 2016) and variable between databases and windows. There are some interesting cases where results close to the expected δ¹³C value are calculated (e.g. windows 1 and 1 for HITRAN at Tsukuba); however given that the same database in the same windows yields a completely inaccurate result at another TCCON site, it is challenging to draw any conclusions without further analysis. What is clear, however, is that the δ¹³C values calculated using ¹³CH₄ retrievals from window 1 tend to have fewer biases than those calculated using window 4 and show less variation between windows and TCCON sites, as well as more consistent results between the spectroscopic databases. The implication of these results is that window 1 is superior to window 4 for retrieving ¹³CH₄ DMF; however whether this is due to superior information content or more accurate knowledge of spectroscopic parameters requires further research.

Given that TCCON retrieves total column estimates and not in situ samples as assumed by Sherwood et al. (2016), this assumption of −47 ‰ is a little unfair, since this is based on lower tropospheric averages and does not take into account sink processes that occur further up into the atmosphere. For example Rigby et al. (2017) assume a −2.6 ‰ fractionation due to the chlorine sink in the stratosphere, and significant fractionation does occur in the troposphere with the OH sink (Röckmann et al., 2011). However, it can be argued here that the priority in calculating an accurate value of δ¹³C from TCCON is a full assessment of all of the systematic biases present in the retrievals, most notably the spectroscopic biases, before discussion of the true δ¹³C value of the total column.

The results in this study have also shown the impact of water vapour is significant when considering inter-window and spectroscopic database retrievals, as identified by Figs. 11 and 9. Therefore further work is necessary to characterise the impacts on the biases exhibited in the results shown in this paper.

5 Conclusions

In this study, using the GGG2014 retrieval environment we retrieve ¹²CH₄ DMFs from four TCCON sites over the course of a year, with the aim of understanding the biases associated with retrieving methane in the TROPOMI spectral region as opposed to standard TCCON methane windows. Four different windows covering the spectral range of the future S5–UVNS instrument and the current S5P–TROPOMI instrument are used. Three of the windows are routinely used in TCCON products, but the TROPOMI–UVNS window in the 4190–4340 cm⁻¹ range is not. We use five sources of spectroscopic parameters, the HITRAN2016, GEISA2020, SEOM-IAS and internal TCCON databases (GGG2014 and GGG2020), in order to assess the impact of spectroscopic database uncertainties.

Firstly we analysed the quality of fit of each of the windows for each of the spectroscopic databases. For each window we find the GGG2020 spectroscopic database shows the best-fit metrics, except in window 1, where SEOM-IAS have the best quality of fit. We note that while each TCCON site shows different fit statistics for each window, the order of the spectroscopic databases in terms of quality of fit remains the same in all cases, with GGG2020 showing the best followed by GGG2014, HITRAN2016 and GEISA2020. Window 1 shows the poorest-quality fit of all of the windows, indicating room to improve the spectroscopic parameters for window 1.

Using metrics based on bias with respect to the standard TCCON methane retrieval window (a weighted average of three windows), we found that each of the TCCON sites, the GGG2014 and GGG2020 databases, exhibited normalised biases <1 in the standard TCCON windows, meaning that these biases were below the retrieval noise limit and were therefore not significant. However in window 1, both GGG2014 and GGG2020 indicated biases >1 for most of the TCCON sites, suggesting that TCCON retrievals in window 1 have a significant bias with respect to the standard TCCON window. Similarly the HITRAN and GEISA databases showed significant biases (in some cases >2) with respect to the standard in windows 1 and 4, indicating significant disagreement between the standard TCCON retrievals and the HITRAN and GEISA databases in these windows. Only the GEISA database showed significant disagreement with the standard in windows 2 and 3, which, based on the other results shown in this paper, suggests that the GEISA database has the largest differences of all of the databases considered in this study.

The sensitivity of the retrieved ¹²CH₄ DMFs to locally changing conditions such as water vapour, SZA and temperature is investigated. We find significant levels of dependence on these variations that are not necessarily mirrored across all of the TCCON sites. We conclude that some retrieval windows and spectroscopic databases are more sensitive to variable conditions than others. This sensitivity is exacerbated at TCCON locations with highly variable and challenging local conditions.

The δ¹³C metric calculated in this study shows significant bias with respect to the expected total column value of −47 ‰. However, the use of the 4265 cm⁻¹ window shows significant benefit over the 6076 cm⁻¹ window and more consistent results across spectroscopic databases. Yet, the high levels of differences between the spectroscopic databases suggest high levels of uncertainty in ¹³CH₄ parameters, and further work must be done to reduce these uncertainties.

The analysis in this study led to two key conclusions: firstly we recommend including the TROPOMI SWIR spectral region (in this study, window 1) in future TCCON methane retrievals. This is based on comparable fit statistics with the original TCCON methane windows, and the significant bias with respect to the standard TCCON retrieval product. Secondly, the different spectral windows used to generate the TCCON methane products are affected by variability in local conditions to varying degrees, suggesting that the weightings normally used to generate TCCON methane products should depend on the TCCON site and season.

Appendix A: Spectral fits

Figure A1 Example residual transmission spectra calculated from measured and fitted spectra from the Darwin site in 2020.

Figure A2 Example residual transmission spectra calculated from measured and fitted spectra from the Tsukuba site in 2019.

Figure A3 Example residual transmission spectra calculated from measured and fitted spectra from the Ascension Island site in 2015.