XCO₂ retrieval for GOSAT and GOSAT-2 based on the FOCAL algorithm

Since 2009, the Greenhouse gases Observing SATellite (GOSAT) has performed radiance measurements in the near-infrared (NIR) and shortwave infrared (SWIR) spectral region. From February 2019 onward, data from GOSAT-2 have also been available.

We present the first results from the application of the Fast atmOspheric traCe gAs retrievaL (FOCAL) algorithm to derive column-averaged dry-air mole fractions of carbon dioxide (XCO₂) from GOSAT and GOSAT-2 radiances and their validation. FOCAL was initially developed for OCO-2 XCO₂ retrievals and allows simultaneous retrievals of several gases over both land and ocean. Because FOCAL is accurate and numerically very fast, it is currently being considered as a candidate algorithm for the forthcoming European anthropogenic CO₂ Monitoring (CO2M) mission to be launched in 2025.

We present the adaptation of FOCAL to GOSAT and discuss the changes made and GOSAT specific additions. This particularly includes modifications in pre-processing (e.g. cloud detection) and post-processing (bias correction and filtering).

A feature of the new application of FOCAL to GOSAT and GOSAT-2 is the independent use of both S- and P-polarisation spectra in the retrieval. This is not possible for OCO-2, which measures only one polarisation direction. Additionally, we make use of GOSAT's wider spectral coverage compared to OCO-2 and derive not only XCO₂, water vapour (H₂O), and solar-induced fluorescence (SIF) but also methane (XCH₄), with the potential for further atmospheric constituents and parameters like semi-heavy water vapour (HDO). In the case of GOSAT-2, the retrieval of nitrous oxide (XN₂O) and carbon monoxide (CO) may also be possible.

Here, we concentrate on the new FOCAL XCO₂ data products. We describe the generation of the products as well as applied filtering and bias correction procedures. GOSAT-FOCAL XCO₂ data have been produced for the time interval 2009 to 2019. Comparisons with other independent GOSAT data sets reveal agreement of long-term temporal variations within about 1 ppm over 1 decade; differences in seasonal variations of about 0.5 ppm are observed. Furthermore, we obtain a station-to-station bias of the new GOSAT-FOCAL product to the ground-based Total Carbon Column Observing Network (TCCON) of 0.56 ppm with a mean scatter of 1.89 ppm.

The GOSAT-2-FOCAL XCO₂ product is generated in a similar way as the GOSAT-FOCAL product, but with adapted settings. All GOSAT-2 data until the end of 2019 have been processed. Because of this limited time interval, the GOSAT-2 results are considered to be preliminary only, but first comparisons show that these data compare well with the GOSAT-FOCAL results and also TCCON.

1 Introduction

Carbon dioxide (CO₂) is the most important greenhouse gas in the context of global warming (e.g. IPCC, 2013). The amount of CO₂ in the atmosphere is primarily determined by natural and anthropogenic sources and sinks, but our current understanding of these sources and sinks has significant gaps (e.g. Ciais et al., 2014; Reuter et al., 2017a; Friedlingstein et al., 2019; Janssens-Maenhout et al., 2020). Retrievals of column-averaged carbon dioxide (XCO₂) from the satellite sensors SCIAMACHY (ENVISAT) (Burrows et al., 1995; Bovensmann et al., 1999; Reuter et al., 2010, 2011) and TANSO FTS (GOSAT) (Kuze et al., 2016), as well as from the Orbiting Carbon Observatory-2 (OCO-2) satellite (Crisp et al., 2004; Eldering et al., 2017; O'Dell et al., 2012, 2018) have been used for over a decade to obtain information on natural CO₂ sources and sinks (e.g. Chevallier et al., 2014; Chevallier, 2015; Reuter et al., 2014b, 2017a; Schneising et al., 2014; Basu et al., 2013; Houweling et al., 2015; Kaminski et al., 2017; Liu et al., 2017; Eldering et al., 2017; Yin et al., 2018; Palmer et al., 2019) and on anthropogenic CO₂ emissions (e.g. Schneising et al., 2008, 2013; Reuter et al., 2014a, 2019a; Nassar et al., 2017; Schwandner et al., 2017; Miller et al., 2019; Labzovskii et al., 2019; Wu et al., 2020; Zheng et al., 2020).

The first satellite measurements of XCO₂ were performed by the Scanning Imaging Absorption Spectrometer for Atmospheric CHartographY (SCIAMACHY) instrument (Bovensmann et al., 1999; Gottwald and Bovensmann, 2011; Reuter et al., 2010, 2011) on the European environmental satellite ENVISAT launched in 2002 and operating until April 2012.

Whereas greenhouse gases were only one field of application among others of SCIAMACHY, later satellite missions focused explicitly on these. In 2009, the Greenhouse gases Observing SATellite (GOSAT; Kuze et al., 2009, 2016) was launched, followed by the Orbiting Carbon Observatory-2 (OCO-2; Crisp et al., 2017; Eldering et al., 2017; O'Dell et al., 2012, 2018) in 2014. Furthermore, in 2016 the Chinese TanSat mission was launched; the first results were presented by Yang et al. (2018). Follow-on instruments to GOSAT and OCO-2 (GOSAT-2; Suto et al., 2021) (OCO-3; Taylor et al., 2020) have been in orbit since 2018 and 2019, respectively. The TanSat, GOSAT, and OCO-2 and OCO-3 instruments are still operating, and several different retrieval algorithms have been developed to derive XCO₂ from their near-infrared (NIR) and shortwave infrared (SWIR) spectra.

The main challenge for spaceborne XCO₂ measurements is the required accuracy of the resulting data products as the atmospheric background of XCO₂ is high compared to the variability, which is typically less than a few percent (about 2 % seasonal cycle variations in the Northern Hemisphere in addition to an annual increase of about 0.5 % yr⁻¹; see e.g. Schneising et al., 2014; Buchwitz et al., 2018). Depending on the application, even higher accuracies are needed. An accurate XCO₂ retrieval usually requires a complex retrieval method and large computational effort. This is no major problem for the number of measurements provided by the GOSAT instruments, but even current OCO-2 retrievals require significantly larger computational effort. However, new missions with much higher spatial resolution and coverage are currently in preparation to address the challenging questions on CO₂ local and global sources and sinks in a changing climate; one amongst them is the forthcoming European anthropogenic CO₂ Monitoring (CO2M) mission (Kuhlmann et al., 2019; Janssens-Maenhout et al., 2020), dramatically increasing the computational power needed for retrievals.

About 3 years ago, Reuter et al. (2017b, c) developed the Fast atmOspheric traCe gAs retrievaL (FOCAL) and applied it to OCO-2 data. To show the applicability of the FOCAL method not only to OCO-2 but also to other satellite sensors, we present in this study a new application of FOCAL to GOSAT and also some first results from an application to GOSAT-2. GOSAT-FOCAL has several advantages over GOSAT BESD (Heymann et al., 2015), the currently used IUP GOSAT XCO₂ retrieval product (Heymann et al., 2015), which provides only XCO₂ data over land. However, FOCAL is able to retrieve not only XCO₂ but – depending on the used spectral ranges – also other atmospheric parameters like XCH₄, H₂O, HDO, CO, and N₂O. In the present study we concentrate on XCO₂, as this is the most important (and because of its high requirements for accuracy possibly the most challenging) anthropogenic greenhouse gas.

The paper is organised as follows: in Sect. 2 we list all data sets used in this study. The retrieval algorithm is described in Sect. 3. Sections 4 and 5 then show the results of the retrieval and the validation. Finally, the conclusions are given in Sect. 6.

2 Data sets used

2.1 GOSAT and GOSAT-2

The Greenhouse gases Observing SATellite (GOSAT; Kuze et al., 2009) was launched in January 2009 and is still in operation. The Thermal And Near infrared Sensor for carbon Observation (TANSO) on-board GOSAT consists of a Cloud and Aerosol Imager (TANSO CAI) and a Fourier Transform Spectrometer (TANSO FTS), which measures radiances in the NIR and SWIR spectral region with S and P polarisation and in the thermal infrared spectral region without polarisation with a spectral resolution of 0.2 cm⁻¹. The FOCAL retrieval uses as the main input calibrated GOSAT L1B V220.220 spectra from the three NIR–SWIR bands (around 0.76, 1.6, and 2.0 µm) of TANSO FTS.

GOSAT-2 (Nakajima et al., 2017; Suto et al., 2021) was launched in October 2018 and comprises similar instrumentation as GOSAT. The GOSAT-2 FTS has the same spectral resolution but an extended spectral range for SIF and CO retrievals.

We use calibrated GOSAT-2 L1B NIR–SWIR data V101.101.

Both GOSAT and GOSAT-2 perform point measurements with a spatial resolution (footprint diameter) of about 10 km. For both instruments, we use a tabulated instrumental line shape (ILS) with a kernel width of 15 cm⁻¹. For GOSAT this has been generated by a theoretical formula parameterising a “real-world” FTS instrument (see e.g. formula 5.21 in Davis et al., 2001), which depends on the maximum optical path difference (MOPD, ±2.5 cm for GOSAT) and the size of the instantaneous field of view (IFOV, 15.8 mrad for GOSAT). The same formula has been used by Heymann et al. (2015). This ILS is symmetric and the same for S and P polarisation.

For GOSAT-2, we use a preliminary tabulated ILS provided by JAXA generated on 16 January 2020, which is different for S and P polarisation and asymmetric, especially in the NIR band. Meanwhile, this ILS has been officially released and is available via the NIES web site.

2.2 Reference spectra and external databases

For the retrieval several reference spectra and databases are used.

The solar spectrum used in the forward model is based on a high-resolution solar transmittance spectrum (O'Dell et al., 2012) in combination with an ISS solar reference spectrum (Meftah et al., 2018). For the SIF retrieval we used a chlorophyll fluorescence spectrum by Rascher et al. (2009), which has been scaled to 1.0 $\mathrm{mW}{\mathrm{m}}^{-\mathrm{2}}{\mathrm{sr}}^{-\mathrm{1}}{\mathrm{nm}}^{-\mathrm{1}}$ at 760 nm.

We use tabulated cross sections at a 0.001 cm⁻¹ sampling based on HITRAN2016 (Gordon et al., 2017) and the absorption cross section database ABSCO v5.0 (Benner et al., 2016; Devi et al., 2016) from the NASA (National Aeronautics and Space Administration) ACOS OCO-2 project.

Surface elevation, surface roughness, and surface type are derived from the Global Multi-resolution Terrain Elevation Data (GMTED2010; Danielson and Gesch, 2011) of the US Geological Survey (USGS) and the National Geospatial-Intelligence Agency (NGA) at a spatial resolution of 0.025^∘. Meteorological information (pressure, temperature, water vapour profiles) is obtained from the ECMWF (European Centre for Medium-Range Weather Forecasts) ERA 5 model data (Hersbach et al., 2020), which are available every 1 h on a 0.25^∘ horizontal grid and on 137 altitude layers.

We use XCO₂ data from the CarbonTracker (CT) model CT2019 and CT-NRT v2020-1 (Jacobson et al., 2020a, b) and data from the Total Carbon Column Observing Network (TCCON; see e.g. Wunch et al., 2011a) in the context of the bias correction reference database (see Sect. 2.3). TCCON data are also used for validation (see Sect. 5). Table 1 lists the TCCON stations which provided data for the present study.

Feist et al. (2014)Deutscher et al. (2019)Notholt et al. (2019a)Morino et al. (2018b)Griffith et al. (2014a)Iraci et al. (2016a)Wunch et al. (2017)Strong et al. (2019)Dubey et al. (2014)Sussmann and Rettinger (2018a)Liu et al. (2018)Iraci et al. (2016b)Blumenstock et al. (2017)Hase et al. (2014)Wennberg et al. (2016)Sherlock et al. (2014a, b)Pollard et al. (2019)Notholt et al. (2019b)Warneke et al. (2019)Te et al. (2014)Wennberg et al. (2017)Wennberg et al. (2014)De Mazière et al. (2017)Morino et al. (2017)Kawakami et al. (2014)Kivi et al. (2014)Morino et al. (2018a)Griffith et al. (2014b)Sussmann and Rettinger (2018b)

Table 1 TCCON stations used in this study.

XCO₂ a priori profiles are derived using the 2018 version of the simple empirical CO₂ model SECM (Reuter et al., 2012). In the context of validation, we use the 2020 version of SECM. XCH₄ a priori data are from the simple CH₄ climatological model SC4C2018 developed and used by Schneising et al. (2019) and briefly described by Reuter et al. (2020).

For CO₂ we use the “synth” a priori error covariance matrix described by Reuter et al. (2017b). For H₂O, we use the same error covariance matrix as Reuter et al. (2017b), but scaled by a factor of 5 to reduce the dependencies of the retrieval results on the a priori. For CH₄, for convenience, we scale the CO₂ matrix to result in an XCH₄ uncertainty of 45 ppb, which is considered to be a reasonable estimate. Note that only the matrices are scaled, not the a priori values.

2.3 The reference database

Quality filtering and bias correction usually require knowledge of a “true” (in this case XCO₂) value. For this, we do not simply use model data as truth, as one aim of XCO₂ products is to improve models. Another method is to take ground-based TCCON measurements as a basis for bias correction. However, although TCCON measurements are very accurate (estimated 1σ precision is 0.4 ppm; see Wunch et al., 2010), they are only available at certain locations and are therefore more suited for validation.

Our choice is therefore to use a database generated from a combination of TCCON measurements and CarbonTracker (CT) model data for a reference year (2018 for GOSAT, 2019 for GOSAT-2), which should on average reproduce large-scale features correctly.

This database is produced in the following way: as a first step, we determine from the CT data global daily 3D maps close to 13:00 local time (i.e. GOSAT and GOSAT-2 Equator crossing time). We reduce the altitude grid to five layers with the same dry-air sub-columns, i.e. the same quantity of particles, and interpolate the data from the native CT horizontal resolution of $\mathrm{3}{}^{\circ }\times \mathrm{2}{}^{\circ }$ to $\mathrm{0.5}{}^{\circ }\times \mathrm{0.5}{}^{\circ }$. Then we determine from the TCCON data for each day mean values (${\mathrm{XCO}}_{\mathrm{2}}^{\mathrm{TCCON}}$) for 13:00±2 h local time. Next, we select collocated CT data and correct them for the TCCON averaging kernels, resulting in a TCCON-corrected CT value at the TCCON location (${\mathrm{XCO}}_{\mathrm{2}}^{\mathrm{CT}}$). The application of the averaging kernels corrects for different vertical resolutions and/or sensitivities (see e.g. Rodgers and Connor, 2003; Wunch et al., 2010). We look for contiguous regions where CT XCO₂ data differ by less than 0.75 ppm from the CT value at the TCCON location; these data are then used for the reference database. The reference database therefore does not contain any TCCON data – it only contains CT data which were confirmed by TCCON, but individual values may differ by up to 1.5 ppm. The choice of the 0.75 ppm ranges is based on a trade-off between accuracy (agreement with TCCON) and spatial coverage of the database.

The results are daily maps containing CO₂ data for five vertical sub-layer altitudes. The spatial coverage is usually not global and varies from day to day. There are typically more data in the Southern Hemisphere during the second half of the year compared to the first half of the year (see Fig. 1). When comparing with GOSAT or GOSAT-2 measurement results, the true XCO₂ is then computed from the CO₂ layers of the reference database considering the retrieval's averaging kernels.

Figure 1 XCO₂ from the GOSAT-FOCAL reference database. The black markers show the locations of TCCON stations. (a) 1 April 2018. (b) 1 October 2018.

Specifically, for GOSAT we use CT2019 data in combination with TCCON GGG2014 (see Table 1) for 2018 here. For GOSAT-2 we also use TCCON GGG2014 data, but need to rely on CT-NRT v2020-1 for 2019. Because the CT NRT data are not yet available for the whole year of 2019, the GOSAT-2 reference database does not cover the whole year; there are essentially no data after August 2019. This is a limiting factor for GOSAT-2, especially because this also means that data in the Southern Hemisphere are less present in the 2019 database.

2.4 GOSAT Level 2 products

To assess the quality of the newly created GOSAT-FOCAL XCO₂ products, they have been compared with several other well-established GOSAT Level 2 data sets (see Sect. 5). The GOSAT BESD v01.04 product from IUP (Heymann et al., 2015) is a near-real-time product generated in the context of the Copernicus Atmospheric Monitoring Service (CAMS, https://atmosphere.copernicus.eu/, last access: 30 July 2020) project. It is available from 2014 onward. The GOSAT RemoTeC v2.3.8 product from SRON (Butz et al., 2011) and the full-physics GOSAT product from the University of Leicester v7.3 (Cogan et al., 2012) were generated in the context of the Copernicus Climate Change Service (C3S, https://climate.copernicus.eu/, last access: 30 July 2020) and cover the GOSAT time series from 2009 until the end of 2019. The recently released NASA GOSAT ACOS v9r product (O'Dell et al., 2012, 2018; Kiel et al., 2019) is also available for the years 2009 to 2019. The operational GOSAT XCO₂ product v02.95 (bias corrected) from NIES (Yoshida et al., 2013) currently ends in August 2020. The BESD product contains only XCO₂ data over land; all other products are available for water and land surfaces.

3 Retrieval algorithm

The retrieval is performed in three main steps: pre-processing, processing, and post-processing. These are described in the following subsections.

3.1 Pre-processing

During pre-processing all required input data for the main processing step are collected. Furthermore, a first filtering of data is performed to reduce processing time.

The pre-processing procedure is largely based on the pre-processing as present in the BESD GOSAT product (Heymann et al., 2015). The sequence of pre-processing activities is as follows.

1. Extract measured spectra, geolocation, and information on quality and measurement modes (e.g. gain, scan direction) from the GOSAT L1B product.

2. Estimate instrument noise and cloud parameters.

3. Filter for data quality, latitudes, solar zenith angle, signal-to-noise ratio, and clouds (see Table 2 for settings).

4. Extract surface type, elevation, and roughness derived from the surface database for each measurement.

5. Add corresponding meteorological information (pressure, temperature, dry-air column, and water vapour profiles) for the time and place of the measurements. This includes a correction for surface elevation; i.e. model profiles are extended or cut according to the value from the surface database.

6. Add a priori gas profiles for each measurement (CO₂ from SECM, CH₄ from SC4C, H₂O from meteorology). For GOSAT-2, a priori profiles for CO and N₂O are also added. The latter do not depend on geolocation; they are based on the tropical reference atmosphere from Anderson et al. (1986), scaled to column average values of XCO =0.1 ppm and XN₂O =330 ppb.

Table 2 Pre-processing filter limits.

Because FOCAL is a fast algorithm and the number of GOSAT and GOSAT-2 measurements is much lower than for OCO-2, we chose to set the pre-processing filters to be relatively relaxed and to apply the quality filtering mostly in the post-processing. As can be seen from Fig. 2 about two-thirds of the measurements are filtered out during pre-processing.

Figure 2 Example for GOSAT data filtering during the different processing steps (April 2019). Filters are listed in sequential order from top to bottom on the vertical axis. Numbers in the horizontal bars denote the percentage of remaining data after this filter was applied. Orange: total number of measurements before filtering. Yellow: pre-processing filters. Blue: step 1 post-processing filters (convergence and noise). Green: random forest post-processing filter. Light blue: additional post-processing filters.

Figure 3 Same as Fig. 2, but for GOSAT-2.

3.1.1 Noise estimate

Similar to Heymann et al. (2015) the spectral noise is initially assumed to be independent from the wavenumber for each band. It is estimated from the standard deviation of the real part of the “dark” off-band signal (i.e. the first 500 spectral points in each band). In a later step (see Sect. 3.2.1) this noise will be modified to account for additional forward model errors and overall scaling.

3.1.2 Cloud filter

The cloud filtering is based on two physical properties of clouds: clouds are (usually) bright and clouds are high (higher than the surface) so that little water vapour is above them. In the pre-processing these properties are described by two quantities: effective albedo and water vapour filter. These are derived for each spectrum as described in Heymann et al. (2015). The effective albedo for each band is estimated from the mean reflectance L within a spectral range outside the absorption. L is determined from the mean radiance I, the mean irradiance I₀, and the solar zenith angle α via

$$\begin{array}{}\text{(1)}& L={\displaystyle \frac{\mathit{\pi }I}{I_{\mathrm{0}}\cos \mathit{\alpha }}}.\end{array}$$

The specific wavenumber ranges and irradiance values used for filtering are given in Table 3.

Table 3 Parameters for cloud filtering.

The water vapour filter is determined from a spectral region with strong water vapour absorption in the SWIR-3 band (see Table 3). This filter is defined as the ratio between the median radiance and the median of the estimated noise in this spectral range.

A ground pixel is assumed to be cloudy if either the effective albedo in one of the bands or the water vapour filter exceeds the thresholds given in Table 2.

3.2 Processing

The processing is based on the Fast atmOspheric traCe gAs retrievaL (FOCAL) algorithm, which is described in detail in Reuter et al. (2017c). A first successful application of this algorithm to OCO-2 data is given in Reuter et al. (2017b). Therefore, we only summarise the main features of the algorithm here and point out the differences to the OCO-2 application.

FOCAL approximates modifications of the direct light path due to scattering in the atmosphere by a single scattering layer, which is characterised by its height (pressure level), its optical thickness, and an Ångström parameter that describes the wavenumber dependence of scattering. The layer height is normalised to the surface pressure. Furthermore, Lambertian scattering on the surface is considered. For atmospheric scattering processes an isotropic phase function is assumed. With this approximation, the FOCAL forward model is essentially an analytical formula; it uses pre-calculated and tabulated cross sections such that calculations can be performed considerably fast. The inversion of the forward model is based on optimal estimation (Rodgers, 2000) and uses the Levenberg–Marquardt–Fletcher method (Fletcher, 1971) to minimise the cost function.

The OCO-2 retrieval of Reuter et al. (2017b, c) uses four fit windows in the NIR (near-infrared) and SWIR spectral range to derive the atmospheric parameters XCO₂, water vapour, and SIF. In contrast to OCO-2, GOSAT and GOSAT-2 cover a wider spectral range and provide spectra in two polarisation directions referred to as S and P. Therefore, we treat both polarisation directions as independent spectra in our retrieval as opposed to the average of both as usually used in other GOSAT retrievals (see e.g. Butz et al., 2011; Cogan et al., 2012; O'Dell et al., 2012). However, recently Kuze et al. (2020) presented a methane retrieval for GOSAT based on an algorithm from Kikuchi et al. (2016), which also makes use of both polarisation directions.

We use both polarisation corrections mainly for the following reasons.

• In principle, information is lost when averaging S and P spectra.

• In general, the sensitivity of the instruments and therefore the calibration of the measured spectra is different for S and P. For example, the measured ILS is given independently for S and P.

• S and P include different information on scattering, which can also be used in filtering and/or bias correction.

Furthermore, the FOCAL fitting windows (see Table 4) have been adapted to the specific GOSAT(-2) spectral bands such that, in addition, other atmospheric constituents and parameters like HDO can be derived. In the case of GOSAT-2, XN₂O and CO total columns can also possibly be retrieved. This results in six fitting windows for GOSAT and eight windows for GOSAT-2 for each polarisation. The retrieval is performed on a wavenumber axis.

Table 4 Definition of GOSAT and GOSAT-2 spectral fit windows (same for S and P). Windows 7 and 8 are only available for GOSAT-2.

Because of the large number of target gases and spectral bands the retrieval requires various state vector elements. These are listed together with the fit windows, from which they are determined, and their a priori values and uncertainty ranges in Table 5 for GOSAT and GOSAT-2.

Table 5 State vector elements and related retrieval settings. A priori values are also used as a first guess. “Fit windows” lists the spectral windows (see Table 4) from which the element is determined. “All” means that an element is determined from all fit windows of the specified polarisation. “Each” means that a corresponding element is fitted in each fit window. A priori values labelled “PP” are taken from pre-processing; “est.” denotes that they have been estimated from the background signal.

Note: for GOSAT-2 the polynomial degree in fit window 1P is (accidentally) set to 2 and in fit window 7S to 1.

For GOSAT, the retrieval determines CO₂, CH₄, and H₂O on five layers with same number of air particles, from which the column average values XCO₂, XCH₄, and XH₂O are then calculated. Furthermore, solar-induced fluorescence (SIF) is determined by scaling a corresponding reference spectrum.

Instead of the HDO column, we fit a scaling factor for the relative abundance of HDO compared to H₂O, δD, which is defined as

$$\begin{array}{}\text{(2)}& \mathit{\delta }D={\displaystyle \frac{R_{\mathrm{meas}}}{R_{\mathrm{VSMOW}}}}-\mathrm{1},\end{array}$$

where R_meas is the ratio of the measured HDO and H₂O columns, and R_VSMOW ($\mathrm{3.1152}\times {\mathrm{10}}^{-\mathrm{4}}$) is the corresponding value for Vienna Standard Mean Ocean Water (VSMOW). δD is usually given in units of per mill.

δD=0 ‰ corresponds to HDO concentrations as in VSMOW and $\mathit{\delta }D=-\mathrm{1000}$ ‰ to no HDO. We assume the same profile shape for HDO as for H₂O. For GOSAT-2, we also fit scaling factors to (fixed) CO and N₂O profiles.

As mentioned above, atmospheric scattering is considered in FOCAL by a single scattering layer, which is described by three parameters (height, optical depth, and Ångström coefficient). As scattering is different for S- and P-polarised light, we fit two independent layers for S and P.

In addition, we determine in each fit window (independently for S and P) a polynomial background function describing the surface albedo. For this we use second-order polynomials except for the small SIF windows (no. 1) for which a linear function is sufficient. Note that in the current version of the GOSAT-2 product the polynomial degree in fit window 1P is accidentally set to 2 and in fit window 7S to 1. This has no major impact on the retrieved XCO₂ and will be corrected in a future product version.

The GOSAT data files only contain a fixed spectral axis. As described in e.g. Heymann et al. (2015), the spectral calibration of GOSAT changes with time, especially at the beginning of the mission. This change can be corrected by a spectral scaling factor. We determine this overall scaling factor by a spectral fit in the SIF window before the retrieval. So far, this spectral pre-fitting seems to be unnecessary for GOSAT-2. In the retrieval, we then additionally consider for both GOSAT and GOSAT-2 possible additional spectral shifts and squeezes in each fit window by corresponding state vector elements, but the influences of these spectral changes on the results is rather small.

Noise model

The noise N derived from the off-band signal is only an estimate. It does not consider a possible wavenumber dependence of the noise within one spectral band. Furthermore, a potential error of the forward model needs to be considered. In the optimal estimation method this can be achieved by including the forward model error in the measurement error covariance. For this, we define a scaling factor s for the estimated noise and the quantity δF, which denotes the relative error of the forward model. The forward model error is proportional to the continuum radiance outside the absorption I, which is estimated from the 0.99 percentile of the measured radiance at the edge of each fit window. The quantities δF and s are determined using the approach described in Reuter et al. (2017b), i.e. by running the retrieval for a representative subset of data and then fitting the function

$$\begin{array}{}\text{(3)}& \text{RSR(NSR)}=\sqrt{(s\text{NSR}{)}^{\mathrm{2}}+\mathit{\delta }{F}^{\mathrm{2}}}\end{array}$$

to binned values of the residual-to-signal ratio (RSR) as a function of the noise-to-signal ratio (NSR). RSR is defined as the standard deviation of the retrieved spectral residual in each fit window divided by the continuum signal I; NSR is the standard deviation of the noise divided by I.

With the method described in Reuter et al. (2017c) it is also possible to define a 2σ outlier limit based on NSR and RSR data, which will be used to filter out data that are too noisy during post-processing (see Sect. 3.3). This is parameterised by a second-order polynomial as a function of the uncorrected NSR:

$$\begin{array}{}\text{(4)}& f_N\left(\text{NSR}\right)=a_{\mathrm{0}}+a_{\mathrm{1}}\text{NSR}+a_{\mathrm{2}}{\text{NSR}}^{\mathrm{2}},\end{array}$$

which is added to the RSR function of Eq. (3). The coefficients a_i are determined via a fit. To avoid extrapolation, f_N is set to the edge values outside the fitting range.

In order to cover the varying signal over the year, we base the noise model fits on data from 1 d per month for one reference year. For GOSAT, we take data from December 2017 to November 2018 (as there are few GOSAT data available in December 2018). For GOSAT-2 we use data from February 2019 to December 2019. In the case of GOSAT-2 we further restrict the input data for the noise model parameter fit to data over land because some of the data over water show an unexpected behaviour (low RSR in the case of large NSR), which needs further investigation. In this sense, the current GOSAT-2 noise model is considered to be preliminary and may need some refinement in the future.

Figure 4 Example results from the noise model. (a) GOSAT fit window 2P. (b) GOSAT 2 fit window 2P. (c) GOSAT fit window 6P. (d) GOSAT 2 fit window 6P.

Figure 4 shows as an example the noise model results for GOSAT and GOSAT-2 in the fit windows 2 (O₂(A) band) and 6 (strong CO₂ absorption) for P polarisation. The orange line gives the fitted RSR function and the red line the outlier limit. The derived values from the noise model for all fitting windows and polarisations are given in Tables 6 and 7 for GOSAT and GOSAT-2. The forward model errors δF are on average slightly larger for GOSAT-2 than for GOSAT. In the SWIR, values similar to OCO-2 are obtained, but in the NIR the OCO-2 δF is typically smaller (about 0.003). This indicates that for GOSAT and GOSAT-2 instrumental and/or calibration effects seem to impact the radiance errors more in the NIR than in the SWIR.

Table 6 Parameters of the GOSAT noise model.

Table 7 Parameters of the GOSAT-2 noise model.

3.3 Post-processing

The purpose of post-processing is to filter out invalid data and to perform a bias correction for the products. The current post-processing focuses on XCO₂. The post-processing is performed in several steps, namely the following:

1. basic filtering based on physical knowledge;

2. filtering out low-quality data using parameters and limits determined with a random forest classifier;

3. application of a bias correction using a random forest regressor; and

4. additional filtering-out of data with a bias correction that is too large.

These steps are described in the following subsections.

3.3.1 Basic filter

The basic filtering removes measurements for which the retrieval does not converge or for which the quality of the fit results is too low. We consider this to be the case if the χ² calculated over all fit windows is larger than 2 or if for at least one of the fit windows the RSR outlier limits (see Sect. 3.2.1) are exceeded. Furthermore, we apply some initial filters for nonphysical values to the derived scattering parameters (i.e. layer height outside the atmosphere, Ångström coefficient not within [1,5]). Note that all retrieved scattering parameters such as the Ångström exponent can be considered “effective” parameters as they have to account for not only cloud–aerosol scattering but also Rayleigh scattering (which has an Ångström coefficient of 4). Because Rayleigh scattering is always present and we filter out cloudy scenes, we usually get higher effective Ångström coefficients than those expected from clouds or aerosols only.

We also limit the maximum allowed optical depth of the scattering layer to 0.02 to filter out clouds or aerosol amounts that are too thick and use a maximum allowed XCO₂ posteriori uncertainty of 2 ppm. As described by Reuter et al. (2017c), FOCAL simulates scattering only for an isotropic phase function. The prominent forward peak, usually existing for Mie scattering phase functions of cloud and aerosol particles, basically does not modify the light path. As FOCAL's optical depths of the scattering layer do not include this forward peak, these optical depths are much smaller than optical depths including a strong forward peak while having a similar influence on the light path modification (see discussion in the publication of Reuter et al., 2017c). The maximum value of 0.02 for the layer optical depth should therefore not be interpreted as e.g. an aerosol optical depth.

The limits for the optical depth of the scattering layer and the XCO₂ error are somewhat arbitrary and actually result from visual inspection of the retrieval results. However, they are only intended as a first rough quality filter to facilitate later filter and bias correction methods, which will partly use the same parameters (see below). The detailed choice of these limits is therefore considered uncritical for the final results.

The above-mentioned filter parameters and limits (see Table 8) are applied to both land and water surfaces and are the same for GOSAT and GOSAT-2 except for the RSR outlier limit, which has been determined individually for each instrument. Figures 2 and 3 show how many data points are typically filtered out in this step. The different performance of the RSR filters for both instruments indicates that the filtering for GOSAT-2 needs some further optimisation, which is planned for the next version of the products.

Table 8 Basic filter parameters.

3.3.2 Random forest filter

In the next step, data are filtered out based on their expected XCO₂ bias, i.e. the difference to a true XCO₂. Of course, this true XCO₂ value is normally not known. We therefore use the XCO₂ reference database (as described in Sect. 2.3) to train a random forest classifier (Pedregosa et al., 2011) to identify those variables which would remove – in combination with a corresponding random forest database – a pre-described percentage p of data based on their XCO₂ bias. This is done independently for data over land and water. Note that we are only interested here in the XCO₂ bias on top of an overall global bias as the latter will be handled via the bias correction.

This method is similar to the random forest filter used by Schneising et al. (2019) in the context of CH₄ and CO retrieval. Other approaches for data filtering used e.g. by Mandrake et al. (2013) and Reuter et al. (2017b) identify the best variables through the minimisation of local XCO₂ variability in the retrieved products. However, all methods essentially serve the same purpose, i.e. to derive a reproducible filtering procedure which does not solely rely on expert knowledge.

We determine the list of relevant variables and the random forest database for the filtering in the following way: we use the (uncorrected) results of the retrieval for the reference and apply the basic filtering as described in Sect. 3.3.1. Then, the subset of these data is selected which has a corresponding true value in the reference database. For these data we determine the XCO₂ bias (measurement – reference XCO₂) and subtract the global median for land and water of this bias. We then sort the data according to this bias and flag those p percent of data with the highest absolute bias values as “bad”.

For GOSAT, this results in a total of 54 317 data points over land and 109 414 over water. These numbers are smaller for GOSAT-2 (land: 10 625, water: 40 459). The random forest classifier is then trained by randomly using 90 % of these data as input. The training is done in two iterations: first with a complete set of possible input variables (“features”) and output variables (“estimators”) and then using only a reduced set consisting of the 10 best features and/or estimators, i.e. those with the highest random forest score (relevance) in the first run. This relevance is a quantity which describes the relative importance of each feature for the filtering. Relevances are normalised such that the sum of all relevances is 1. The random forest classifier then decides for each measurement based on these 10 variables if it is filtered out or not.

The initial list of possible features and estimators essentially includes all quantities available after the retrieval, including viewing angles, surface properties, and the continuum signal for each fit window. Furthermore, the retrieved values of the state vector elements and their errors are included in this list as are averaging kernels for the profiles. We explicitly exclude the geolocation of the measurement (latitude, longitude) and the retrieved values (but not the errors) for the data products we are interested in, i.e. the gases and SIF. This is to avoid e.g. the filtering-out of certain geographical regions or removing all points with high XCO₂ values.

Note that in glint mode over ocean, the inclusion of information about the viewing geometry as possible features bears the risk that the random forest procedure may infer the geolocation from a combination of these.

However, we include as a possible filter variable the gradient of the retrieved CO₂ profile (i.e. the difference between the two lowermost layers) as this has shown to be a suitable quantity.

The original number of candidate variables presented to the random forest classifier is quite high (193 for GOSAT and 246 for GOSAT-2) as can be seen from Figs. 5 and 6 (top left plots), but there are few with high relevance.

Figure 5 Random forest results for GOSAT. (a–c) Results from the random forest filter. (d–f) Results from random forest bias correction. (a, d) Normalised relevance (score) of all filter variables. (b, e, c, f) Selected variables and their relevance for the land and water surfaces.

Figure 6 As Fig. 5, but for GOSAT-2.

The 10 best variables selected partly differ for land and water surface (as shown in the middle and left top panels), but they usually comprise scattering parameters, polynomial coefficients, spectral corrections, and some XCO₂-related parameters.

The other 10 % of the input data are used to test the performance of the classifier. The results from this test and other cross-validation activities indicate that the random forest classification is – depending on surface – only accurate in about two-thirds of the cases. This accuracy is defined as the fraction of correctly classified samples. This means that the filtering also removes possibly valid data points and does not remove all possibly bad ones. However, we do not expect a perfect classification because it is not possible to describe all inter-dependencies via the set of input features. Note that the performance of the filter is similar for both the training and the test data sets. This indicates that there are no problems with over-fitting.

To obtain high quality of the remaining XCO₂ data, we therefore need to filter out quite a large percentage of data (and perform an additional filtering at a later time; see below). For future data products further investigations are planned to improve the performance of the classifier, e.g. by providing additional features from the combination of existing ones (like the already used CO₂ gradient). The percentage p of data to be filtered out is usually a trade-off between data quality and the remaining amount of data. In the present case a 50 % limit has been selected. Actually, as can be seen from Figs. 2 and 3, the relative amount of data filtered out via the random forest classifier is not exactly 50 % of the data remaining after the previous filters.

3.3.3 Bias correction and filtering

XCO₂ retrieval methods usually require a bias correction to be applied to the data. This correction is often based on multi-linear regressions using parameters identified from correlation analyses of differences to an assumed true XCO₂ data set. Different data products use different methods to define this “truth”. In many cases, ground-based TCCON measurements are taken as a reference for the true XCO₂, like in the GOSAT BESD product (Heymann et al., 2015), the SRON products (Guerlet et al., 2013), the product from the University of Leicester (Cogan et al., 2012), and also the operational GOSAT product from NIES (Inoue et al., 2016). The bias correction of the NASA ACOS OCO-2 product is quite complex (O'Dell et al., 2018; Kiel et al., 2019); it uses as a reference a modification of the Southern Hemisphere approximation introduced by Wunch et al. (2011b). The ACOS product takes multi-model mean data as a reference and derives corrections from data in the Southern Hemisphere below 20^∘ S where CO₂ is assumed to be quite uniform in small areas. A similar small area approximation is also used by Reuter et al. (2017c) to correct FOCAL OCO-2 data. This correction method is not possible for GOSAT and GOSAT-2 data because of their sparse sampling. We therefore follow a different approach here.

For the bias correction we use as input the same data set as for the random forest filter, but with this random forest filter applied; 50 % of the resulting data set is then used to train a random forest regressor (see also Schneising et al., 2019), which aims to minimise the true XCO₂ bias, i.e. the difference to the reference database value without the global median subtracted, as a function of the specified features. To create the bias correction database and the corresponding list of best features we again run the training twice, first with the full list (the same as for the filter) and then with the top 10 features. Again, we use different corrections for land and water. The resulting parameters and their performance are shown in the bottom panels of Figs. 5 and 6. The bias correction selects similar best features as the filter, but not exactly the same quantities in the same sequence.

The actual number of features and variables to be used for both filtering and bias correction is a trade-off between many variables (explaining all relations with a risk of over-fitting and high computational effort) and few variables (no over-fitting, but maybe missing some relations). Considering 10 features for the bias correction is more than other algorithms typically use, but this is appropriate in our case because we take into account the different relevance of these parameters, which drops off rapidly within the first 10 variables (see top panels of Figs. 5 and 6).

The validity of this choice is confirmed by the application of the bias correction to the training data set and the other 50 % of the input data, which shows a comparable reduction of the XCO₂ scatter. This is an indication for a good performance (e.g. no over-fitting) of the regressor.

During application of the bias correction, the random forest regressor estimates the XCO₂ bias based on the values of the input variables. This bias is then subtracted from the retrieved value.

Currently, there is only a bias correction for XCO₂, but in principle this method is also applicable to other quantities depending on the availability of a corresponding reference database.

After the bias correction there are still a few outliers left in the XCO₂ data. These are filtered out by an additional filter on the XCO₂ bias derived via the random forest classifier. The limits for this filter are the global median bias for the test data set plus and minus 2 ppm. The median bias is different for land and water surfaces and also for GOSAT and GOSAT-2. The actual limits are given in Table 9. The value 2 ppm is estimated from visual inspection of the data. Figures 2 and 3 show that typically less than 1 %–2 % of the remaining data (less than 0.1 % of all) are affected by this last filter.

The spatial variability of the finally derived bias correction (see Fig. 7) is typically below 1 ppm, but as mentioned above there is a systematic difference between land and water data of about of 1–1.5 ppm (see Table 9).

Figure 7 Gridded GOSAT (a) and GOSAT-2 (b) XCO₂ bias correction for 2019.

Table 9 Bias filter limits.

4 Results

The FOCAL retrieval has been applied to all GOSAT and GOSAT-2 measurements until the end of 2019. On average, FOCAL needs 22 s with six iterations for the processing of one GOSAT ground pixel. For GOSAT-2 the numbers are slightly higher (28 s for seven iterations) because of the additional fit windows and state vector elements. All performance values are given for a single core of an Intel Xeon E5-2667v3 CPU (3.2 GHz). These numbers are actually about 1 order of magnitude larger than the ones given in Reuter et al. (2017b, c) for the FOCAL application to OCO-2. This is because we use separate S- and P-polarisation spectra and more retrieved variables for GOSAT(-2), which requires more and larger fit windows. For each of these fit windows and parameters, weighting functions have to be calculated, which involves a convolution with the ILS. This convolution is the most time-consuming part of the FOCAL retrieval. This is even more relevant for GOSAT(-2) because the FTS ILS is in principle sinc-shaped; i.e. it has a sharp peak in the centre but wide wings, which requires a large kernel width (in our case 15 cm⁻¹) for the convolution.

The left plots in Figs. 8 and 9 show examples for measured and fitted nadir-mode radiance spectra for GOSAT and GOSAT-2 over land in the different fitting windows for P polarisation. Since the difference between measured and modelled spectra is small and thus hard to see, we show in these figures on the right side the corresponding residuals and the estimated noise. The results for S polarisation look similar and are therefore not shown here. The residuals are on the order of magnitude of the noise, which is slightly higher for P polarisation than for S polarisation. Some small spectral structures are visible in the residuals; they appear more clearly in the smoothed residuals (convoluted with a 21-pixel boxcar), e.g. for GOSAT and GOSAT-2 in the O₂(A) band (window 2), and some broadband oscillations in window 4 and 5 for GOSAT-2. These features are present in both S and P polarisations and also occur in other measurements, so they seem systematic. A reduction of these features could further improve future products.

Figure 8 Example for a single GOSAT measurement (P polarisation): (a, c, e, g, i, k) measured (red) and retrieved (green) spectra in the different fit windows; because of the good agreement the red curve is essentially barely visible below the green curve. (b, d, f, h, j, l) Corresponding residuals (measurement – fit). Light blue: unsmoothed. Blue: smoothed with a boxcar width of 21 spectral pixels (=4.2 cm⁻¹). Red: estimated noise error range. The radiance unit is W cm⁻² cm sr⁻¹.

Figure 9 Same as Fig. 8, but for GOSAT-2.

In Fig. 10 some statistical information about the GOSAT-FOCAL data products is given. A time series for the number of valid data is given in the top plot. In recent years, about 5 %–6 % of the available measurements could be transferred to valid XCO₂ data. The number of valid data points increases from 2009 to 2019. This is mainly due to an increase in the data over water, which is related to optimisations in GOSAT operations (better use of glint geometry) over water. As expected, the mean global XCO₂ shown in the middle plot increases with time. Global mean values over water are typically slightly higher than over land. The observed XCO₂ variability (standard deviation, bottom plot) is larger over land. Part of this variability is attributed to influences of surface elevation and to different scattering pathways between the land and water measurements. For GOSAT-2, only retrieved data from 2019 are available so far. The total number of available measurements is about 2.8 million compared to about 3.5 million GOSAT measurements in 2019. Only about 3 % of the GOSAT-2 data remain after all filtering and post-processing, which is roughly half of the corresponding number for GOSAT (but similar to the first year of GOSAT). As can be seen from Fig. 3 more GOSAT-2 data are filtered out due to failed or bad convergence and by the RSR outlier limits than for GOSAT (Fig. 2). Future improvements of the GOSAT-2 calibration or the noise model are considered to help here.

Figure 10 Statistics for valid GOSAT measurements (after pre- and post-processing filtering) for each year. Blue: measurements over water. Green: measurements over land. Red: all data. (a) Number of valid measurements, including the percentage of all originally available measurements. (b) Global mean XCO₂. (c) Corresponding standard deviation.

For further analyses, we have generated monthly maps on a $\mathrm{5}{}^{\circ }\times \mathrm{5}{}^{\circ }$ grid. Example plots for the months April and August 2019 (the beginning and end of the growing season) are shown in Fig. 11 for GOSAT and in Fig. 12 for GOSAT-2. The data are not filtered for low numbers of input data in the grid points, which explains some individual outliers in the plots. Overall, the spatial patterns observed by GOSAT and GOSAT-2 look reasonable. The north–south gradient in XCO₂ is visible with a different sign in April and August for both instruments. The spatial coverage of the GOSAT-2 data is lower than for GOSAT because more data are filtered out (see above). This especially affects regions like the northern part of Africa.

Figure 11 Example for gridded GOSAT XCO₂ data. (a) April 2019. (b) August 2019.

Figure 12 Example for gridded GOSAT-2 XCO₂ data. (a) April 2019. (b) August 2019.

5 Verification and validation

For the verification and validation of the GOSAT- and GOSAT-2-FOCAL products we perform a comparison with various reference data sets (see Sect. 2), namely the following.

• The GOSAT BESD v01.04 product from IUP (referred to as BESD later).

• The GOSAT ACOS v9r product from NASA (referred to as ACOS later).

• The GOSAT UoL_FP v7.3 product from the University of Leicester (referred to as UoL later).

• The GOSAT RemoTeC v2.3.8 product from SRON (referred to as SRON later).

• The GOSAT operational product v02.95 (bias corrected) from NIES (referred to as NIES later).

• Collocated TCCON GGG2014 data (referred to as TCCON later).

For the comparisons, all data have been adjusted using the same a priori (SECM2020).

Since all GOSAT products use different retrieval and filter methods, they do not contain the same number of data (see Fig. 13). Currently, the NASA ACOS product has the largest number of valid data points, followed by the new GOSAT-FOCAL product with about 20 % less data.

Figure 13 Number of valid XCO₂ data points in the different GOSAT products from 2009 to 2019.

5.1 Direct comparisons

There are enough common measurement points included in the different GOSAT products to perform a direct comparison. Figure 14 shows as an example a comparison between the GOSAT-FOCAL data for the year 2018 with the corresponding ACOS, BESD, SRON, NIES, and UoL products. For each plot we only use data for which both data sets have a valid XCO₂ value. For these data we performed a linear regression using the orthogonal distance regression (ODR) method (see e.g. Boggs et al., 1987). Unlike common linear regression, ODR considers uncertainties for both axes (data sets) by minimising the orthogonal distances between the model curve and the data points. The ODR results are shown by the red line and its label. The number of collocations, the median and mean, and the standard deviations of the differences are given in the titles.

Figure 14 Comparison of GOSAT-FOCAL data (x axis) from 2018 with other GOSAT data (y axis). The colour of the data points corresponds to the density of data points at that location (normalised to a maximum value of 1). The dashed line corresponds to perfect agreement. The red line shows the result of a linear fit using the orthogonal distance regression (ODR) method. The total number of collocated data as well as the median, mean, and standard deviation of the XCO₂ differences between the two data sets are given in the title of the sub-plots. (a) FOCAL vs. ACOS. (b) FOCAL vs. BESD. (c) FOCAL vs. NIES. (d) FOCAL vs. SRON. (e) FOCAL vs. UoL.

Overall, the data scatter around the 1:1 line in a similar way for all comparisons. ODR slopes vary between the data sets from 0.84 (for FOCAL vs. ACOS) up to 1.08 (for FOCAL vs. BESD). Most collocations are available for the ACOS data set because this has the largest number of valid data. Mean and median differences are quite similar and reach from −0.17 ppm (comparison to BESD) to 0.67 ppm (comparison with UoL). The scatter (standard deviation of the differences) reaches from 1.4 ppm (ACOS, NIES) to 1.8 ppm (BESD).

5.2 TCCON comparisons

TCCON provides high-quality XCO₂ (and other) data, which are currently considered to be the main reference for greenhouse gas data obtained from satellite measurements. Therefore, we compared the different GOSAT data sets with collocated TCCON measurements from 2009 to the end of 2018. BESD data are not included because they do not cover the complete time interval. Collocation criteria are the following:

• maximum time difference of 2 h;

• maximum spatial distance of satellite measurement from TCCON station of 500 km; and

• maximum surface elevation difference between satellite measurement and TCCON station of 250 m.

In addition to these criteria we also consider in the validation only stations and TCCON data sets which have at least 50 collocations for all algorithms. This improves the comparability of regional and seasonal biases. As a consequence, not all stations listed in Table 1 contribute to the validation.

The comparison procedure is the same as used by Reuter et al. (2020) and described by Reuter et al. (2019b). In summary, for each TCCON site, the time series of satellite minus TCCON differences are computed under consideration of the averaging kernels, i.e. different vertical sensitivities. The resulting time series are fitted with a trend model, which includes an offset term, a slope term, and a sine term for seasonal fluctuations. The offset term is considered the station bias, and the station scatter is computed from the standard deviation of the fit residual. Results for the GOSAT time series at the TCCON stations are shown in Fig. 15. Overall, the temporal variations of XCO₂ are well reproduced by all data.

Figure 15 Time series of collocated GOSAT data at various TCCON sites for different data products including the new GOSAT-FOCAL data set.

Figure 16 shows as a summary of the GOSAT TCCON comparisons the derived bias and scatter for the different stations and products. The new GOSAT-FOCAL product compares well with the other data sets. Its differences to TCCON have a station-to-station bias (the standard deviation of the station bias) of 0.56 ppm and a mean scatter (root mean square, or rms, scatter per station) of 1.89 ppm. The seasonal component of the bias has a station-to-station average standard deviation of 0.37 ppm. Overall, the ACOS product performs best in this comparison.

Figure 16 Overview of TCCON validation results for GOSAT.

Note that the biases shown in Fig. 16 essentially correspond to a bias anomaly since the mean bias over all stations was removed from all products. This is because for most applications this mean bias is not relevant since most information is contained in gradients. Subtracting a mean bias also facilitates a comparison of different bias patterns between the algorithms. The subtracted mean station bias is actually small; for GOSAT it varies between 0.17 ppm (for ACOS) and 0.64 ppm (for NIES). Because of the subtracted mean bias different signs of biases for different products could be coincidental. However, the biases of FOCAL and ACOS are consistent with the biases found by Reuter et al. (2020).

In Figs. 17 and 18 we show results from a preliminary comparison of the new GOSAT-2-FOCAL data with TCCON. These results are considered less reliable because they are based on a data set which covers less than a year. There are sufficient collocations for only seven TCCON stations, and some of them comprise only a few months of data, which limits the fit of our trend model. The mean scatter of the GOSAT-2 data is 1.86 ppm and therefore similar to the one for GOSAT. The derived station-to-station bias for GOSAT-2-FOCAL is 1.14 ppm. This high value is mainly due to the biases derived for the stations Orleans and Réunion Island (where few data are available) as well as the station Pasadena, which also showed larger discrepancies to the GOSAT products (see Fig. 16). We are confident that the station-to-station bias will improve when more and improved GOSAT-2 data are available.

Figure 17 Time series of collocated FOCAL GOSAT-2 data at various TCCON sites.

Figure 18 Overview of TCCON validation results for the GOSAT-2-FOCAL product.

Via the TCCON comparison it is also possible to validate the reported precision of the FOCAL data products (i.e. the specified XCO₂ error). The basic idea is to estimate the true precision from the variability of the XCO₂ bias relative to trend-corrected, collocated TCCON data. For this purpose, we define 20 bins with increasing reported XCO₂ uncertainty and compute the corresponding true precision from the scatter relative to TCCON (i.e. the fit residual mentioned above).

The corresponding scatter plot is shown in Fig. 19. We use the fitted linear curve to correct the reported uncertainty of the GOSAT-FOCAL data. After the correction, all data scatter around the 1:1 line (dashed).

Figure 19 Comparisons of the (binned) original XCO₂ errors (reported precision without correction) of the GOSAT-FOCAL product with estimated errors based on collocated TCCON data (true precision).

A similar correction will be performed for the GOSAT-2-FOCAL product as soon as sufficient data (GOSAT-2 and TCCON) are available, which is currently not the case.

5.3 Time series

To investigate the temporal behaviour of the FOCAL XCO₂ data sets, we performed comparisons based on monthly data from 2009 to 2019, which were spatially gridded to $\mathrm{5}{}^{\circ }\times \mathrm{5}{}^{\circ }$ (examples are shown in Figs. 11 and 12). Similar data sets have been generated for the SRON, UoL, ACOS, and NIES GOSAT products. We also produced a corresponding gridded GOSAT BESD data set; since these are near-real-time (NRT) data only, there are no GOSAT BESD data before 2014 available (when the NRT processing started). GOSAT-2 data start in February 2019.

We then selected grid points for which the standard error of the mean is less than 1.6 ppm for each combination of GOSAT-FOCAL XCO₂ and a correlative data set (as a basic quality filter, similar as done by Reuter et al., 2020). These data were then averaged over different latitudinal ranges, namely

• global (90^∘ S–90^∘ N),

• the Northern Hemisphere (25–90^∘ N),

• the tropics (25^∘ S–25^∘ N), and

• the Southern Hemisphere (25–90^∘ S).

Figure 20 shows the results of these comparisons. The left plots display time series of the different data sets and the right plots the difference between the GOSAT-FOCAL XCO₂ and the reference data. All data products reproduce the overall increase in XCO₂ with time and the seasonal variations. On average, FOCAL data are typically about 0.5 ppm higher than the other data sets (except for BESD). This is related to the choice of the true XCO₂ for the bias correction. There are long-term changes of the order of 1 ppm over the complete time series, which differ for each data set. For example, the GOSAT-FOCAL data show in the tropics relative to SRON a higher value at the start of the time series, but both data sets agree at the end. On the other hand, the average difference to the UoL data in the Northern Hemisphere is negative during the first years, but it increases to an almost constant small positive offset below about 0.5 ppm. There is not much difference in the temporal behaviour between the GOSAT-FOCAL and the ACOS and NIES time series. The seasonal shapes also differ slightly, with amplitudes of about 0.5 ppm and somewhat larger differences in the Southern Hemisphere where seasonal variations are generally smaller.

Figure 20 Gridded monthly mean time series of different GOSAT XCO₂ products. (a, c, e, g) Time series of mean XCO₂ for four different regions (from top to bottom: global, Northern Hemisphere, tropics, Southern Hemisphere). (b, d, f, h) Corresponding differences to the GOSAT-FOCAL product.

Overall, the agreement within the GOSAT data sets is broadly consistent with the systematic regional and seasonal biases derived from the TCCON validation, especially considering that all gridded data sets are based on a different spatial and temporal sampling. Also, the FOCAL products for GOSAT and GOSAT-2 seem to agree quite well, but more GOSAT-2 data are needed to confirm this.

6 Conclusions

Based on the FOCAL retrieval method a new XCO₂ data set for GOSAT and a first XCO₂ data set for GOSAT-2 have been generated, making use of both measured polarisation directions. The GOSAT-FOCAL data set compares well with corresponding data from other currently available GOSAT retrieval algorithms, i.e. the RemoTeC product from SRON, the UoL FP product, the NASA ACOS product, the NIES product, and the BESD product from IUP. All data sets use different filtering and bias correction schemes and therefore also comprise a different number and sampling of data. The GOSAT-FOCAL product performs well in this context and has almost as many valid data as the ACOS product. Based on gridded data, differences in long-term variations of all data sets of the order of 1 ppm per decade are observed. Also, seasonal variations differ by about 0.5 ppm.

Comparisons with ground-based TCCON data reveal for the GOSAT-FOCAL product an overall station-to-station bias of 0.56 ppm and a mean scatter of 1.89 ppm. These values are comparable to and in some cases even better than those of the already existing GOSAT products, some of which have less valid data.

The first GOSAT-2 results using the FOCAL method are also quite promising, but further investigations, longer time series, and more correlative data sets are required for a quantitative assessment of the GOSAT-2-FOCAL data quality.

Overall, the FOCAL method has proven to be computationally fast and to produce XCO₂ results with similar accuracy as other, typically more time-consuming, retrieval algorithms. This is the case not only when applied to OCO-2 but also for GOSAT and GOSAT-2. FOCAL is therefore considered to be a good candidate algorithm for future satellite sensors producing large amounts of data, like the forthcoming European anthropogenic CO₂ Monitoring (CO2M) mission.