Synergetic use of IASI profile and TROPOMI total-column level 2 methane retrieval products

The thermal infrared nadir spectra of IASI (Infrared Atmospheric Sounding Interferometer) are successfully used for retrievals of different atmospheric trace gas profiles. However, these retrievals offer generally reduced information about the lowermost tropospheric layer due to the lack of thermal contrast close to the surface. Spectra of scattered solar radiation observed in the near-infrared and/or shortwave infrared, for instance by TROPOMI (TROPOspheric Monitoring Instrument), offer higher sensitivity near the ground and are used for the retrieval of total-column-averaged mixing ratios of a variety of atmospheric trace gases. Here we present a method for the synergetic use of IASI profile and TROPOMI total-column level 2 retrieval products. Our method uses the output of the individual retrievals and consists of linear algebra a posteriori calculations (i.e. calculation after the individual retrievals). We show that this approach has strong theoretical similarities to applying the spectra of the different sensors together in a single retrieval procedure but with the substantial advantage of being applicable to data generated with different individual retrieval processors, of being very time efficient, and of directly benefiting from the high quality and most recent improvements of the individual retrieval processors.

We demonstrate the method exemplarily for atmospheric methane (CH₄). We perform a theoretical evaluation and show that the a posteriori combination method yields a total-column-averaged CH₄ product (XCH₄) that conserves the good sensitivity of the corresponding TROPOMI product while merging it with the high-quality upper troposphere–lower stratosphere (UTLS) CH₄ partial-column information of the corresponding IASI product. As a consequence, the combined product offers additional sensitivity for the tropospheric CH₄ partial column, which is not provided by the individual TROPOMI nor the individual IASI product. The theoretically predicted synergetic effect is verified by comparisons to CH₄ reference data obtained from collocated XCH₄ measurements at 14 globally distributed TCCON (Total Carbon Column Observing Network) stations, CH₄ profile measurements made by 36 individual AirCore soundings, and tropospheric CH₄ data derived from continuous ground-based in situ observations made at two nearby Global Atmospheric Watch (GAW) mountain stations. The comparisons clearly demonstrate that the combined product can reliably detect the actual variations of atmospheric XCH₄, CH₄ in the UTLS, and CH₄ in the troposphere. A similar good reliability for the latter is not achievable by the individual TROPOMI and IASI products.

1 Introduction

Measurements from different ground- or satellite-based sensors target at the observations of the same atmospheric parameters (e.g. the same trace gases) but with different characteristics (e.g. sensitivities for different vertical regions). Often the different sensors use different observation geometries (limb scanning, nadir, solar light reflected at the Earth's surface) and/or different spectral regions (e.g. UV–Vis, near-infrared, thermal infrared, and microwave). Dedicated experts and efforts are needed to develop retrieval techniques that are specifically optimized for an individual sensor. An algorithm that uses coincident measurements of all the different sensors for a multispectral approach (synergetic use of level 1 data) for the optimal estimation of the atmospheric state would exploit the synergies of the different observation geometries and spectral regions well and thus allows for detection of the atmospheric state in more detail than is achievable by individual optimal estimation retrievals.

There is a variety of studies investigating the multispectral synergism when retrieving atmospheric trace gases from space. Examples of theoretical studies using synthetic thermal infrared and UV spectra for a simulated synergistic retrieval of atmospheric ozone (O₃) are Landgraf and Hasekamp (2007), Worden et al. (2007), Cuesta et al. (2013), and Costantino et al. (2017). These studies considered the thermal infrared spectra of TES (Tropospheric Emission Spectrometer) and IASI (Infrared Atmospheric Sounding Interferometer as well as its successor IASI – New Generation) and UV spectra of OMI (Ozone Monitoring Instrument) and GOME-2 (Global Ozone Monitoring Experiment – 2 as well as its successor UVNS – Ultraviolet Visible Near-infrared Shortwave-infrared) and are complemented by studies with real spectra (e.g. Cuesta et al., 2013; Fu et al., 2013). Another example of a study with real spectra is Luo et al. (2013), who examine the combination of the TES thermal nadir spectra with the MLS (Microwave Limb Sounder) microwave limb spectra for a synergetic retrieval of atmospheric carbon monoxide (CO) profiles. All the different studies clearly show that the synergetic use of the measured spectra results in an increased sensitivity with respect to the targeted trace gases.

However, the development of these multispectral retrievals requires experts in different remote-sensing techniques to work closely together. Furthermore, as soon as measurements from a new sensor become available (or as soon as sensors are modified/improved), such multispectral processors have to be adapted accordingly; i.e. continuous collaborative retrieval developments are required. While this is certainly possible, it might not be the most efficient way, in particular considering the steadily increasing number of available satellite data products (level 2 retrieval products). The optimal synergetic exploitation of the level 2 retrieval products that are already available would be much less computationally expensive than running dedicated multispectral retrievals. Such synergetic combination of level 2 products is the topic of this paper.

There are already several examples of a level 2 product fusion discussed in literature (the following list is not intended to be complete): Worden et al. (2015) combine the thermal and near-infrared level 2 products of methane (CH₄) of TES and GOSAT (Greenhouse gas mOnitoring SATellite), respectively, by performing approximative calculations and with a focus on monthly mean data. Data aggregation is necessary due to the reduced temporal and horizonal coverage of TES and GOSAT and their imperfect collocation. Cortesi et al. (2016) combine the thermal infrared MIPAS-STR (Michelson Interferometer for Passive Atmospheric Sounding – STRatospheric aircraft) and microwave MARSCHALS (Millimetre-wave Airborne Receivers for Spectroscopic Characterization in Atmospheric Limb Sounding) aircraft-based remote-sensing products of O₃, nitric acid (HNO₃), water vapour (H₂O), and atmospheric temperature (applying the so-called “measurement-space solution” data fusion method of Ceccherini et al., 2009). Another example is Warner et al. (2014), who use a Kalman filter for combining the CO data products of AIRS (Atmospheric Infrared Sounder) – available for a large horizontal area but with weak vertical details – and TES (and MLS) – available with detailed vertical information but only for very localized areas.

Here, we present a method for fusing the available level 2 CH₄ profile product of IASI and the XCH₄ (total-column-averaged methane) product of TROPOMI (Tropospheric Monitoring Instrument) by means of a Kalman filter approach. Our objective is a data product with improved vertical profile information (determine tropospheric CH₄ independently from CH₄ at higher altitudes, which is not possible by IASI or TROPOMI data alone) by synergetically exploiting the different vertical sensitivities of the two products.

The method allows for a computationally very efficient generation of global daily maps of the combined data product and only needs the individually retrieved states, averaging kernels, and noise covariances provided by the respective remote-sensing experts in the context of their standard retrieval work. The proposed method can be used flexibly for combining measurement information of different satellite sensors and is in particular interesting for combining a profile product with a total-column product. The method has strong theoretical similarities to a dedicated combined optimal estimation retrieval that uses the combined IASI and TROPOMI spectra as input (synergetic use of level 1 data).

The reliable and global detection of tropospheric CH₄ independently from CH₄ at higher altitudes can lead to an improved understanding of the CH₄ cycle. Respective data allow for a more direct investigation of the CH₄ boundary layer source and sink signals than total-column-averaged mixing ratios (XCH₄) provided globally for instance by GOSAT (e.g. Parker et al., 2020) or TROPOMI (Lorente et al., 2021a). This is because XCH₄ signals are strongly affected by vertical shifts of the tropopause altitude; i.e. their use for investigating CH₄ absorption and release at the ground depends on the correct consideration of the tropopause altitude by model simulations (Pandey et al., 2016).

This paper is organized as follows. Section 2 briefly presents the used IASI and TROPOMI products (generated by two individual retrievals). Section 3 presents the equations for the optimal a posteriori combination of the two independent retrieval outputs (level 2 product combination) and performs a theoretical evaluation of the individual and combined products. Section 4 validates the total- column and tropospheric and UTLS (upper troposphere–lower stratosphere) partial-column products obtained by the individual IASI and TROPOMI retrievals and by the a posteriori combination by an inter-comparison to reference data from the Total Carbon Column Observing Network (TCCON), AirCore, and Global Atmospheric Watch (GAW). Section 5 discusses the global consistency of the products and shows global maps. Section 6 resumes the results of our study and briefly discusses upcoming possibilities. Furthermore, in Appendix A we give a brief overview on retrieval theory, and in Appendix B we discuss the theory of our a posteriori combination method and show that the method has strong similarities to performing a full multispectral optimal estimation retrieval. Appendix C introduces the operator for transferring logarithmic-scale differentials into linear-scale differentials. Appendix D presents the operators used for converting vertical profile data into total- and partial-column data. Appendix E examines the dislocation error, i.e. to what extent the temporal and spatial dislocation of the IASI and TROPOMI observations (the two sensors are on two different satellites with different orbits) impacts the combined data product. Appendix F explains how we assess the comparability of the satellite products with the reference data and reveals the reasonable agreement between the characteristics of the satellite products and the results of the validation study.

2 Satellite data

In this section we briefly present the satellite data products that are used for the combination procedure. These are the XCH₄ data obtained from the analysis of the near-infrared and shortwave infrared (SWIR and NIR) spectra measured by TROPOMI and the CH₄ profiles derived from IASI thermal nadir (TIR) spectra. In addition, we explain the criteria used for collocating the two satellite observations.

2.1 RemoTeC TROPOMI XCH₄

The TROPOMI XCH₄ data used in this study are generated by the RemoTeC algorithm (Butz et al., 2011), which is used for the operational processing of Sentinel 5 Precursor/TROPOMI XCH₄ data (Hu et al., 2016; Hasekamp et al., 2021). Here we work with data of the operational processing algorithm version 2.2.0 (which has been presented and validated in Lorente et al., 2021a). The TROPOMI output files provide the XCH₄ data together with the a priori data used (constructed from simulations of the global chemistry-transport model TM5; Krol et al., 2005), the column averaging kernels, and the error values. Here we work with all TROPOMI data that pass the standard quality filtering (TROPOMI output variable qa_value must be equal to 1.0, which means a filtering according to Table A1 of Lorente et al., 2021a). In addition, we remove observations over ground covered by snow (which show a high bias as discussed in Lorente et al., 2021a) by requiring a blended albedo (A_b, calculated from the NIR, A_NIR, and SWIR, A_SWIR, albedos according to Wunch et al., 2011b, as $A_b=\mathrm{2.4}{A}_{\mathrm{NIR}}-\mathrm{1.13}{A}_{\mathrm{SWIR}}$) that is smaller than 0.85.

2.2 MUSICA IASI CH₄ profiles

As the IASI CH₄ data product, we use the data generated by the retrieval processor MUSICA (MUlti-platform remote Sensing of Isotopologues for investigating the Cycle of Atmospheric water, a European Research Council project between 2011 and 2016). The MUSICA IASI data full retrieval product encompasses trace gas profiles of H₂O, the HDO$/$H₂O ratio, N₂O, CH₄, and HNO₃. The data have been validated in several previous studies (Schneider et al., 2016; Borger et al., 2018; García et al., 2018), and it has been shown that the CH₄ product can detect the CH₄ signals originating in the upper troposphere–lower stratosphere particularly well. MUSICA IASI data using processor versions 3.2.1 and 3.3.0/1 are currently available for the 2014 to 2021 period and are presented in Schneider et al. (2022). This MUSICA IASI data set is best suited for a posteriori data reusage (e.g. Diekmann et al., 2021) because in addition to the retrieved trace gas profiles, it contains full information on retrieval settings (a priori states and constraints) and on averaging kernel and error covariance matrices. In order to ensure the highest MUSICA IASI data quality, here we require the flag variable musica_fit_quality_flag to be set to 3 (the spectral fit of the MUSICA IASI retrieval has a good quality, and the spectral residuals are close to the instrumental noise level). Furthermore, we only use MUSICA IASI data for which the flag variable eumetsat_cloud_summary_flag is set to 1, which guarantees that the IASI instrumental field of view is cloud-free.

A particularity of the MUSICA IASI processor is that the trace gas inversions are performed on a logarithmic scale. In Appendix B of Schneider et al. (2022), it is shown that the MUSICA IASI retrieval can be considered a moderately non-linear problem, in particular if the differentials (averaging kernels and covariances) are used on the logarithmic scale. In the following equations, we take special care with the correct usage of the corresponding logarithmic-scale differentials. Nevertheless, all equations are also applicable for retrievals done on a linear scale by replacing in the following the operator L (which is introduced in Appendix C) by the identity operator.

2.3 Collocation of TROPOMI and IASI observations

As a temporal collocation criterion we use 6 h, for a valid horizontal collocation the centres of the TROPOMI and IASI ground pixels must be closer than 50 km, and the difference between the ground pressure at the TROPOMI and IASI ground pixels must be within 50 hPa. Generally multiple TROPOMI–IASI ground pixel pairs fulfil the aforementioned criteria. In such cases, we use the pair with the smallest distance metric. This metric is defined as the Euclidean distance that considers a norm of 12 h for the temporal distance, a norm of 50 km for the horizontal distance, and a norm of 5 hPa for the vertical distance. The possible small difference in the TROPOMI and IASI ground pixel pressures is taken into account by correcting the TROPOMI XCH₄ values according to Appendix B of Sha et al. (2021).

3 Presentation and characterization of the combination method

3.1 Calculation of the combined state vector

For this study we use the CH₄ a priori profile as provided by the TROPOMI product as the common a priori profile for all products (these are simulations of the global chemistry-transport model TM5; Krol et al., 2005). For this purpose we modify the MUSICA IASI product and bring it in line with the TROPOMI a priori profile choice by applying Eq. (B13).

For updating the IASI CH₄ profile product using the TROPOMI XCH₄ observation, we apply a Kalman filter and obtain the combined CH₄ state as

$$\begin{array}{}\text{(1)}& {\widehat{\mathit{x}}}_{\mathit{C}}^{\mathbf{l}}={\widehat{\mathit{x}}}_{\mathit{I}}^{\mathbf{l}}+{\mathbf{L}}^{-\mathrm{1}}\mathit{m}[{\widehat{x}}_T^{*}-{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T{\widehat{\mathit{x}}}_{\mathit{I}}]-{\mathbf{L}}^{-\mathrm{1}}\mathit{m}({{\mathit{w}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T){\mathit{x}}_{\mathit{a}}.\end{array}$$

Here the vector ${\widehat{\mathit{x}}}_{\mathit{I}}$ and scalar ${\widehat{x}}_T^{*}$ are the MUSICA IASI CH₄ profile and the TROPOMI XCH₄ column-averaged products. The row vector ${{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T$ is the total-column-averaged mixing ratio kernel of the TROPOMI product interpolated to the vertical grid used by the MUSICA IASI processor, and the row vector ${{\mathit{w}}^{\mathbf{*}}}^T$ is the operator for converting mixing ratio vertical profiles into total-column-averaged mixing ratios (for details on the interpolation, see Appendix D). The state vector ${\widehat{\mathit{x}}}_{\mathit{C}}^{\mathit{l}}$ represents the logarithmic-scale combined CH₄ profile product (i.e. the MUSICA IASI CH₄ data updated with the TROPOMI XCH₄ observation). The superscript “^l” used with ${\widehat{\mathit{x}}}_{\mathit{C}}^{\mathbf{l}}$ and ${\widehat{\mathit{x}}}_{\mathit{I}}^{\mathbf{l}}$ indicates the use of the logarithmic scale. Here and in the following, we will mark scalars, vectors, or matrix operators that are in logarithmic scale by the superscript “^l”. The matrix L is the operator for the transformation of differentials or small changes (as given by averaging kernels or error covariances) from the logarithmic to the linear scale (for more details, see Appendix C).

The column vector m is the Kalman gain operator, and it is given by

$$\begin{array}{}\text{(2)}& \mathit{m}={\mathbf{LS}}_{{\widehat{\mathbf{x}}}_{\mathbf{I}}}^{\mathrm{l}}{\mathbf{L}}^T{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}({{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T{\mathbf{LS}}_{{\widehat{\mathbf{x}}}_{\mathbf{I}}}^{\mathrm{l}}{\mathbf{L}}^T{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}+S_{{\widehat{x}}_T,n}^{*}{)}^{-\mathrm{1}},\end{array}$$

with the matrix ${\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{I}}}^{\mathrm{l}}$ and the scalar $S_{{\widehat{x}}_T,n}^{*}$ being the logarithmic-scale a posteriori covariance of the MUSICA IASI CH₄ product and the noise error variance of the TROPOMI XCH₄ product, respectively. The vector operator ${\mathit{a}}_{\mathit{T}}^{\mathbf{*}}$ is the transpose of the TROPOMI column averaging kernel; i.e. ${\mathit{a}}_{\mathit{T}}^{\mathbf{*}}=({{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T{)}^T$.

Except for the logarithmic-scale transformation, Eqs. (1) and (2) are analogous to Eqs. (B9) and (B10). As demonstrated in Appendix B, this kind of Kalman filter application has a strong similarity to an optimal estimation retrieval that uses a combined IASI and TROPOMI measurement vector (synergetic use of level 1 data). The application of this Kalman filter is possible because the MUSICA IASI data are provided with full information on a priori states, constraints, error covariances, and averaging kernels (Schneider et al., 2022), and because the TROPOMI data are provided together with their a priori state, averaging kernel, and retrieval noise error (Lorente et al., 2021a).

The Kalman gain according to Eq. (2) describes how differences between the MUSICA IASI and TROPOMI XCH₄ product are used to update the MUSICA IASI CH₄ profile. An example for a Kalman gain operator is depicted in Fig. 1. It shows that a positive difference of +1 ppb of $[{\widehat{x}}_T^{*}-{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T{\widehat{\mathit{x}}}_{\mathit{I}}]$ will lead to a combined CH₄ profile product that has been modified with respect to the MUSICA IASI CH₄ product by almost +3 ppb in the lowermost troposphere, by about −0.5 ppb at 10 km, and by about +1.5 ppb above 20 km.

Figure 1 Visualization of a Kalman gain operator for optimally combining TROPOMI XCH₄ data with MUSICA IASI CH₄ profile data. This is the column vector m according to Eq. (2). The example shown is for a late summer atmosphere (27 September 2018) over central Europe.

3.2 Vertical resolution and representativeness

In this section, we compare the vertical resolution and representativeness of the individual retrieval products with those achieved when combining the two retrieval products. According to Eq. (1), the averaging kernels for the combined data product can be calculated as

$$\begin{array}{}\text{(3)}& {\mathbf{A}}_{\mathbf{C}}^{\mathrm{l}}={\mathbf{A}}_{\mathbf{I}}^{\mathrm{l}}+{\mathbf{L}}^{-\mathrm{1}}\mathit{m}({{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T{\mathbf{LA}}_{\mathbf{I}}^{\mathrm{l}}{\mathbf{L}}^{-\mathrm{1}})\mathbf{L}.\end{array}$$

Here, ${\mathbf{A}}_{\mathbf{I}}^{\mathrm{l}}$ and ${\mathbf{A}}_{\mathbf{C}}^{\mathrm{l}}$ are the logarithmic-scale averaging kernels of the MUSICA IASI CH₄ product and of the combined product (the MUSICA IASI CH₄ product after being updated with the information provided by the TROPOMI XCH₄ product), respectively. These are the kernels for the profile products represented in the number of atmospheric levels (nal); i.e. they are matrices of dimension nal × nal. Logarithmic-scale averaging kernels are also called fractional or relative averaging kernels (e.g. Keppens et al., 2015).

Figure 2 Logarithmic-scale row kernels for (a) the MUSICA IASI and (b) the combined product for the same late summer observations as used in the context of Figs. 1 and 3. The symbols mark the kernel values at the nominal altitude.

Figure 2 depicts the rows of typical averaging kernels for the MUSICA IASI product (Fig. 2a) and the combined data product (Fig. 2b). Adding the information provided by TROPOMI clearly improves the sensitivity in the lower troposphere: for the MUSICA IASI product, the lower tropospheric kernels generally peak at the upper limit of the lower troposphere (at about 5 km a.s.l.). For the combined product, these peak values are obtained at significantly lower altitudes (at about 2 km a.s.l.). In the UTLS, we see no significant difference between the kernels.

Figure 3 Total-column and partial-column amount kernels corresponding to the TROPOMI, MUSICA IASI, and the combined product for the same late summer observation as used in Figs. 1 and 2: (a) total-column amount kernels; (b) lower tropospheric partial-column amount kernels, surface − 6 km a.s.l.; and (c) upper troposphere–lower stratosphere (UTLS) partial-column amount kernels, 6–20 km a.s.l.

In this work we focus on the total-column and the partial columns between the surface and 6 km a.s.l. (the tropospheric partial column) and between 6 and 20 km a.s.l. (the UTLS partial column). The total- and partial-column kernels are calculated from ${\mathbf{A}}_{\mathbf{I}}^{\mathrm{l}}$ and ${\mathbf{A}}_{\mathbf{C}}^{\mathrm{l}}$ by their transformation on a linear scale (see Appendix C) and the vertical resampling as explained in Appendix D. Figure 3 depicts the total- and partial-column kernels corresponding to the row kernels of Fig. 2.

Total-column amount kernels are available for all three products (see Fig. 3a): the TROPOMI, the MUSICA IASI, and the combined product. The TROPOMI kernel is close to unity for all altitudes, documenting the good sensitivity for CH₄ at all altitudes. The combined total-column amount kernel is even closer to unity than the respective TROPOMI kernel, which means that the combined retrieval product does also reflect the actual atmospheric total-column amounts well. The MUSICA IASI kernel has relatively low values in the lower troposphere and above 15 km; only in the UTLS region are the kernel values between 0.75 and 1.25. This means that MUSICA IASI can actually not detect total-column amounts well because it lacks sensitivity in the lower troposphere. The altitude regions where the MUSICA IASI product has reduced sensitivities are the regions where TROPOMI's total-column information has the strongest impact on the combined product (see Fig. 1).

Partial-column amount kernels are only available for profile products, i.e. the MUSICA IASI and the combined product (MUSICA IASI updated with information from TROPOMI). Figure 3b shows tropospheric partial-column amount kernels. For the MUSICA IASI product, we observe values that are generally lower than 0.5. The highest values are achieved around 6 km a.s.l., i.e. at the upper boundary of the vertical layer we defined as the tropospheric partial column. The kernel of the combined product shows a good sensitivity, with peak values of almost 0.95 at 2.5 km a.s.l. and values above 0.75 for almost all altitudes between the surface and 6 km a.s.l.

UTLS partial-column amount kernels are depicted in Fig. 3c. The values are closest to unity for the altitudes that we attributed to the UTLS layer (altitudes between 6 and 20 km a.s.l.). There is almost no difference between the MUSICA IASI and the combined kernels, meaning that the information provided by TROPOMI has almost no effect on the UTLS partial column, which is because the MUSICA IASI product is already very sensitive to this altitude region.

Figure 4 Time series of the degree of freedom for signal (DOFS; example for central Europe). Black squares: TROPOMI (please note that only the total-column data are made available); red dots: MUSICA IASI; blue crosses: combined product.

The example kernels document that the combined product allows for detection of tropospheric CH₄ largely independent from CH₄ in the UTLS, which is not possible by the IASI product alone. Figure 4 shows a time series of the degree of freedom for signal (DOFS; it is calculated as the trace of the averaging kernels and is a measure for the profiling capability; Rodgers, 2000). It documents that the combination of TROPOMI with IASI improves the profiling capability of IASI rather consistently throughout all seasons. Here we also show the DOFS values of the TROPOMI retrieval, but please note that only the total-column data are made available; i.e. there is no profile information in the provided TROPOMI CH₄ data product.

An optimal estimation retrieval updates the a priori knowledge with information provided by a measurement. The a posteriori uncertainty is the uncertainty achieved by optimally combing the a priori knowledge (captured by the inverse of the a priori covariance matrix, i.e. ${{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}$) with the measurement. As shown in Appendix A, the a posteriori uncertainty covariance is the sum of the noise covariance and the representativeness error covariance (called “smoothing error” covariance in Rodgers, 2000).

According to Eq. (A7) the representativeness error matrix is calculated from the averaging kernel (A) and the a priori covariance (S_a) as

$$\begin{array}{}\text{(4)}& {\mathbf{S}}_{\widehat{\mathbf{x}},\mathbf{r}}^{\mathrm{l}}=({\mathbf{A}}^{\mathrm{l}}-\mathbf{I}){\mathbf{S}}_{\mathbf{a}}^{\mathrm{l}}({\mathbf{A}}^{\mathrm{l}}-\mathbf{I}{)}^T,\end{array}$$

with I being the identity operator. By using the kernels ${\mathbf{A}}_{\mathbf{I}}^{\mathrm{l}}$ and ${\mathbf{A}}_{\mathbf{C}}^{\mathbf{l}}$, we can calculate the representativeness error for the MUSICA IASI and the combined product, respectively. The resampling of ${\mathbf{S}}_{\widehat{\mathbf{x}},\mathbf{r}}^{\mathrm{l}}$ on total and partial columns is done according to Eq. (D7). For the TROPOMI total-column-averaged mixing ratios we can calculate the representativeness error by $({{\mathit{w}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T){\mathbf{S}}_{\mathbf{a}}({{\mathit{w}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T{)}^T$. For more details, see Appendix D.

Figure 5 depicts the representativeness error relative to the retrieved values for the total column and the tropospheric and UTLS partial columns. Shown are time series for measurements over central Europe, which confirm the observations made in the context of the example kernels of Fig. 3: for the total column (Fig. 5a) the representativeness error on the TROPOMI and the combined products is rather small and can be neglected; i.e. both products can detect total-column signals. In contrast the MUSICA IASI representativeness error is much larger, and the respective data do not represent the total column well; i.e. they provide no independent observations of the total column. Concerning partial-column products (Fig. 5b and c), we can compare the MUSICA IASI and the combined product (the TROPOMI product has no information on the vertical distribution). The tropospheric MUSICA IASI partial column has a significant representativeness error (and a seasonal cycle with highest values of about 3 % in winter). In the combined product, this error is generally smaller than 1 % throughout all seasons. In the UTLS both the MUSICA IASI and combined products are strongly representative of the actual atmospheric methane concentration signals (representativeness error is generally between 0.5 % and 1 %). In summary, TROPOMI only provides total-column data. With IASI alone, we can detect signals in the UTLS well but not in the lower troposphere. The detection of signals in both altitude regions independently from the a priori information is only possible using the combined product.

Figure 5 Time series of the representativeness error (example for central Europe). Black squares: TROPOMI; red dots: MUSICA IASI; blue crosses: combined product. (a) Total column, (b) tropospheric partial column, and (c) UTLS partial column.

3.3 Retrieval noise error

After documenting the representativeness error in the previous subsection, here we investigate the retrieval noise error. We compare the retrieval noise errors of the individual retrieval products with those achieved when combining the two retrieval products. According to Eq. (1), we can calculate the retrieval noise covariance matrix for the combined data product by

$$\begin{array}{}\text{(5)}& \begin{array}{rl}{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{C}},\mathbf{n}}^{\mathbf{l}}& =(\mathbf{I}-{\mathbf{L}}^{-\mathrm{1}}\mathit{m}{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T\mathbf{L}){\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{I}},\mathbf{n}}^{\mathbf{l}}(\mathbf{I}-{\mathbf{L}}^{-\mathrm{1}}\mathit{m}{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T\mathbf{L}{)}^T\\ & +\left({\mathbf{L}}^{-\mathrm{1}}\mathit{m}\right)S_{{\widehat{x}}_T^{*},n}({\mathbf{L}}^{-\mathrm{1}}\mathit{m}{)}^T.\end{array}\end{array}$$

Here ${\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{I}},\mathbf{n}}^{\mathrm{l}}$ is the retrieval noise covariance matrix of the MUSICA IASI retrieval. The error covariances resampled to the total and partial columns are then determined according to Appendix D. Figure 6 shows the retrieval noise errors (which are the square root values of the error variances) relative to the retrieved values for the total column and the tropospheric and UTLS partial columns.

Figure 6 Time series of estimated relative noise error for the retrieved products (example for central Europe). Colours are as in Fig. 5. (a) Total column, (b) tropospheric partial column, and (c) UTLS partial column.

The errors for the total columns (Fig. 6a) are generally below 0.2 % for the TROPOMI product. For the MUSICA IASI product, they are rather stable at about 0.6 %. Concerning the combined product, the retrieval noise error is very similar to the retrieval noise error of the TROPOMI data.

For the tropospheric partial columns (Fig. 6b), the error is in general above 1 % for the MUSICA IASI product and below 1 % for the combined product. For the UTLS partial columns (Fig. 6c), we observe an error of generally below 1 % and no significant difference between the MUSICA IASI and the combined data products. This suggests that the error in the combined product is dominated by the error in the MUSICA IASI data, which reveals the very limited impact of the TROPOMI data on the combined UTLS data product.

3.4 Dislocation error

As mentioned in Sect. 2.3, we allow for small dislocations between the TROPOMI and IASI observations of up to 6 h and 50 km. As derived in Appendix E, the dislocation error covariance matrix is calculated by

$$\begin{array}{}\text{(6)}& {\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{C}},\mathrm{dl}}^{\mathrm{l}}={\mathbf{A}}_{\mathbf{C},\mathrm{dl}}^{\mathrm{l}}{\mathbf{S}}_{{\mathbf{\Delta }}_{\mathrm{dl}}}^{\mathrm{l}}{{\mathbf{A}}_{\mathbf{C},\mathrm{dl}}^{\mathrm{l}}}^T,\end{array}$$

where ${\mathbf{A}}_{\mathbf{C},\mathrm{dl}}^{\mathrm{l}}$ is the dislocation kernel, and ${\mathbf{S}}_{{\mathbf{\Delta }}_{\mathrm{dl}}}^{\mathrm{l}}$ is the covariance matrix for the CH₄ dislocation uncertainty, whose main characteristics are visualized in Figs. E1 and E2. The low entry of the dislocation kernel at low altitudes (for a typical example, see Fig. E4) reduces the impact of the spatial and temporal dislocation on the total and tropospheric partial columns of the combined product.

Over central Europe, we estimate an error in the combined product due to the dislocations between IASI and TROPOMI as shown in Fig. 7. For the total column, the error is below 0.1 %, and for the tropospheric and UTLS partial columns, it is generally below 0.8 %. If compared to the noise error (see Fig. 6), the dislocation error is of secondary importance. Details on the estimation of these dislocation errors and examples for other locations are given in Appendix E.

Figure 7 Time series of estimated relative dislocation error for the combined product due to temporal and spatial dislocation of the TROPOMI and IASI satellite ground pixels (example for central Europe). (a) Total column, (b) tropospheric partial column, and (c) UTLS partial column.

4 Validation

In this section we empirically evaluate the quality of the TROPOMI, MUSICA IASI, and combined products by their inter-comparison to different reference data products. As reference for the total-column-averaged mixing ratio (XCH₄), we use TCCON (Total Carbon Column Observing Network; Wunch et al., 2011a) ground-based remote-sensing data from 14 sites located in different climate zones. As reference for the total and the partial columns, we use in situ profiles measured by the AirCore system (Karion et al., 2010) at two geophysically different European locations. Furthermore, we use in situ data measured at two nearby central European Global Atmospheric Watch (GAW) mountain stations.

Figure 8 depicts the geographical location of the reference observations. Figure 8a shows that the considered TCCON stations are distributed around the globe (more detailed information on these sites is given in Table 1). Figure 8b gives details on the different European reference sites and the areas accepted for a valid collocation. For collocation with TCCON, the satellite ground pixels should fall within a circle around the stations with a radius of 100 km (red crosses and circles). For the comparison to the GAW data, the collocation circle has a radius of 150 km (grey circle) and is centred in the middle of the two GAW stations (Jungfraujoch in Switzerland and Schauinsland in south-western Germany, indicated by the grey dots). For the comparison with AirCore, we relax the radius of the collocation circle to 500 km in order to achieve a sufficient high number of coincidences between AirCore and satellite observations. The two AirCore sites (Trainou in France and Sodankylä in Finland) and the collocation circles are indicated by the blue stars and circles.

Figure 8 Maps showing the location of the reference measurements used for the validation. (a) Global map with the 14 TCCON stations (more detailed information on these sites is given in Table 1). (b) Map indicating the areas accepted for valid horizontal collocations in the surroundings of the European reference stations: TCCON sites and the 100 km collocation radius (red crosses and circles), AirCore sites and the spatial collocation circles with 500 km radius (blue stars and circles), and GAW sites and the collocation circle with 150 km radius (grey dots and circle).

Appendix F reveals that the following validation results are in reasonable agreement with sensitivities and errors of the different satellite data products as shown in Sects. 3.2–3.4.

4.1 TCCON XCH₄

We use TCCON ground-based remote-sensing data from 14 sites located in different climate zones representative of high, middle, and low latitudes. Details on the locations of these sites, the respective data amounts, and references are given in Table 1. We use the TROPOMI a priori setting for the comparison between the ground-based TCCON and the satellite-based remote-sensing products. For this purpose the TCCON product is adjusted to the TROPOMI a priori settings according to Eq. (B13), which ensures the usage of the same a priori data for all the remote-sensing products. As spatial collocation criteria, we require the TROPOMI and IASI ground pixels to be located within 100 km of the TCCON station (where we consider the viewing direction of the TCCON spectrometer by using as location, the TCCON's line of sight {latitude, longitude} at 5 km altitude). Differences in the satellite and TCCON ground pressures are taken into account by correcting the TCCON XCH₄ values according to Appendix B of Sha et al. (2021). For collocation with respect to time, as TCCON reference we use the median XCH₄ value calculated from all TCCON data measured within 2 h of the TROPOMI observation. Furthermore, we require stable conditions for atmospheric CH₄. This is achieved by performing the comparisons only if there are at least three individual TCCON observations that fulfil the collocation criterion, if the timestamps of these observations have a 1σ standard deviation of 1 h, and if the 1σ standard deviation of the respective XCH₄ data is smaller than 0.5 %.

Strong et al. (2019)Kivi et al. (2014)Kivi and Heikkinen (2016)Wunch et al. (2018)Hase et al. (2015)Warneke et al. (2019)Wennberg et al. (2017)Morino et al. (2018)Wennberg et al. (2016)Iraci et al. (2016)Kawakami et al. (2014)Velazco et al. (2017)Griffith et al. (2014a)Griffith et al. (2014b)Sherlock et al. (2014)Pollard et al. (2019)

Table 1 Locations of TCCON sites, amount of the satellite data compared to TCCON, and references. “Number” gives the total number of single satellite footprint collocations and “Days” the number of days with collocations.

Figure 9 Comparison of the different XCH₄ satellite products with TCCON XCH₄ data from 14 globally representative stations (the different colours correspond to the stations as shown in Fig. 8a). Data for all individual coincidences are plotted in the background as squares, and daily mean data are depicted as crosses with error bars representing the 1σ standard deviation (daily means are only calculated if there are at least three observations per day): (a–c) time series of the differences. (d–f) Correlations between TCCON and satellite data (the black line is the one-to-one diagonal). (g–i) Correlations between the a priori free TCCON data (${\mathrm{\Delta }}_a\mathrm{TCCON}={\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{TCCON}\right)-{\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{a}\mathrm{priori}\right)$) and the a priori free satellite data (${\mathrm{\Delta }}_a\mathrm{Satellite}={\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{Satellite}\right)-{\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{a}\mathrm{priori}\right)$). The inserted text reports median and scatter (hIPR68.2; a–c) and the coefficients of determination, the slope, and the intercept of the robust linear regression model (R², m, and b; d–i). Black and grey fonts represent the values for the daily mean data and for data from all individual collocations, respectively.

In Fig. 9 the TROPOMI, MUSICA IASI, and combined XCH₄ products are compared to the TCCON XCH₄ data. The crosses represent the daily mean data, and the filled symbols in the background show all data corresponding to all individual valid collocations (between all single pixel satellite observations and individual TCCON observations). Figure 9a–c show time series of the differences with respect to the TCCON references. The daily mean data have error bars, which is the 1σ standard deviation of the data used for calculating the daily mean.

Statistics in form of the median of the difference and the scatter around the median difference are given in each panel (for statistics using daily mean data in black fonts and for statistics using all individual valid collocations in grey fonts). Here we use the median in order to be less affected by outliers. For the same reason, as metric for the scatter we use the half inter-percentile range between the 15.9 and 84.1 percentiles (hIPR68.2, which is analogous to the 1σ standard deviation in case of a pure Gaussian distribution). Concerning TROPOMI (Fig. 9a), we observe a good agreement. For daily mean data as well as for the statistics based on all individual differences, the median difference is within 0.1 %, and the scatter lies below 0.7 %.

A similar good agreement and low values for median difference and scatter are also achieved for the combined product (Fig. 9c). For the MUSICA IASI product (Fig. 9b), we have reduced sensitivity in the lower troposphere (see Figs. 3 and 5). Because of uncertainties in the a priori assumptions, the agreement with the TCCON XCH₄ data is weaker (uncertainties in the a priori assumption are less well detected; see Fig. F2). We observe no significant systematic negative or positive difference for the satellite versus TCCON comparisons; i.e. the satellite data sets seem to be in good absolute agreement with TCCON. In general the observed scatter values are within the range that can be expected from the data uncertainties and the data comparability (for more details, see Appendix F).

Figure 9d–f depict the correlation plots. In order to reduce the effect of outliers, we apply a robust linear regression model (the iteratively reweighted least-squares algorithm with Tukey's bisquare weight function and the respective tuning parameter set to the commonly used value of 4.685). For daily mean data the obtained coefficients of determination (R²) are larger than 80 % for the TROPOMI and the combined product. The slope of the obtained linear regression line is very close to unity. When considering all individual coincidences, the R² values are about 70 %. The error bars on the daily mean data are the 1σ standard deviations of the data used for calculating the daily mean. For the MUSICA IASI product, we observe a similar good correlation as for the TROPOMI and the combined products. However, concerning the MUSICA IASI data, part of the common signal might be due to the a priori data on which the MUSICA IASI total-column product is not independent (the MUSICA IASI data have a reduced sensitivity, i.e. an increased representativeness error; see Fig. 5a).

Figure 9g–i reveal the information gained by the satellite data with respect to the a priori data. The correlation is depicted between the collocated a priori free TCCON data (${\mathrm{\Delta }}_a\mathrm{TCCON}={\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{TCCON}\right)-{\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{a}\mathrm{priori}\right)$) and the a priori free satellite data (${\mathrm{\Delta }}_a\mathrm{Satellite}={\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{Satellite}\right)-{\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{a}\mathrm{priori}\right)$). The same data as in Fig. 9d–f are shown but with the a priori knowledge removed. We find that the TROPOMI and the combined data product add a significant amount of information to the a priori knowledge (R² values for the respective linear correlations of above 32 %). This information gain is much smaller in the case of the MUSICA IASI data, which confirms that the good correlation as observed in Fig. 9e is to a large extent due to the good quality of the a priori data.

4.2 AirCore in situ CH₄ profiles

We use the AirCore balloon-borne in situ measurements (Karion et al., 2010) as the reference for CH₄ total columns as well as for the CH₄ vertical distribution. The AirCore system samples the vertical distribution of CH₄ with a much better vertical resolution than the satellite remote-sensing systems. For this reason, we can generate an AirCore profile (${\widehat{\mathit{x}}}_{\mathrm{AC}}$) that has the same vertical sensitivity and resolution characteristics as the remote-sensing data. According to Eqs. (A2) and (A4), for the MUSICA IASI and the combined retrieval data we can write

$$\begin{array}{}\text{(7)}& {{\widehat{\mathit{x}}}^{\mathbf{l}}}_{\mathrm{AC}}={\mathit{x}}_{\mathit{a}}^{\mathbf{l}}+{\mathbf{A}}^{\mathrm{l}}({{\mathit{x}}^{\mathbf{l}}}_{\mathrm{AC}}-{\mathit{x}}_{\mathit{a}}^{\mathbf{l}}).\end{array}$$

Here A^l and ${\mathit{x}}_{\mathit{a}}^{\mathbf{l}}$ are the logarithmic-scale averaging kernels and the logarithmic-scale a priori state of the satellite retrieval, respectively, and ${{\mathit{x}}^{\mathbf{l}}}_{\mathrm{AC}}$ is the measured logarithmic-scale AirCore profile regridded to the atmospheric model grid used for the satellite retrievals. The resampling of these data on total and partial columns is done with the linear-scale data according to Eq. (D5). For the TROPOMI total-column-averaged mixing ratios, we calculate the adjusted AirCore total-column-averaged mixing ratio (a scalar) by ${\widehat{x}}_{\mathrm{AC}}^{*}={{\mathit{w}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}}+{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T({\mathit{x}}_{\mathrm{AC}}-{\mathit{x}}_{\mathit{a}})$. For more details, see Appendix D.

As spatial collocation criteria, we require that the ground pixels of the TROPOMI and IASI measurement fall within a circle with a radius of 500 km around the mean horizontal location of the AirCore system when sampling between the 450 and 550 hPa pressure levels. The temporal collocation requirements for both satellite observations is 6 h. AirCore data are typically not available close to the ground and above the burst altitude of the balloon (approximately 25 hPa). At low altitudes we extend the profile with the concentrations closest to the ground. At high altitudes we extend the profile with the TM5 model data, with a smooth transition between the measured values and the modelled data.

Table 2 gives an overview of the satellite data amount compared to the AirCore profiles measured at Trainou (France; 48.0^∘ N, 2.1^∘ E) and Sodankylä (Finland; 67.4^∘ N, 26.6^∘ E). In total we have 36 individual AirCore profiles measured on 31 different days for which collocated satellite observations exist. The total number of collocated single pixel satellite observations is 34 784. We estimate that the AirCore data can serve as reliable references for the validation of the total column as well as for the validation of the tropospheric and UTLS partial columns (see Appendix F).

Table 2 Locations of AirCore sites and amount of satellite data compared to AirCore. “Number” gives the total number of single satellite footprint collocations, “Days” the number of days with collocations, and “AC number” the number of collocated AirCore profiles.

Figure 10 Comparison of the different satellite XCH₄ products with adjusted AirCore XCH₄ measured at Trainou (black) and Sodankylä (blue). Data for all individual coincidences are shown in the background as pale crosses, and daily averages are depicted as crosses with error bars (daily means are only calculated if there are at least three observations per day, which is the case on 29 d of the total 31 d with AirCore observations). (a–c) Time series of the differences (error bars represent the daily 1σ standard deviation of the difference). (d–f) Correlation between AirCore and satellite data (the black line is the one-to-one diagonal, and the x-axis error bars represent the mean uncertainty estimated for the AirCore data – according to Eq. (F3) – and y-axis error bars the daily 1σ standard deviation of the satellite data, respectively). (g–i) Correlation between the a priori free AirCore and satellite data (error bars as in d–f). The inserted text reports median and scatter (hIPR68.2; a–c) and the coefficients of determination, the slope, and the intercept of the robust linear regression model (R², m, and b; d–i). Black and grey fonts represent the values for the daily mean data and for data from all individual collocations, respectively.

The comparison between the satellite and the AirCore XCH₄ data is shown in Fig. 10. The differences of collocated measurements are shown in Figs. 10a–c. The agreement between the different satellite products and AirCore is good: the scatter around the median difference is low and similar to the comparison with TCCON. Furthermore, we observe no significant bias in any of the satellite data, which demonstrates the good consistency between the RemoTeC TROPOMI and MUSICA IASI XCH₄ data.

Figure 10d–f depict the correlation between the satellite and the adjusted AirCore data. Here we apply the same iteratively reweighted least-squares as in Sect. 4.1. The obtained R² values are high (for all products above 60 % for daily mean data) although a bit lower than the R² value achieved for the correlation with TCCON data; however, we have to consider that the amplitude in the analysed total-column signals is much smaller in the AirCore data set (data from two northern hemispheric sites only) if compared to the TCCON data set (data from 14 globally distributed sites). As for the comparison to TCCON, we examine the correlation between the a priori free reference data (${\mathrm{\Delta }}_a\mathrm{AirCore}={\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{AirCore}\right)-{\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{a}\mathrm{priori}\right)$) and the a priori free satellite data (${\mathrm{\Delta }}_a\mathrm{Satellite}={\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{Satellite}\right)-{\mathrm{XCH}}_{\mathrm{4}}\left(\mathrm{a}\mathrm{priori}\right)$). These correlations are visualized in Fig. 10g–i. We find reasonable correlation for the daily mean TROPOMI and combined data products but no significant correlation for the daily mean IASI product. This indicates that the correlation as observed between the IASI and the adjusted AirCore data in Fig. 10e is mainly due to the a priori data; i.e. IASI adds almost no information with respect to XCH₄ to what is already known by the a priori model. These findings are in line with the vertical resolution and representativeness analyses of Sect. 3.2.

Figure 11 Same as Fig. 10 but for the comparisons of tropospheric partial-column-averaged CH₄ AirCore and satellite products (IASI and combined).

Figure 12 Same as Fig. 11 but for comparisons of UTLS partial-column-averaged CH₄ data.

Figure 11 presents the comparison between the AirCore and satellite tropospheric partial-column CH₄ data. The differences between the satellite and the AirCore data are depicted in Fig. 11a and b. If compared to the total-column data, the agreement worsens a bit (increased median difference and scatter). Nevertheless, the agreement is still good and close to what can be expected from the uncertainty and the comparability of the different data (see Appendix F). Concerning the daily mean data, the combined product has a median difference and hIPR68.2 scatter of below about 0.9 %. These values increase to about 1.25 % for the IASI product. These results might indicate a weak systematic bias in the MUSICA IASI lower tropospheric partial columns.

Figure 11c and d show the correlation plots. In particular for the combined product we observe a reasonable correlation (R² of about 26 % for daily mean data obtained using the robust linear regression model). For the IASI product the correlation strength is reduced (R² of about 18 % for daily mean data). Furthermore, we have to consider that the IASI product has a rather limited tropospheric sensitivity (see Sect. 3.2), which means that a large part of the observed correlation is due to the a priori data: according to Eq. (7), for low entries in A^l the variability in the satellite data as well as in the adjusted AirCore data is determined by the variability in the a priori (${\mathit{x}}_{\mathit{a}}^{\mathbf{l}}$). This is confirmed by Fig. 11e and f, which show the correlations after removing the a priori data. We still observe a good correlation for the combined product (R² of about 44 % and regression line slope of 0.93 for daily mean data) but only a weak correlation for the IASI daily mean data (R² of about 11 %). This clearly documents the importance of combining IASI and TROPOMI in order to be sensitive to and reliably detect tropospheric CH₄ variations.

Concerning the UTLS partial column, we find a very good agreement between the adjusted AirCore data and the IASI and combined satellite data products (see Fig. 12a, b): the median difference calculated from the daily mean data is about −0.3 %, and the scatter values are within about 0.5 %. We find no indication of a bias in the satellite data product. The scatter observed between the AirCore and satellite data is even better than what we estimate from the data uncertainty and the data comparability analysis (see Appendix F). Figure 12c and d show that in the UTLS the AirCore and satellite data are strongly correlated (for daily mean data and when using the robust linear regression model, we get R² values of up to about 82 % and regression line slopes of very close to unity). In this altitude region the MUSICA IASI and the combined products have a very good sensitivity (see Sect. 3.2). This means that the entries in A^l of Eq. (7) are large, and any deviation between the a priori and the actual CH₄ concentrations in the UTLS are captured well by the adjusted AirCore and satellite data products. Nevertheless, the correlation strength observed for the a priori free data (Fig. 12e, f) is relatively weak (R² values of 20 %–23 % for daily mean data). This indicates that the a priori model does generally capture well the actual variation of the CH₄ concentration in the UTLS above France and northern Scandinavia.

4.3 GAW surface in situ CH₄ measurements

At many globally distributed sites, atmospheric trace gas in situ measurements are made continuously within the Global Atmospheric Watch (GAW; https://community.wmo.int/activity-areas/gaw, last access: 5 July 2022) programme. Appendix A of Sepúlveda et al. (2014) presents a method for filtering common signals in night-time CH₄ data of the two nearby mountain GAW stations Jungfraujoch (46.5^∘ N, 8.0^∘ E; 3580 m a.s.l.) and Schauinsland (47.9^∘ N, 7.9^∘ E; 1205 m a.s.l.). Data were retained as common signals when deviations of observations (after correction for vertical gradient, i.e. application of an offset, and a temporal shift in the annual cycles) at both sites were below a certain threshold. Sepúlveda et al. (2014) showed that the common signals are strongly representative of a broader layer in the lower free troposphere. Here we follow this approach and use the mean of the Jungfraujoch and Schauinsland CH₄ mixing ratio – whenever identified as a common signal – as a validation reference for the remote-sensing data in south-western Germany and northern Switzerland (indicated by the grey circle in Fig. 8b). We assume that the signals obtained from this GAW data filtering are strongly representative of the tropospheric partial-column-averaged mixing ratios (surface–6 km a.s.l.) and compare these data directly to different satellite products as a fully independent data set: we do not adjust the data to a common a priori data usage as in Sect. 4.1 because the in situ data represent absolute measurements and do not depend on any a priori information. Furthermore, we do not adjust sensitivities as in Sect. 4.2 (see Eq. 7), which means that here we also validate the sensitivities of the products.

Figure 13 Comparison of GAW measurements made at Jungfraujoch and Schauinsland with the TROPOMI tropospheric CH₄ proxy product according to Eq. (8) and the IASI and the combined tropospheric CH₄ products. Data for all individual coincidences are shown in the background as squares, and daily averages are depicted as crosses with error bars representing the daily 1σ standard deviations (daily means are only calculated if there are at least three observations per day). (a–c) Time series of differences. (d–f) Correlation between GAW and satellite data (the black line is the one-to-one diagonal). (g–i) Correlations between the difference of GAW and a priori data (${\mathrm{\Delta }}_a\mathrm{GAW}={\mathrm{troXCH}}_{\mathrm{4}}\left(\mathrm{GAW}\right)-{\mathrm{troXCH}}_{\mathrm{4}}\left(\mathrm{a}\mathrm{priori}\right)$) and the a priori free satellite data (${\mathrm{\Delta }}_a\mathrm{Satellite}={\mathrm{troXCH}}_{\mathrm{4}}\left(\mathrm{Satellite}\right)-{\mathrm{troXCH}}_{\mathrm{4}}\left(\mathrm{a}\mathrm{priori}\right)$). The inserted text reports median and scatter (hIPR68.2; a–c) and the coefficients of determination, the slope, and the intercept of the robust linear regression model (R², m, and b; d–i). Dark and pale coloured fonts represent the values for the daily mean data and for data from all individual collocations, respectively.

In order to be able to compare TROPOMI data to the GAW data, we calculate a proxy (f_T(TROPOMI)) from the TROPOMI XCH₄ data that represents the tropospheric column-averaged mixing ratios:

$$\begin{array}{}\text{(8)}& \begin{array}{rl}& {\mathrm{troXCH}}_{\mathrm{4}}\left(\mathrm{TROPOMI}\right)\approx f_T\left(\mathrm{TROPOMI}\right)=\\ & {\displaystyle \frac{X_{\mathrm{air}}}{{\mathrm{troX}}_{\mathrm{air}}}}\left[{\mathrm{XCH}}_{\mathrm{4}}\right(\mathrm{TROPOMI})-{\mathrm{XCH}}_{\mathrm{4}}(\mathrm{a}\mathrm{priori}\left)\right]+{\mathrm{troXCH}}_{\mathrm{4}}\left(\mathrm{a}\mathrm{priori}\right).\end{array}\end{array}$$

In Eq. (8) X_air and troX_air are the dry-air total and tropospheric partial columns, respectively, and troXCH₄(a priori) is the tropospheric column-averaged CH₄ a priori data. In the case that the CH₄ a priori data in the UTLS is of very good quality, this proxy is strongly representative of the tropospheric CH₄ variations.

Figure 13 shows the comparison with the different satellite products. For the tropospheric proxy product calculated from the XCH₄ product of TROPOMI, we observe no systematic difference and a scatter of the daily mean differences of within 1.3 % (Fig. 13a). However, the correlation is rather weak (from the robust linear regression model we get R² values of about 10 % and regression line slope m of below 0.5; see Fig. 13d), which might suggest that this proxy is affected by signals in the UTLS, where CH₄ values are dominated by shifts of the tropopause height.

For the MUSICA IASI tropospheric partial-column-averaged mixing ratio product (Fig. 13b, e) we observe a smaller median difference than for the comparison with the TROPOMI tropospheric proxy CH₄ data but at the same time an increased scatter. The R² values are larger than for the correlation of TROPOMI proxy data; however, we have to be careful because in the lower troposphere, the MUSICA IASI CH₄ data have a limited sensitivity (see Fig. 5b). This means that the respective data are significantly affected by the a priori assumptions, and the observed correlation might actually be due to a correlation with the tropospheric a priori data. This is confirmed by Fig. 13h, which shows the correlation after removing the a priori data. Then the correlation strength is weaker if compared to the data that include the a priori information (R² decreases from about 25 % to 20 % and from about 20 % to 10 % for correlations with daily mean and all individual data, respectively).

The combined product has a good sensitivity in the troposphere (see Fig. 5b); i.e. the respective partial-column-averaged mixing ratio product is practically independent from the a priori assumptions. We find a good agreement and correlation between the GAW data and the combined products as illustrated in Fig. 13c and f: for instance, for daily mean data, the difference and scatter is +0.28 % ±1.05 %, the R² value is about 37 %, and the regression line slope is very close to 1.0. This demonstrates that the combined product can reliably capture actual tropospheric CH₄ variations independently from the UTLS CH₄ variations and from the a priori assumption. The latter is confirmed by Fig. 13i, which shows the correlation after removing the tropospheric a priori information. We observe that the good correlation remains, even after removing the a priori information (for daily mean data the R² value is about 39 % and the regression line slope close to 1.0). A similar good correlation is not achieved by the TROPOMI tropospheric proxy and the IASI product.

5 Global data

5.1 Discussion on global data consistency

The TCCON and AirCore comparisons of Sects. 4.1 and 4.2 suggest that the combined total-column and UTLS partial-column products have no significant bias with respect to reference data. However, there might be a weak bias in the troposphere (see discussions in the context of Fig. 11). In general we have to consider that the study on biases in the profile data is limited to the two sites where AirCore references are available: Sodankylä in northern Scandinavia and Trainou in France. In this section, we argue that it is reasonable to assume similar insignificant or low biases also for other locations.

According to Eq. (A2) a varying error in the a priori state together with a poor sensitivity (i.e. an averaging kernel being very different from an identity matrix) can cause a varying bias. If the error in the a priori state is latitudinally dependent, the bias will also be latitudinally dependent. Similarly, a systematic error source (like an error in a spectroscopic parameter) can have a variable impact on the remote-sensing product, if the sensitivity is variable. If the sensitivity has a dependency on latitude, a systematic error source can thus also cause a latitudinally dependent bias. In this context, variabilities (e.g. latitudinal dependencies) of biases are likely for a low or variable sensitivity. In contrast, inconsistencies in the bias are unlikely in the case of a high and constant sensitivity (as observed in Fig. 5 for the total column and tropospheric and UTLS partial columns of the combined data product).

Figure 14 Latitudinal dependency of the overall mean values obtained at the 14 TCCON and two AirCore observation sites. Grey colour represents the TCCON and AirCore reference data and black, red, and blue colours the TROPOMI, MUSICA IASI, and combined satellite data, respectively: (a) for total columns (XCH₄) and (b) for tropospheric and UTLS partial columns. The error bars on the AirCore data describe the variability range due to the AirCore data treatment – according to Eq. (7) – with the different averaging kernels of the TROPOMI, MUSICA IASI, and combined data product.

Figure 14 depicts the overall mean total- and partial-column values obtained at the 14 TCCON and two AirCore observation sites. For total-column data (Fig. 14a), we achieve a good latitudinal coverage by the TCCON observation sites and can investigate possible latitudinal inconsistencies in the satellite data products. We find that the TROPOMI product and the combined satellite data product capture a latitudinal dependency that is similar to the dependency as seen in the TCCON data.

Figures 3a and 5a reveal that for the TROPOMI and the combined XCH₄ products, the sensitivities are very high and stable, in contrast to the MUSICA IASI data product, which has a relatively weak and seasonally (and supposed latitudinally) varying sensitivity. This explains that in Fig. 14a the latitudinal dependency of the MUSICA IASI XCH₄ data is different from the TCCON data. Table 3 resumes the statistics done with the overall mean XCH₄ values obtained for the 14 TCCON observation sites. For the TROPOMI and the combined data product, the 1σ standard deviation calculated from the mean difference of the 14 stations is about 0.4 %. A standard linear least-squares fit results in R² values of almost 100 % and regression line slope values of close to unity, which confirms the very good latitudinal data consistency of the TROPOMI and combined data products. The MUSICA IASI XCH₄ product shows poorer performance with regard to the values of standard deviation and R², which is in line with its weak and varying sensitivity.

Table 3 Statistics based on the comparisons between satellite and TCCON data of the overall mean XCH₄ values obtained for the 14 TCCON sites of Table 1.

A similar study of the latitudinal consistency of the partial-column data products is compromised by the lack of profile references for low latitudes and southern hemispheric sites (Fig. 14b). Nevertheless, because the combined product has a rather high and constant sensitivity for the tropospheric as well as the UTLS partial column (see Figs. 3 and 5), we expect – as for XCH₄ – a good latitudinal consistency, i.e. a bias at low and/or southern latitudes that is similarly insignificant or as low as the biases observed at two AirCore stations in the middle and high northern latitudes.

5.2 Example of global maps

The proposed synergetic use method needs no extra retrievals and is thus computationally very efficient. This makes it ideal for combining the large TROPOMI and IASI data sets on global scale. Figure 15 shows monthly mean global maps ($\mathrm{1}{}^{\circ }\times \mathrm{1}{}^{\circ }$ resolution) of TROPOMI XCH₄ data and the tropospheric and UTLS partial-column CH₄ data of the combined product. The maps are generated from about 1.62 million and 3.77 million individual data points in January and July 2020, respectively. These are the data that remain after requiring collocation of the quality filtered IASI and TROPOMI data according to Sect. 2. TROPOMI alone only reports the XCH₄ data (Fig. 15a, b). We observe low XCH₄ values at high latitudes. The lowest values are encountered in the summertime Southern Hemisphere. The highest XCH₄ values are present between northern low and middle latitudes. Here Fig. 15a and b show the TROPOMI data (the XCH₄ data of the combined product are very similar). The combined product offers the most reliable tropospheric partial columns. Respective maps are shown in Fig. 15c and d. We observe partial-column-averaged CH₄ mixing ratios that are almost monotonically increasing from south to north. In northern hemispheric winter (January 2020), this gradient is significantly stronger than in northern hemispheric summer (July 2020). The latitudinal patterns of tropospheric CH₄ are significantly different from the respective patterns of XCH₄, which might indicate to an extra potential of this tropospheric CH₄ data when investigating the CH₄ sources and sinks. Figure 15e and f show the respective maps of the UTLS partial columns (here we depict the combined data product; the respective MUSICA IASI data are very similar). We observe the highest partial-column-averaged mixing ratios at low latitudes (in January 2020 around the Equator and in July 2020 in the northern subtropics). The mixing ratios are lowest in high latitudes. This latitudinal pattern is in line with the tropopause height, which increases from high to low latitudes.

Figure 15 Global maps with $\mathrm{1}{}^{\circ }\times \mathrm{1}{}^{\circ }$ (longitude × latitude) resolution of monthly mean data for January and July 2020. (a, b) XCH₄ as observed by TROPOMI. (c, d) Tropospheric partial columns of CH₄ as obtained by the combined product. (e, f) UTLS partial columns of CH₄ as obtained by the combined product.

6 Summary and outlook

We present a method for a synergetic combination of the IASI vertical profile and TROPOMI total-column level 2 retrieval products. It is computationally very efficient because it is based on simple linear algebra calculations that work with the output already available from individual IASI and TROPOMI retrievals. Nevertheless, theoretically it approximates closely to a computationally expensive multispectral retrieval that uses the TROPOMI and IASI level 1 data (see Appendix B). We apply the method to CH₄ level 2 products. By providing a compilation with all important equations, we support its application to other data products.

We theoretically examine the sensitivity, vertical resolution, and errors of the individual TROPOMI and IASI products and of the combined product. The TROPOMI product consists of reliable total-column CH₄ data but does not offer information on the vertical distribution. The IASI product offers some information on the vertical distribution and has the best sensitivity in the UTLS region but lacks sensitivity in the lower troposphere and in consequence to the total column. We show that the combined product combines both strengths: it is a reliable reference for the total column and also for the UTLS partial column. In addition, we found as a clear synergetic effect that the combined product is theoretically able to distinguish variations of CH₄ that take place in the troposphere from variations at higher altitudes (it is a reliable reference for the tropospheric partial columns). We empirically demonstrate the functionality of the synergetic use method by comparing the different satellite CH₄ products to reference data of TCCON, AirCore and GAW.

The TCCON data offer good references for XCH₄. In this study we use data from 14 stations covering different climate regions in the Northern and Southern Hemisphere. For the TROPOMI and the combined data products, which are highly sensitive for XCH₄, we get an agreement with the TCCON data within about 0.7 % (the agreement is slightly poorer with the IASI satellite product due to its reduced sensitivity).

AirCore offers XCH₄ references as well as references for the vertical distribution of CH₄. For this study 36 individual AirCore profiles measured at two sites in the northern hemispheric high and middle latitudes are available. Concerning XCH₄, the comparisons to AirCore data confirm the results obtained by the comparison to TCCON data and in addition demonstrate a very good consistency between the TROPOMI and the IASI product. Concerning CH₄ in the UTLS – where the MUSICA IASI and the combined data product are highly sensitive – we find that both products agree well with the respective AirCore references (agreement within 0.7 %).

The validation study with the TCCON and AirCore references shows that the total column and the UTLS partial column of the combined product has almost the same good quality as the respective products of TROPOMI and MUSICA IASI. This allows two conclusions: firstly, the assumption of the moderate non-linearity – required for a reliable functionality of the level 2 product combination according to Eq. (1) – is valid, and secondly, the combined product's tropospheric data are also of good quality (good total column and UTLS data quality is an indirect proof of a good tropospheric data quality).

The good quality of the combined product in the troposphere is in addition directly proven by the comparison to tropospheric reference data. We find an agreement of the daily mean tropospheric AirCore and combined product data within 1 %. This validation result is confirmed by the statistically very robust comparison with CH₄ data observed continuously at two nearby GAW stations (the collocated GAW reference data cover all seasons for more than 3 years and represent more than 186 different days). The GAW and the combined product's data capture very similar tropospheric CH₄ short-term variabilities and seasonal cycles. Similar good agreements are not achieved by comparisons to the individual MUSICA IASI or TROPOMI data products; i.e. we empirically and directly prove the synergetic effect of the level 2 product combination.

The proposed method takes benefit from the outputs generated by the dedicated individual TROPOMI and IASI retrievals; it needs no extra retrievals and is thus computationally very efficient. This is ideal for an operational combination of IASI and TROPOMI products in an efficient and sustained manner. This has a particular attraction because IASI and TROPOMI successor instruments will be jointly aboard the upcoming Metop-SG (Meteorological operational – Second Generation) satellites (guaranteeing observations from the 2020s to the 2040s). IASI and TROPOMI successor instruments will have globally distributed and perfectly collocated observations (over land) in the order of several hundred thousands per day, for which a combined product can be generated in a computationally very efficient way.

Appendix A: Basics on retrieval theory

In this appendix, we give a brief overview of the theory of optimal estimation remote-sensing methods and follow the notation as recommended by the TUNER activity (von Clarmann et al., 2020), which is closely in line with the notation used by Rodgers (2000). The overview focuses on the equations that are important for our work, i.e. the a posteriori combination of two independently retrieved optimal estimation remote-sensing products. For a more detailed and general insight into the theory of optimal estimation remote-sensing methods, we refer to Rodgers (2000).

Atmospheric remote-sensing instruments measure radiance spectra (written as state vector y), which can be simulated by models (F) well whenever the actual atmospheric state (the vector x) is known. Using the a priori atmospheric state vector x_a we can linearize and write

$$\begin{array}{}\text{(A1)}& \mathit{F}\mathbf{\left(}\mathit{x}\mathbf{\right)}-\mathit{F}\mathbf{\left(}{\mathit{x}}_{\mathit{a}}\mathbf{\right)}=\mathrm{\Delta }\mathit{y}=\mathbf{K}(\mathit{x}-{\mathit{x}}_{\mathit{a}}).\end{array}$$

Here, K is the Jacobian matrix, i.e. derivatives that capture how the measurement vector (the measured radiances) will change for changes of the atmospheric state (the atmospheric state vector x). A remote-sensing retrieval inverts Eq. (A1) and provides an estimation of the difference between the atmospheric state and the a priori atmospheric state. For a moderately non-linear problem (according to Chapter 5 of Rodgers, 2000), the retrieved optimal estimation product ($\widehat{\mathit{x}}$) can be written as

$$\begin{array}{}\text{(A2)}& \widehat{\mathit{x}}-{\mathit{x}}_{\mathit{a}}=\mathbf{G}\mathrm{\Delta }\mathit{y}=\mathbf{G}\left[\mathbf{K}\right(\mathit{x}-{\mathit{x}}_{\mathit{a}}\left)\right].\end{array}$$

G is the gain matrix and realizes the inversion from the measurement domain (radiances) to the domain of the atmospheric states. It consists of derivatives that capture how the retrieved atmospheric state vector will change for changes in the measurement vector:

$$\begin{array}{}\text{(A3)}& \begin{array}{rl}\mathbf{G}& =({\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}\mathbf{K}+{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}{)}^{-\mathrm{1}}{\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}\\ & ={\mathbf{S}}_{\mathbf{a}}{\mathbf{K}}^T({\mathbf{KS}}_{\mathbf{a}}{\mathbf{K}}^T+{\mathbf{S}}_{\mathbf{y},\mathrm{n}}{)}^{-\mathrm{1}},\end{array}\end{array}$$

with S_y,n and $\mathbf{R}={{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}$ being the retrieval's noise covariance and the constraint matrices (in a strict optimal estimation sense, the constraint matrix is the inverse of the a priori covariance matrix S_a), respectively. The equivalence of both lines in Eq. (A3) is demonstrated in Chapter 4.1 of Rodgers (2000), where the first line is called the n form and the second line the m form.

The averaging kernel,

$$\begin{array}{}\text{(A4)}& \mathbf{A}=\mathbf{GK},\end{array}$$

is an important component of a remote-sensing retrieval because according to Eq. (A2), it reveals how changes of the actual atmospheric state vector x affect the retrieved atmospheric state vector $\widehat{\mathit{x}}$.

A valuable diagnostic quantity is the a posteriori covariance matrix, which can be calculated as follows:

$$\begin{array}{}\text{(A5)}& {\mathbf{S}}_{\widehat{\mathbf{x}}}=({\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}\mathbf{K}+{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}{)}^{-\mathrm{1}}.\end{array}$$

The linearized formulation of the retrieval solution according to Eq. (A2) is very useful for the analytic characterization of the product. The retrieval state's noise error covariance matrix for noise can be analytically calculated as

$$\begin{array}{}\text{(A6)}& \begin{array}{rl}{\mathbf{S}}_{\widehat{\mathbf{x}},\mathbf{n}}& ={\mathbf{GS}}_{\mathbf{y},\mathrm{n}}{\mathbf{G}}^T\\ & ={\mathbf{S}}_{\widehat{\mathbf{x}}}{\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{KS}}_{\widehat{\mathbf{x}}},\end{array}\end{array}$$

where S_y,n is the covariance matrix for noise on the measured radiances y. The second line of Eq. (A6) is obtained by substituting G by ${\mathbf{S}}_{\widehat{\mathbf{x}}}{\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}$ according to Eqs. (A3) and (A5). The representativeness error reveals the deficit of the retrieval product in representing the actual variations of the state vector x. In Chapter 3 of Rodgers (2000), it is called the smoothing error and can be calculated as (with I being the identity matrix)

$$\begin{array}{}\text{(A7)}& \begin{array}{rl}{\mathbf{S}}_{\widehat{\mathbf{x}},\mathbf{r}}& =(\mathbf{A}-\mathbf{I}){\mathbf{S}}_{\mathbf{a}}(\mathbf{A}-\mathbf{I}{)}^T\\ & ={\mathbf{S}}_{\widehat{\mathbf{x}}}{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}{\mathbf{S}}_{\widehat{\mathbf{x}}}.\end{array}\end{array}$$

The second line of Eq. (A7) is obtained using

$$\begin{array}{}\text{(A8)}& {\mathbf{S}}_{\widehat{\mathbf{x}}}=(\mathbf{I}-\mathbf{A}){\mathbf{S}}_{\mathbf{a}},\end{array}$$

which in turn follows from Eqs. (A3)–(A6).

Using Eqs. (A5)–(A7) reveals that the a posteriori covariance is the sum of the noise error covariance and the representativeness error covariance:

$$\begin{array}{}\text{(A9)}& {\mathbf{S}}_{\widehat{\mathbf{x}}}={\mathbf{S}}_{\widehat{\mathbf{x}},\mathbf{n}}+{\mathbf{S}}_{\widehat{\mathbf{x}},\mathbf{r}}.\end{array}$$

Appendix B: Theory on the optimal combination of retrieval data products

In this section, we discuss an optimal estimation retrieval that uses a combined measurement vector (two measurements from different instruments). First we show that the retrieval output of two profile retrievals performed on the same vertical grid can be used in a way that yields the same results as performing a retrieval with the combined measurement vector. Then we present an approach for combining the outputs of a retrieval that provides profiles and another retrieval that provides column data. We show that the combination of profile and column data can be realized in a computationally efficient manner via a Kalman filter. Finally, we discuss the validity of the methods and the requirements on the individual retrieval products.

B1 Inversion of a combined measurement vector

According to Eqs. (A2), (A3), and (A5) the retrieval product obtained from measurement y can be written as

$$\begin{array}{}\text{(B1)}& \widehat{\mathit{x}}-{\mathit{x}}_{\mathit{a}}=({\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}\mathbf{K}+{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}{)}^{-\mathrm{1}}{\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}\mathbf{K}(\mathit{x}-{\mathit{x}}_{\mathit{a}}).\end{array}$$

In the case of two individual measurements (measurement 1 and 2), from using a combined measurement vector {y₁,y₂} we obtain

$$\begin{array}{}\text{(B2)}& \begin{array}{rl}\widehat{\mathit{x}}-{\mathit{x}}_{\mathit{a}}& =({{\mathbf{K}}_{\mathbf{1}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{1}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{1}}+{{\mathbf{K}}_{\mathbf{2}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{2}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{2}}+{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}{)}^{-\mathrm{1}}\\ & \times ({{\mathbf{K}}_{\mathbf{1}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{1}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{1}}+{{\mathbf{K}}_{\mathbf{2}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{2}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{2}})(\mathit{x}-{\mathit{x}}_{\mathit{a}})\\ & =({{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}}^{-\mathrm{1}}+{{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{2}}}}^{-\mathrm{1}}-{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}{)}^{-\mathrm{1}}\\ & \times ({{\mathbf{K}}_{\mathbf{1}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{1}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{1}}+{{\mathbf{K}}_{\mathbf{2}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{2}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{2}})(\mathit{x}-{\mathit{x}}_{\mathit{a}}),\end{array}\end{array}$$

where ${\mathbf{S}}_{{\mathbf{y}}_{\mathbf{1}},\mathrm{n}}$ and ${\mathbf{S}}_{{\mathbf{y}}_{\mathbf{2}},\mathrm{n}}$ are the respective measurement noise covariances, K₁ and K₂ the respective Jacobians, and ${\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}$ and ${\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{2}}}$ the respective a posteriori covariances. The second line follows from Eq. (A5). According to Eqs. A3–A5 we can substitute ${\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}\mathbf{K}(\mathit{x}-{\mathit{x}}_{\mathit{a}})$ by ${{\mathbf{S}}_{\widehat{\mathbf{x}}}}^{-\mathrm{1}}(\widehat{\mathit{x}}-{\mathit{x}}_{\mathit{a}})$ and write Eq. (B2) as

$$\begin{array}{}\text{(B3)}& \begin{array}{rl}\widehat{\mathit{x}}-{\mathit{x}}_{\mathit{a}}& =({{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}}^{-\mathrm{1}}+{{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{2}}}}^{-\mathrm{1}}-{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}{)}^{-\mathrm{1}}\\ & \times \left[{{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}}^{-\mathrm{1}}\right({\widehat{\mathit{x}}}_{\mathbf{1}}-{\mathit{x}}_{\mathit{a}})+{{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{2}}}}^{-\mathrm{1}}({\widehat{\mathit{x}}}_{\mathbf{2}}-{\mathit{x}}_{\mathit{a}}\left)\right].\end{array}\end{array}$$

Using Eq. (B3) we can realize an optimal combination of the two retrieval products that only needs the a priori covariance, the a posteriori covariances, and the two retrieval products. The Jacobians are not needed. This combination is mathematically equivalent to using the Jacobians of a combined measurement vector {y₁,y₂}; i.e. within a linear subspace (validity of moderate non-linearity according to Chapter 5 of Rodgers, 2000), it is equivalent to a synergetic use of level 1 data in the form of a multispectral retrieval.

B2 Combining profile and column data products

Equation (B3) requires two retrieval results on the same vertical grid and can be used to combine two profile products. Here we will develop a method for combining a profile and a column data product. For a column retrieval we can write in analogy to Eq. (A1)

$$\begin{array}{}\text{(B4)}& \mathrm{\Delta }{x}^{*}={{\mathit{a}}^{\mathbf{*}}}^T(\mathit{x}-{\mathit{x}}_{\mathit{a}}),\end{array}$$

where ${{\mathit{a}}^{\mathbf{*}}}^T$ is the column-averaged mixing ratio according to Appendix D2. Equation (B4) poses an inverse problem of the same kind as Eq. (A1), and in order to optimally estimate a profile from an available column product, we can apply the same solution approach as in Eqs. (A2) and (A3). A similar application of this approach is also presented in Sect. 4.2 of Rodgers and Connor (2003). For the application here we substitute in Eq. (A3) K by ${{\mathit{a}}^{\mathbf{*}}}^T$ and S_y,n by the scalar $S_{\widehat{x},n}^{*}$ (the noise error variance of the column data product) and get the profile

$$\begin{array}{}\text{(B5)}& \begin{array}{rl}\widehat{\mathit{x}}-{\mathit{x}}_{\mathit{a}}& =({\mathit{a}}^{\mathbf{*}}{S_{\widehat{x},n}^{*}}^{-\mathrm{1}}{{\mathit{a}}^{\mathbf{*}}}^T+{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}{)}^{-\mathrm{1}}{\mathit{a}}^{\mathbf{*}}{S_{\widehat{x},n}^{*}}^{-\mathrm{1}}{{\mathit{a}}^{\mathbf{*}}}^T(\mathit{x}-{\mathit{x}}_{\mathit{a}})\\ & =({\mathit{a}}^{\mathbf{*}}{S_{\widehat{x},n}^{*}}^{-\mathrm{1}}{{\mathit{w}}^{\mathbf{*}}}^T{\mathbf{S}}_{\widehat{\mathbf{x}}}{\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}\mathbf{K}+{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}{)}^{-\mathrm{1}}\\ & \times {\mathit{a}}^{\mathbf{*}}{S_{\widehat{x},n}^{*}}^{-\mathrm{1}}{{\mathit{w}}^{\mathbf{*}}}^T{\mathbf{S}}_{\widehat{\mathbf{x}}}{\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}\mathbf{K}(\mathit{x}-{\mathit{x}}_{\mathit{a}}).\end{array}\end{array}$$

For the second line of Eq. (B5) we use ${{\mathit{a}}^{\mathbf{*}}}^T={{\mathit{w}}^{\mathbf{*}}}^T\mathbf{A}$ – according to Eq. (D6) – and $\mathbf{A}={\mathbf{S}}_{\widehat{\mathbf{x}}}{\mathbf{K}}^T{{\mathbf{S}}_{\mathbf{y},\mathrm{n}}}^{-\mathrm{1}}\mathbf{K}$ – according to Eqs. (A3–A5). We write this second line to discuss similarities with Eq. (B1). The comparison of Eq. (B5) with Eq. (B1) reveals that for a retrieval providing only a column product, the Jacobian information provided by K is vertically aggregated according to the operator ${\mathit{a}}^{\mathbf{*}}{S_{\widehat{x},n}^{*}}^{-\mathrm{1}}{{\mathit{w}}^{\mathbf{*}}}^T{\mathbf{S}}_{\widehat{\mathbf{x}}}$. The term ${\mathbf{S}}_{\widehat{\mathbf{x}}}$ is the vertically resolved a posteriori covariance, which exists for a retrieval that internally inverts profiles but only distributes the column products; however, it is only an internal measure of the retrieval and not actually available.

Instead of the term of Eq. (B4), we now invert the term $\mathrm{\Delta }{x}^{*}={{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T(\mathit{x}-{\widehat{\mathit{x}}}_{\mathbf{1}})$; i.e. we replace x_a by the profile product ${\widehat{\mathit{x}}}_{\mathbf{1}}$ of a first retrieval (retrieval 1) on the right side of Eq. (B4) and use ${{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T$ and $S_{{\widehat{\mathit{x}}}_{\mathbf{2}},n}^{*}$ for the column averaging kernel and the noise error variance of a second retrieval (retrieval 2), respectively. Here and in the following, retrieval 1 is the profile retrieval and retrieval 2 the retrieval that provides only column products. The solution can easily be achieved by substituting in Eq. (B5) S_a by ${\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}$, which is the a posteriori covariance of retrieval 1:

$$\begin{array}{}\text{(B6)}& \begin{array}{rl}\widehat{\mathit{x}}-{\widehat{\mathit{x}}}_{\mathbf{1}}& =({\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{x}}_{\mathrm{2}},n}^{*}}^{-\mathrm{1}}{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T+{{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}}^{-\mathrm{1}}{)}^{-\mathrm{1}}\\ & \times {\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{x}}_{\mathrm{2}},n}^{*}}^{-\mathrm{1}}{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T(\mathit{x}-{\widehat{\mathit{x}}}_{\mathbf{1}}).\end{array}\end{array}$$

We modify Eq. (B6) using ${\widehat{\mathit{x}}}_{\mathbf{1}}={\mathbf{A}}_{\mathbf{1}}(\mathit{x}-{\mathit{x}}_{\mathit{a}})+{\mathit{x}}_{\mathit{a}}$:

$$\begin{array}{}\text{(B7)}& \begin{array}{rl}\widehat{\mathit{x}}-{\mathit{x}}_{\mathit{a}}& ={\mathbf{A}}_{\mathbf{1}}(\mathit{x}-{\mathit{x}}_{\mathit{a}})+({{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}}^{-\mathrm{1}}+{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{x}}_{\mathrm{2}},n}^{*}}^{-\mathrm{1}}{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{)}^{-\mathrm{1}}\\ & \times {\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{\mathit{x}}}_{\mathbf{2}},n}}^{-\mathrm{1}}{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T(\mathbf{I}-{\mathbf{A}}_{\mathbf{1}})(\mathit{x}-{\mathit{x}}_{\mathit{a}})\\ & =({{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}}^{-\mathrm{1}}+{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{x}}_{\mathrm{2}},n}^{*}}^{-\mathrm{1}}{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{)}^{-\mathrm{1}}\\ & \times [{{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}}^{-\mathrm{1}}{\mathbf{A}}_{\mathbf{1}}+{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{x}}_{\mathrm{2}},n}^{*}}^{-\mathrm{1}}{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T](\mathit{x}-{\mathit{x}}_{\mathit{a}})\\ & =({{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}}^{-\mathrm{1}}+{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{x}}_{\mathrm{2}},n}^{*}}^{-\mathrm{1}}{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{)}^{-\mathrm{1}}\\ & \times \left[{{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}}^{-\mathrm{1}}\right({\widehat{\mathit{x}}}_{\mathbf{1}}-{\mathit{x}}_{\mathit{a}})+{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{x}}_{\mathrm{2}},n}^{*}}^{-\mathrm{1}}({\widehat{x}}_{\mathrm{2}}^{*}-{{\mathit{w}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}}\left)\right].\end{array}\end{array}$$

In the third line of Eq. (B7) we use the column product ${\widehat{x}}_{\mathrm{2}}^{*}={{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T(\mathit{x}-{\mathit{x}}_{\mathit{a}})+{{\mathit{w}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}}$. Similarly to Eq. (B3), we can generate a combined product without the need of the Jacobian matrices. The combination is possible using the profile and the column product (${\widehat{\mathit{x}}}_{\mathbf{1}}$ and ${\widehat{x}}_{\mathrm{2}}^{*}$, respectively), together with the a posteriori covariance of the profile product and the noise error and averaging kernel of the column product.

If we substitute in the second line of Eq. (B7) ${{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}}^{-\mathrm{1}}$ by ${{\mathbf{K}}_{\mathbf{1}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{1}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{1}}+{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}$ , according to Eq. (A5), ${{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T$ by ${{\mathit{w}}^{\mathbf{*}}}^T{\mathbf{A}}_{\mathbf{2}}$, according to Eq. (D6), and then A₂ by ${\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{2}}}{{\mathbf{K}}_{\mathbf{2}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{2}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{2}}$, according to Eqs. (A3)–(A5), we get

$$\begin{array}{}\text{(B8)}& {\displaystyle }\begin{array}{rl}& \widehat{\mathit{x}}-{\mathit{x}}_{\mathit{a}}=\\ & ({{\mathbf{K}}_{\mathbf{1}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{1}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{1}}+{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{x}}_{\mathrm{2}},n}^{*}}^{-\mathrm{1}}{{\mathit{w}}^{\mathbf{*}}}^T{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{2}}}{{\mathbf{K}}_{\mathbf{2}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{2}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{2}}+{{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}{)}^{-\mathrm{1}}\\ & [{{\mathbf{K}}_{\mathbf{1}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{1}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{1}}+{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{x}}_{\mathrm{2}},n}^{*}}^{-\mathrm{1}}{{\mathit{w}}^{\mathbf{*}}}^T{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{2}}}{{\mathbf{K}}_{\mathbf{2}}}^T{{\mathbf{S}}_{{\mathbf{y}}_{\mathbf{2}},\mathrm{n}}}^{-\mathrm{1}}{\mathbf{K}}_{\mathbf{2}}](\mathit{x}-{\mathit{x}}_{\mathit{a}}).\end{array}\end{array}$$

This equation has strong similarities to the first line of Eq. (B2), i.e. the retrieval product obtained when using the combined measurement vector {y₁,y₂}. The only difference is that in Eq. (B8), the information provided by Jacobian ${{\mathbf{K}}_{\mathbf{2}}}^T$ is vertically aggregated according to the operator ${\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}{S_{{\widehat{x}}_{\mathrm{2}},n}^{*}}^{-\mathrm{1}}{{\mathit{w}}^{\mathbf{*}}}^T{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{2}}}$.

B3 Linear Kalman filter

Here we show that the approach developed in Appendix B2 is equivalent to a Kalman filter. An important application of a Kalman filter (Kalman, 1960; Rodgers, 2000) is data assimilation in the context of atmospheric modelling. There, the filter operates sequentially in different time steps. Kalman filter data assimilation methods determine the analysis state (${\widehat{\mathit{x}}}^{\mathit{a}}$) by optimally combining the background (or forecast) state (${\widehat{\mathit{x}}}^{\mathit{b}}$) with the information as provided by a new observation (${\widehat{\mathit{x}}}^{\mathit{o}}$):

$$\begin{array}{}\text{(B9)}& {\widehat{\mathit{x}}}^{\mathit{a}}={\widehat{\mathit{x}}}^{\mathit{b}}+\mathbf{M}[{\widehat{\mathit{x}}}^{\mathit{o}}-\mathbf{H}{\widehat{\mathit{x}}}^{\mathit{b}}].\end{array}$$

“Optimal” means here that the uncertainties of both, the background state and the observation, are correctly taken into account by the Kalman gain matrix (M):

$$\begin{array}{}\text{(B10)}& \mathbf{M}={\mathbf{S}}_{{\widehat{\mathbf{x}}}^{\mathbf{b}}}{\mathbf{H}}^T({\mathbf{HS}}_{{\widehat{\mathbf{x}}}^{\mathbf{b}}}{\mathbf{H}}^T+{\mathbf{S}}_{{\widehat{\mathbf{x}}}^{\mathbf{o}},\mathbf{n}}{)}^{-\mathrm{1}},\end{array}$$

with ${\mathbf{S}}_{{\widehat{\mathbf{x}}}^{\mathbf{b}}}$ and ${\mathbf{S}}_{{\widehat{\mathbf{x}}}^{\mathbf{o}},\mathbf{n}}$ being the uncertainty covariances of background state and the new measurement, respectively. The matrix H is the measurement forward operator, which maps the background domain into the measurement domain.

By rearranging the n form of Eq. (B6) as the m form – in analogy to Eq. (A3) – and using ${\widehat{\mathit{x}}}_{\mathbf{1}}={\mathbf{A}}_{\mathbf{1}}(\mathit{x}-{\mathit{x}}_{\mathit{a}})+{\mathit{x}}_{\mathit{a}}$ and ${\widehat{x}}_{\mathrm{2}}^{*}={{\mathit{a}}^{\mathbf{*}}}^T(\mathit{x}-{\mathit{x}}_{\mathit{a}})+{{\mathit{w}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}}$ again, we get

$$\begin{array}{}\text{(B11)}& \begin{array}{rl}\widehat{\mathit{x}}& ={\widehat{\mathit{x}}}_{\mathbf{1}}+{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}({{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}+S_{{\widehat{x}}_{\mathrm{2}},n}^{*}{)}^{-\mathrm{1}}{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T(\mathit{x}-{\widehat{\mathit{x}}}_{\mathbf{1}})\\ & ={\widehat{\mathit{x}}}_{\mathbf{1}}+{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}({{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}+S_{{\widehat{x}}_{\mathrm{2}},n}^{*}{)}^{-\mathrm{1}}\\ & \times [{\widehat{x}}_{\mathrm{2}}^{*}-{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{\widehat{\mathit{x}}}_{\mathbf{1}}-({{\mathit{w}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}}-{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}}\left)\right]\\ & ={\widehat{\mathit{x}}}_{\mathbf{1}}+\mathit{m}({\widehat{x}}_{\mathrm{2}}^{*}-{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{\widehat{\mathit{x}}}_{\mathbf{1}})-\mathit{m}({{\mathit{w}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}}-{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}}),\end{array}\end{array}$$

with

$$\begin{array}{}\text{(B12)}& \mathit{m}={\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}({{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}+S_{{\widehat{x}}_{\mathrm{2}},n}^{*}{)}^{-\mathrm{1}}.\end{array}$$

Disregarding the term that accounts for the a priori information ($\mathit{m}({{\mathit{w}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}}-{{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}})$), Eqs. (B11) and (B12) are the same as the Kalman filter equations, Eqs. (B9) and (B10): retrieval 1 provides the background state and retrieval 2 the new observation. Compared to Eqs. (B7) and (B8) the form of Eq. (B11) has the advantage that no matrices have to be inverted, only the scalar $({{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}}^T{\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{1}}}{\mathit{a}}_{\mathbf{2}}^{\mathbf{*}}+S_{{\widehat{x}}_{\mathrm{2}},n}^{*})$.

We have shown that Eq. (B11) is mathematically the same as Eqs. (B7) and (B8). The latter is in turn very similar to the synergetic use of level 1 data in the form of a multispectral retrieval as discussed in the context of Eq. (B3).

B4 Discussion and requirements

In Appendix B2 and B3, we assume the usage of the same a priori state for the two individual retrievals. Since generally two individually performed retrievals use two different a priori settings, we have to perform an a priori adjustment. Using the a priori state of retrieval 2 as the reference (${\mathit{x}}_{\mathbf{2}\mathbf{,}\mathit{a}}={\mathit{x}}_{\mathit{a}}$), we can adjust the output of retrieval 1 by (see Eq. (10) of Rodgers and Connor, 2003)

$$\begin{array}{}\text{(B13)}& {{\widehat{\mathit{x}}}_{\mathbf{1}}}^{\prime }={\widehat{\mathit{x}}}_{\mathbf{1}}+({\mathbf{A}}_{\mathbf{1}}-\mathbf{I})({\mathit{x}}_{\mathbf{1}\mathbf{,}\mathit{a}}-{\mathit{x}}_{\mathbf{2}\mathbf{,}\mathit{a}}),\end{array}$$

where x_1,a is the a priori state used by retrieval 1.

For a combination according to Eq. (B3), we need retrieval 1 and 2 outputs obtained using the same constraint (the inverse of the a priori covariance S_a). This has to be accounted for before applying Eq. (B3), by adjusting the constraint according to the formalism as presented in Chapter 10.4 of Rodgers (2000) or Sect. 4.2 of Rodgers and Connor (2003). By applying Eq. (B7) or the Kalman filter according to Eq. (B11) the common constraint is automatically set to the constraint of the retrieval 1 product, and no extra modification is necessary.

The synergetic combination of remote-sensing profile and column products according to Eq. (B7) or Eq. (B11) is possible whenever (1) the two remote-sensing observations are made at the same time and detect the same location, (2) the problems is moderately non-linear (according to Chapter 5 of Rodgers, 2000), and (3) the individual retrieval output as listed by Eqs. (B7) or  (B11) is made available: for the profile retrieval, we need the a posteriori covariances (${\mathbf{S}}_{\widehat{\mathbf{x}}}$, which might also be reconstructed from A and $\mathbf{R}={{\mathbf{S}}_{\mathbf{a}}}^{-\mathrm{1}}$ according to Eq. (A8)), the averaging kernels (A), and the retrieved and a priori state vectors ($\widehat{\mathit{x}}$ and x_a, respectively). For the column retrieval, we need the noise variances (the scalar $S_{\widehat{x},n}^{*}$), the column averaging kernels (the row vector ${{\mathit{a}}^{\mathbf{*}}}^T$), the column product (${\widehat{x}}_{\mathrm{2}}^{*}$), and the a priori column data (${{\mathit{w}}^{\mathbf{*}}}^T{\mathit{x}}_{\mathit{a}}$), respectively.

Appendix C: Operator for transformation between linear and logarithmic scales

Linear-scale differentials and logarithmic-scale differentials are related by $\partial x=x\partial \ln x$. For transforming differentials or covariances of a state vector with the dimension of the number of atmospheric levels (nal) from logarithmic to linear scale, we define the nal × nal diagonal matrix L:

$$\begin{array}{}\text{(C1)}& \mathbf{L}=\left(\begin{array}{cccc}{\widehat{x}}_{\mathrm{1}}& \mathrm{0}& \mathrm{\cdots }& \mathrm{0}\\ \mathrm{0}& {\widehat{x}}_{\mathrm{2}}& \mathrm{\cdots }& \mathrm{0}\\ \mathrm{⋮}& \mathrm{⋮}& \mathrm{\ddots }& \mathrm{⋮}\\ \mathrm{0}& \mathrm{0}& \mathrm{\cdots }& {\widehat{x}}_{\mathrm{nal}}\end{array}\right).\end{array}$$

Here ${\widehat{x}}_i$ is the value of the ith element of the retrieved state vector (i.e. in the case of an atmospheric CH₄ state vector the CH₄ mixing ratios retrieved at the ith model level).

Approximatively, a logarithmic-scale averaging kernel matrix A^l can then be expressed in the linear scale as

$$\begin{array}{}\text{(C2)}& \mathbf{A}\approx {\mathbf{LA}}^{\mathrm{l}}{\mathbf{L}}^{-\mathrm{1}}.\end{array}$$

This is an approximation here because on the right side, the operator L should contain the actual instead of the retrieved mixing ratios. It is a valid approximation as long as the a priori data are reasonable, and there is no large bias in the retrieval data.

Similarly, a logarithmic-scale covariance matrix S^l can be approximately expressed in the linear scale as

$$\begin{array}{}\text{(C3)}& \mathbf{S}\approx {\mathbf{LS}}^{\mathrm{l}}{\mathbf{L}}^T.\end{array}$$

Here the approximation is because Δx≈xΔln x.

Appendix D: Operators for column data

This appendix explains the calculation of operators for partial (and total) column data. Although some sections are similar to Appendix C of Schneider et al. (2022), we think it is a very useful reference here because it facilitates the reproducibility of our results.

For converting mixing ratio profiles into amount profiles, we set up a pressure weighting operator Z, as a diagonal matrix with the following entries:

$$\begin{array}{}\text{(D1)}& Z_{i,i}={\displaystyle \frac{\mathrm{\Delta }{p}_i}{g_i{m}_{\mathrm{air}}(\mathrm{1}+\frac{m_{{\mathrm{H}}_{\mathrm{2}}\mathrm{O}}}{m_{\mathrm{air}}}{\widehat{x}}_i^{{\mathrm{H}}_{\mathrm{2}}\mathrm{O}})}}.\end{array}$$

Using the pressure p_i at atmospheric grid level i we set $\mathrm{\Delta }{p}_{\mathrm{1}}=\frac{p_{\mathrm{2}}-p_{\mathrm{1}}}{\mathrm{2}}-p_{\mathrm{1}}$, $\mathrm{\Delta }{p}_{\mathrm{nal}}=p_{\mathrm{nal}}-\frac{p_{\mathrm{nal}}-p_{\mathrm{nal}-\mathrm{1}}}{\mathrm{2}}$, and $\mathrm{\Delta }{p}_i=\frac{p_{i+\mathrm{1}}-p_i}{\mathrm{2}}-\frac{p_i-p_{i-\mathrm{1}}}{\mathrm{2}}$ for $\mathrm{1}<i<\mathrm{nal}$. Furthermore, g_i is the gravitational acceleration at level i, m_air and $m_{{\mathrm{H}}_{\mathrm{2}}\mathrm{O}}$ the molecular mass of dry air and water vapour, respectively, and ${\widehat{x}}_i^{{\mathrm{H}}_{\mathrm{2}}\mathrm{O}}$ the retrieved or modelled water vapour mixing ratio at level i.

We define an operator W^T for resampling fine gridded atmospheric amount profiles into coarse gridded atmospheric partial-column amount profiles. It has the dimension c×nal, where c is the number of the resampled coarse atmospheric grid levels and nal the number of atmospheric levels of the original fine atmospheric grid. Each line of the operator has the value “1” for the levels that are resampled and “0” for all other levels:

$$\begin{array}{}\text{(D2)}& {\mathbf{W}}^T=\left(\begin{array}{ccccccccc}\mathrm{1}& \mathrm{\cdots }& \mathrm{1}& \mathrm{0}& \mathrm{\cdots }& \mathrm{\cdots }& \mathrm{\cdots }& \mathrm{\cdots }& \mathrm{0}\\ \mathrm{0}& \mathrm{\cdots }& \mathrm{0}& \mathrm{1}& \mathrm{\cdots }& \mathrm{1}& \mathrm{0}& \mathrm{\cdots }& \mathrm{0}\\ \mathrm{0}& \mathrm{\cdots }& \mathrm{\cdots }& \mathrm{\cdots }& \mathrm{\cdots }& \mathrm{0}& \mathrm{1}& \mathrm{\cdots }& \mathrm{1}\end{array}\right).\end{array}$$

In analogy we can define a row vector w^T (with the dimension 1× nal) with all elements having the value “1”, which allows for the resampling of the total-column amounts.

D1 Column amounts

The kernel that describes how a change in the amount at a certain altitude affects the retrieved partial (or total) column amount can be calculated as

$$\begin{array}{}\text{(D3)}& {\mathbf{A}}^{\prime }={\mathbf{W}}^T{\mathbf{ZAZ}}^{-\mathrm{1}}.\end{array}$$

For the total column, we replace W^T by w^T and get the row vector ${{\mathit{a}}^{\mathbf{\prime }}}^T$ (dimension 1× nal). This is the total-column kernel provided by the TROPOMI data, and it is typically written as a^T. Figure 3 shows examples of such total and partial-column amount kernels. The total-column amount kernel can be interpolated to different altitude grids. For the applications in Sects. 3 and 4 we interpolate the TROPOMI total-column amount kernel to the vertical grid used by the MUSICA IASI retrieval. For all vertical interpolations, we use the logarithmic pressure values as the reference for the vertical dimension.

D2 Column-averaged mixing ratios

We can also combine the operators Z and W^T for the calculation of a pressure weighted resampling operator by

$$\begin{array}{}\text{(D4)}& {{\mathbf{W}}^{*}}^T=({\mathbf{W}}^T\mathbf{ZW}{)}^{-\mathrm{1}}{\mathbf{W}}^T\mathbf{Z}.\end{array}$$

This operator resamples linear-scale mixing ratio profiles into linear-scale partial-column-averaged mixing ratio profiles. The respective total-column operator ${{\mathit{w}}^{\mathbf{*}}}^T$ can be calculated in analogy to Eq. (D4) by replacing W^T by w^T.

With operator ${{\mathbf{W}}^{*}}^T$ we can calculate a coarse gridded partial-column-averaged state ${\widehat{\mathit{x}}}^{*}$ from the fine gridded linear mixing ratio state $\widehat{\mathit{x}}$ by

$$\begin{array}{}\text{(D5)}& {\widehat{\mathit{x}}}^{*}={{\mathbf{W}}^{*}}^T\widehat{\mathit{x}}.\end{array}$$

The kernels matrix of the partial-column-averaged mixing ratio state can then be calculated from the fine gridded linear-scale kernel matrix (A) by

$$\begin{array}{}\text{(D6)}& {\mathbf{A}}^{*}={{\mathbf{W}}^{*}}^T\mathbf{A}.\end{array}$$

This kernel describes how a change in the mixing ratio at a certain altitude affects the retrieved partial-column-averaged mixing ratio. Covariances of the partial-column-averaged mixing ratio state can be calculated from the corresponding covariance matrices of the fine gridded linear scale (S) by

$$\begin{array}{}\text{(D7)}& {\mathbf{S}}^{*}={{\mathbf{W}}^{*}}^T{\mathbf{SW}}^{*}.\end{array}$$

The respective calculations for total-column-averaged mixing ratios can be done by replacing ${{\mathbf{W}}^{*}}^T$ by ${{\mathit{w}}^{\mathbf{*}}}^T$. For the total-column-averaged mixing ratios, the covariance is a simple variance (the scalar S^*), and the kernel has the dimension 1× nal; i.e. it is a row vector ${{\mathit{a}}^{\mathbf{*}}}^T$.

The total-column amount kernel (${{\mathit{a}}_{\mathit{T}}}^T$) provided with the TROPOMI data set can be converted into a total-column-averaged mixing ratio kernel ${{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T$ by the following calculation (using Eqs. D3, D4, and D6):

$$\begin{array}{}\text{(D8)}& {{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T={{\mathit{w}}^{\mathbf{*}}}^T{\mathbf{A}}_{\mathbf{T}}=({\mathit{w}}^T\mathbf{Z}\mathit{w}{)}^{-\mathrm{1}}{{\mathit{a}}_{\mathit{T}}}^T\mathbf{Z}.\end{array}$$

The total-column-averaged mixing ratio kernel ${{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T$ used in Sects. 3 and 4 is valid for the vertical grid used by the MUSICA IASI retrieval. It is calculated from the TROPOMI total-column amount kernel (${{\mathit{a}}_{\mathit{T}}}^T$) provided in the TROPOMI output files according to Eq. (D8), after its interpolation onto the MUSICA IASI grid (see also Appendix D1).

Appendix E: Dislocation of TROPOMI and IASI

IASI is on an orbit with descending node Equator crossing at 9:30 mean local solar time. TROPOMI is on an orbit with ascending node Equator crossing at 13:30 mean local solar time. In this work we require a temporal collocation within at least 6 h. This requirement causes the following typical time difference (IASI–TROPOMI) for observing the same location: at northern high latitudes −0.6 to +3.7 h, at northern middle latitudes −3.3 to −2.2 h, at the Equator −4.5 to −3.5 h, at southern middle latitudes −5.4 to −4.3 h, and at southern high latitudes −5.9 to −4.5 h. This means that at all latitudes, we find data that fulfil the temporal collocation requirements and that in the Southern Hemisphere the temporal collocation is typically larger than in the Northern Hemisphere. Furthermore, there are horizontal dislocations. In this work we use a horizontal collocation threshold of 50 km. In this appendix we estimate the impact of these spatial and temporal dislocations on the combined product.

Figure E1 Square root values of the diagonal entries of different dislocation covariance matrices (${\mathbf{S}}_{{\mathbf{\Delta }}_{\mathbf{dl}}}^{\mathrm{l}}$). These values are the root-mean-square (rms) of the difference between the reference methane profile (CAMS forecast for location 49.1^∘ N, 8.4^∘ E, corresponding to the location of Karlsruhe) and other forecasted profiles dislocated with respect to the reference: (a) horizontal dislocations and (b) temporal dislocations. The dashed black lines indicate the collocation threshold values used for valid combinations of IASI and TROPOMI.

E1 Uncertainty source

For investigating the spatial and temporal variability of the atmospheric CH₄ fields, we use the CAMS (Copernicus Atmospheric Monitoring Service; https://atmosphere.copernicus.eu/, last access: 5 July 2022) CH₄ forecast product at the highest available resolution (≈9 km; Barré et al., 2021). By analysing the profiles forecasted for the same location but different timestamps, we can determine the temporal covariance of the vertical CH₄ fields. Similarly, by analysing the profiles forecasted for the same timestamp but different locations, we get the spatial covariance of the vertical CH₄ fields. The analyses are done with CAMS data between November 2017 and December 2020 for central Europe in an area around Karlsruhe. The results are depicted in Figs. E1 to E3.

Figure E2 Characteristics of the vertical dependencies covered by the matrices ${\mathbf{S}}_{{\mathbf{\Delta }}_{\mathbf{dl}}}^{\mathrm{l}}$. Shown are the vertical correlation matrices for the difference between the reference CH₄ profile (location 49.1^∘ N, 8.4^∘ E, corresponding to the location of Karlsruhe) and profiles dislocated with respect to the reference: (a) horizontal dislocation of 50 km and (b) temporal dislocation of 6 h.

Figure E1 shows the root-mean-square (rms) of the difference between the forecasted reference methane profile and forecasted profiles that are dislocated with respect to the reference by different spatial distances and time differences. These are the square root values of the diagonal entries of the respective dislocation covariance matrices (${\mathbf{S}}_{{\mathbf{\Delta }}_{\mathbf{dl}}}^{\mathrm{l}}$; we use here the superscript “^l” for logarithmic scale because we work with relative covariances: $\mathrm{\Delta }\ln x\approx \mathrm{\Delta }x/x$). The dashed black lines indicate our collocation threshold values used for the combination of TROPOMI and IASI (TROPOMI and IASI are only combined as long as the horizontal distance of their ground pixels is within 50 km, and the time difference is within 6 h). Naturally, the respective rms values increase with increasing horizontal distance and time difference. The values are largest in a small layer close to the surface and in the stratosphere but relatively small in the free troposphere. For a horizontal dislocation of 50 km, the rms value is about 2 % very close to the surface and between 0.3 % and 0.5 % for the rest of the troposphere, and then it increases again to about 2 % above 25 km altitude. For a time difference of 6 h, the rms value is about 2.5 % in a very small layer above the ground and 0.6 %–0.8 % in the free troposphere below 10 km, and it reaches about 1.5 % at 15 km and 3.5 % at 30 km altitude.

Figure E2 reveals to what extent the dislocation uncertainties as shown in Fig. E1 are vertically correlated. Depicted are the vertical correlations for the example of a spatial dislocation of 50 km (Fig. E2a) and a temporal dislocation of 6 h (Fig. E2b). We observe that for both spatial and temporal dislocations, the vertical behaviour of the vertical correlation length (distance where correlation coefficient decays to 0.5) is similar. The vertical correlation lengths are rather short close to the surface (only 100–200 m). They are larger for higher altitudes: in the middle and upper troposphere and in the stratosphere they increase to about 1000 and 6000 m, respectively.

Figure E3 Same as Fig. E1 but for column-averaged data: total column, tropospheric partial column, and UTLS partial column.

Figure E4 Example of dislocation kernel ${\mathbf{A}}_{\mathbf{C},\mathbf{dl}}^{\mathbf{l}}$ – calculated according to Eq. (E3) – for the same late summer observation as used in the context of Figs. 1 to 3.

Figure E5 Comparison of the dislocation error (due to the CH₄ dislocation uncertainty) and the noise error (an example of the typical temporal dependencies of the noise error is shown in Fig. 6). The comparison is depicted for a northern high-latitude location (Sodankylä; violet crosses), a tropical location (Darwin; orange crosses), and a southern middle latitudinal location (Lauder; red crosses). (a) Total-column product, (b) tropospheric partial-column product, and (c) UTLS partial-column product.

The dislocation error for total and partial columns can be calculated by resampling the spatial and temporal dislocation covariance matrices according to Eq. (D7) (for more details, see Appendix D). The results of these resampling calculations are shown for the spatial dislocation in Fig. E3a and for the temporal dislocation in Fig. E3b. Naturally the dislocation uncertainties increase with increasing horizontal distance and time difference. For our horizontal collocation threshold values of 50 km, the uncertainty (rms value is used as the metric) is about 0.2 % for the total-column data. For our time difference collocation threshold of 6 h, it is about 0.3 % for the total-column data. For the tropospheric and upper tropospheric–lower stratospheric partial columns, the respective relative uncertainties values are slightly larger.

E2 Impact on the combined CH₄ product

For calculating the error in the combined profile due to the horizontal and spatial dislocation between IASI and TROPOMI, we substitute ${\widehat{\mathit{x}}}_{\mathit{I}}$ in Eq. (1) by ${\widehat{\mathit{x}}}_{\mathit{I}}+{\mathbf{A}}_{\mathbf{I}}{\mathbf{\Delta }}_{\mathbf{dl}}$, where Δ_dl is the dislocation uncertainty of CH₄, as shown in Figs. E1 and E2. This results in a new term in Eq. (1) that gives the dislocation error in the combined profile:

$$\begin{array}{}\text{(E1)}& {\mathbf{\Delta }}_{\mathbf{dl}}{\widehat{\mathit{x}}}_{\mathit{C}}^{\mathit{l}}=(\mathbf{I}-{\mathbf{L}}^{-\mathrm{1}}\mathit{m}{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T\mathbf{L}){\mathbf{A}}_{\mathbf{I}}^{\mathbf{l}}{\mathbf{\Delta }}_{\mathbf{dl}}^{\mathit{l}}.\end{array}$$

The respective error covariance matrix is

$$\begin{array}{}\text{(E2)}& {\mathbf{S}}_{{\widehat{\mathbf{x}}}_{\mathbf{C}},\mathrm{dl}}^{\mathbf{l}}={\mathbf{A}}_{\mathbf{C},\mathrm{dl}}^{\mathbf{l}}{\mathbf{S}}_{{\mathbf{\Delta }}_{\mathrm{dl}}}^{\mathbf{l}}{{\mathbf{A}}_{\mathbf{C},\mathrm{dl}}^{\mathbf{l}}}^T,\end{array}$$

where ${\mathbf{S}}_{{\mathbf{\Delta }}_{\mathrm{dl}}}^{\mathbf{l}}$ is the covariance matrix for the CH₄ dislocation uncertainty, whose main characteristics are visualized in Figs. E1 and E2. Here,

$$\begin{array}{}\text{(E3)}& {\mathbf{A}}_{\mathbf{C},\mathrm{dl}}^{\mathbf{l}}=(\mathbf{I}-{\mathbf{L}}^{-\mathrm{1}}\mathit{m}{{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T\mathbf{L}){\mathbf{A}}_{\mathbf{I}}^{\mathbf{l}}\end{array}$$

is the dislocation averaging kernel. Figure E4 shows an example of this dislocation averaging kernel. For the altitudes where the dislocation uncertainty of CH₄ is largest (close to the ground and above 20 km; see Fig. E1), the dislocation kernel has rather low values (i.e. there the combination procedure has only limited sensitivity to the dislocation uncertainty).

We calculate the dislocation error covariance matrices according to Eq. (E2) for different locations and then determine the corresponding total- and partial-column dislocation errors by summing up the temporal and spatial dislocation covariances and performing an subsequent resampling of the covariance matrices according to Eq. (D7) (for more details, see Appendix D). Figure E5 depicts this dislocation error in comparison to the noise error (respective resampling of the covariance matrices obtained by Eq. (5); see also Fig. 6). We focus here on three different latitudinal locations: Sodankylä (northern high latitudes), Darwin (low latitudes), and Lauder (southern middle latitudes). We find that for the northern high-latitude site (where horizontal and temporal dislocation are of similar importance) but also for the tropical and southern hemispheric middle-latitude sites (where the temporal dislocation is dominating), the dislocation uncertainty is generally much smaller than the noise error.

Appendix F: Data comparability

The satellite data products are representative of broad vertical layers of the atmosphere (see averaging kernels as shown in Figs. 2 and 3). Also the TCCON and AirCore reference data are sensitive to atmospheric CH₄ at different vertical regions. If we furthermore assume that the TCCON and the AirCore data offer a stable absolute calibration reference, their inter-comparisons with the satellite data as shown in Sect. 4 can in principle be used for empirically validating the characteristics (sensitivity and error) of the satellite data products. The level of agreement that can be expected between the reference data and the satellite products depends on the reliability of the references and the characteristics of the satellite data products. In Appendix F1 and F2 we estimate the reliability of the TCCON and AirCore data, respectively, to serve as reference for the satellite data products. Then in Appendix F3 we show that the results of the inter-comparison as shown in the context of Figs. 9 to 12 are in a reasonable agreement with the reliability of the references and the characteristics of the different satellite data products. This confirms the validity of the sensitivity of the satellite data products as shown in Sect. 3.2 and the validity of the errors of the satellite data as documented in Sect. 3.3.

F1 TCCON versus satellite

For estimating the reliability of the TCCON data as reference for the satellite data products we consider the TCCON retrieval noise errors, the incomparableness of TCCON and satellite data caused by their different averaging kernels, and the collocation mismatch between the TCCON and the satellite observations. The total-column uncertainty variance (the scalar $S_{\mathrm{ref}}^{*}$) for using the TCCON data as reference for the satellite data can be estimated by

$$\begin{array}{}\text{(F1)}& \begin{array}{rl}{S}_{\mathrm{ref}}^{*}& =S_{\mathrm{\Delta }\mathrm{TC}}^{*}+({{\mathit{a}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T){\mathbf{S}}_{\mathbf{\Delta }\mathbf{a}}({{\mathit{a}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T{)}^T\\ & +{{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T({\mathbf{S}}_{\mathbf{\Delta }\mathrm{h}}+{\mathbf{S}}_{\mathbf{\Delta }\mathrm{t}}){\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}.\end{array}\end{array}$$

The first term (the scalar $S_{\mathrm{\Delta }\mathrm{TC}}^{*}$) is the TCCON retrieval error variance (the TCCON error is provided with the TCCON data is typically 1 ‰). The second term accounts for the different averaging kernels. The row vectors ${{\mathit{a}}^{\mathbf{*}}}^T$ and ${{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T$ are the total-column-averaged mixing ratio kernels of the satellite and the TCCON retrievals, respectively (calculated according to Appendix D). The matrix S_Δa describes the uncertainty covariances of the a priori data used and the matrices S_Δh and S_Δt the covariances for horizontal and temporal collocation mismatches.

Figure F1 Comparison of the CH₄ state obtained from the TROPOMI a priori model TM5 (x_TM5) and the collocated CAMS high-resolution forecasts (x_CAMS). (a) The rms of the relative differences; (b) Matrix showing the correlations of TM5-CAMS differences at different altitudes.

For estimating S_Δa we use the difference between the CH₄ state as modelled by TM5 (x_TM5) and provided by the high-resolution CAMS forecast (x_CAMS; e.g. Barré et al., 2021). Figure F1 shows the results of these calculations for the surroundings of Karlsruhe documented by the rms values of the differences in the vertical profiles (Fig. F1a) and the vertical correlation matrix of the differences (Fig. F1b). We estimate an uncertainty of the TM5 a priori model of about 6 % close to the surface, about 2 % up to the middle troposphere, a gradual increase to about 7.5 % between the UTLS and about 23 km altitude, and a maximum value of about 27 % in the stratosphere at about 30 km. The vertical correlation lengths (altitude range where the correlation coefficient decreases to about 0.5) are a few hundred metres close to the surface, about 5000 m in the middle troposphere, about 2500 m in the UTLS, and about 7500 m in the stratosphere above 30 km altitude. We find that this relatively large disagreement between the TM5 a priori data and the high-resolution forecast of CAMS is significantly influenced by inconsistencies between TM5 and CAMS in the years 2019 and 2020: after 2018 the TM5 model shows an increase of about 1 % per year, but the CAMS high-resolution forecast shows no significant increase.

Figure F2 Error in the comparison of TCCON and satellite products due to the a priori model error and the different column sensitivities of the TCCON product and the satellite products. These values are calculated as $({{\mathit{a}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T)({\mathit{x}}_{\mathbf{TM}\mathbf{5}}-{\mathit{x}}_{\mathbf{CAMS}})$ and are represented in Eq. (F1) by the square root value of the term $({{\mathit{a}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T){\mathbf{S}}_{\mathbf{\Delta }\mathbf{a}}({{\mathit{a}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T{)}^T$.

Figure F2 shows the value of the term $({{\mathit{a}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T)({\mathit{x}}_{\mathbf{TM}\mathbf{5}}-{\mathit{x}}_{\mathbf{CAMS}})$ for the different satellite data products; i.e. it reveals the uncertainty in the comparison with TCCON data due to differences in the averaging kernels and the a priori model uncertainty, which in Eq. (F1) is represented by the square root value of the term $({{\mathit{a}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T){\mathbf{S}}_{\mathbf{\Delta }\mathbf{a}}({{\mathit{a}}^{\mathbf{*}}}^T-{{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T{)}^T$. Because the TROPOMI and the TCCON kernels both have a similar good column sensitivity throughout the troposphere, the respective uncertainty is generally within 0.1 % (see black squares in Fig. F2). The same is true for the validation of the total column of the combined product (see blue crosses in Fig. F2). For the validation of the total column of the MUSICA IASI product, this error is larger because the total-column sensitivity of IASI is significantly different from the respective sensitivity of the TCCON product and the other satellite products (see Fig. 3a). For the comparison of the IASI and TCCON total-column data, we estimate that the error due to the different sensitivities (of IASI and TCCON) can occasionally be even above 2 % (see red dots in Fig. F2). This error is largest to the end of the time series because then the TM5 a priori model error is largest (increasing difference between the TM5 model and the CAMS high-resolution forecast after 2018).

The collocation mismatch covariances S_Δh and S_Δt are the linear-scale versions of the matrices ${\mathbf{S}}_{{\mathbf{\Delta }}_{\mathrm{dl}}}^{\mathbf{l}}$ (characterized in Figs. E1 and E2) interpolated to the actual temporal and horizontal mismatch of the satellite and the TCCON measurements. The effect of this collocation mismatch on the comparison of the total columns (i.e. the term ${{\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}}^T({\mathbf{S}}_{\mathbf{\Delta }\mathrm{h}}+{\mathbf{S}}_{\mathbf{\Delta }\mathrm{t}}){\mathit{a}}_{\mathbf{TC}}^{\mathbf{*}}$) is estimated to be between 0.1 % and 0.4 %.

Figure F3 Theoretically predicted and observed 1σ scatter for the comparison of single pixel satellite data with individual TCCON and AirCore reference data. Black, red, and blue colours represent TROPOMI, MUSICA IASI, and combined satellite data, respectively. The squares and vertical crosses are for XCH₄ comparisons with TCCON and AirCore references, respectively. The diagonal crosses and stars are for tropospheric and UTLS partial-column comparisons, respectively, with AirCore references.

F2 AirCore versus satellite

Similar to the TCCON data, we estimate the reliability of the AirCore profile data as reference for the satellite observations. For this estimation we consider an AirCore measurement noise covariance (S_ΔAC,n). It is calculated assuming an uncertainty for altitudes with AirCore CH₄ data of 0.3 % (Karion et al., 2010) and the uncertainty according to S_Δa from Sect. F1 for all other altitudes. The outer diagonal elements are determined by assuming the same vertical correlation as derived for S_Δa. In addition, we consider uncertainties in the height attribution, which is according to Wagenhäuser et al. (2021) below 10 m close to ground, about 200 m at 20 km a.s.l., and about 1 at 27 km a.s.l.

We construct a respective height attribution uncertainty covariance (S_ΔAC,v) by assuming a very strong correlation of the height attribution uncertainties between different altitude levels. The temporal and spatial collocation mismatch covariance between the AirCore and the satellite observations (S_Δh and S_Δt, respectively) is calculated as described in Sect. F1.

All the aforementioned uncertainties are independent, and we can calculate the total uncertainty as

$$\begin{array}{}\text{(F2)}& {\mathbf{S}}_{\mathbf{\Delta }\mathrm{AC}}={\mathbf{S}}_{\mathbf{\Delta }\mathrm{AC},\mathrm{n}}+{\mathbf{S}}_{\mathbf{\Delta }\mathrm{AC},\mathbf{v}}+{\mathbf{S}}_{\mathbf{\Delta }\mathrm{h}}+{\mathbf{S}}_{\mathbf{\Delta }\mathrm{t}}.\end{array}$$

The reliability of the AirCore data – after its adjustment according to Eq. (7) – as reference for the MUSICA IASI and combined satellite data can then be estimated by

$$\begin{array}{}\text{(F3)}& {\mathbf{S}}_{\mathrm{ref}}^{\mathrm{l}}={\mathbf{A}}^{\mathrm{l}}{\mathbf{S}}_{\mathbf{\Delta }\mathrm{AC}}^{\mathrm{l}}{{\mathbf{A}}^{\mathrm{l}}}^T.\end{array}$$

Here and in Eq. (F2), the covariances are determined for the full vertical profile. Respective covariances for total or partial columns can be derived according to Appendix D. The reliability for the TROPOMI total-column-averaged mixing ratio data can be calculated by $S_{\mathrm{ref}}^{*}={{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}}^T{\mathbf{S}}_{\mathbf{\Delta }\mathrm{AC}}{\mathit{a}}_{\mathit{T}}^{\mathbf{*}}$.

In order to get a reasonable number of collocated AirCore data, we relax the collocation criteria: we require a temporal collocation within 6 h and a spatial collocation within 500 km (see Sect. 4.2). In particular, a loose spatial collocation requirement results in theoretically large collocation mismatch uncertainties. For instance, Fig. E3 reveals that a spatial mismatch of 400 km clearly dominates the temporal mismatch, whose threshold is set to 6 h, but it is actually only typically greater than 3 h in the Southern Hemisphere. The spatial mismatch uncertainty also dominates the uncertainties in the AirCore data due to measurement noise and uncertain height attribution; i.e. it is the term that most affects the comparability of the AirCore and satellite measurements. We estimate a spatial mismatch error that has to be considered for the AirCore satellite inter-comparison of about 0.5 % for the total-column data and of about 0.6 % for the tropospheric and UTLS partial-column data.

F3 Summary

The sum of the uncertainty (co)variance of using TCCON or AirCore as the reference ($S_{\mathrm{ref}}^{*}$ or ${\mathbf{S}}_{\mathrm{ref}}^{\mathrm{l}}$; see Sects. F1 and F2, respectively) and the noise and dislocation error (co)variances of the satellite data products (see Sects. 3.3 and 3.4, respectively) gives the covariance that can be theoretically expected for the scatter between the TCCON or AirCore reference data and the satellite data products.

Figure F3 shows the correlations between the theoretically expected scatter (mean value of the scatter expected for the individual data points) and the actually observed scatter (the hIPR68.2 of the individual differences between the reference data and the satellite data products). Shown is one data point for the XCH₄ comparisons with the TCCON references, for the XCH₄ comparisons with the AirCore references, and further data points for the comparisons of the tropospheric and UTLS partial columns with the AirCore references. A detailed contemplation suggests that the scatter observed for the total-column data of the TROPOMI and the combined data products, as well as the scatter observed for the tropospheric partial-column data of the IASI and combined products, is slightly larger than their theoretically expected counterparts. Conversely, in the UTLS the scatter observed in the IASI and combined data products seems to be a bit smaller than the theoretically expected scatter values. However, the data points group reasonably well around the one-to-one diagonal; i.e. there is overall a good agreement between the theoretically expected scatter and the actually observed scatter. This means that the inter-comparison results as shown in Sect. 4 confirm the satellite data quality characterization of Sects. 3.2–3.4.