Comment on “Synergetic use of IASI profile and TROPOMI total-column level 2 methane retrieval products” by Schneider et al. (2022)

A great interest is growing about methods that combine measurements from two or more instruments that observe the same species either in different spectral regions or with different geometries. Recently, a method based on the Kalman filter has been proposed to combine IASI (Infrared Atmospheric Sounding Interferometer) and TROPOMI (TROPOspheric Monitoring Instrument) methane products. We show that this method is equivalent to the Complete Data Fusion method. Therefore, the choice between these two methods is driven only by the advantages of the different implementations. From the comparison of the two methods, a generalization of the Complete Data Fusion formula, which is valid also in the case that the noise error covariance matrices of the fused products are singular, is derived. This comment uses several equations reported in the preprint version of Schneider et al. (2022); therefore, reference is made to the preprint version of this paper.

1 Introduction

Remote atmospheric measurements are obtained with retrieval techniques (Rodgers, 2000) that determine the distribution of an atmospheric parameter by fitting forward model simulations to the available observations. Generally, the fitting procedure consists of the minimization of a cost function made of the summation of two terms. The first term is the square of the differences between observations and simulations, weighted with the inverse covariance matrix (CM) of the observations, and the second term is a constraint that reflects our a priori knowledge of the distribution of the atmospheric parameter. When two or more instruments sound the same portion of the atmosphere and observe the same atmospheric parameter either in different spectral regions or with different geometries, the problem arises of how to identify the best strategy to combine the different measurements in order to exploit all the information that is provided by the observations in presence of the retrieval constraints.

Recently, Schneider et al. (2021) presented a method for the combined use of the individual retrieval products of IASI (Infrared Atmospheric Sounding Interferometer) and TROPOMI (TROPOspheric Monitoring Instrument). The method is based on the Kalman filter (Rodgers, 2000) and, as stated by the authors, is largely equivalent to using the spectra of the different sensors together in a single retrieval procedure (simultaneous retrieval). A method based on the same principle, i.e., the application of the Kalman filter for combining different satellite sensor observations, was already presented in Warner et al. (2014) and used to combine CO products of AIRS (Atmospheric InfraRed Sounder) and TES (Tropospheric Emission Spectrometer), as well as of AIRS and MLS (Microwave Limb Sounder).

The purpose of this comment is to study the relationship between the method presented in Schneider et al. (2021) and the Complete Data Fusion (CDF) method presented in Ceccherini et al. (2015). CDF also uses the output of the individual retrievals and, in the case that the linear approximation holds in the range of the retrieved products, provides products equivalent to those of the simultaneous retrieval.

The formulae of the two methods are algebraically different, but the connection of both methods to the simultaneous retrieval suggests a close relationship between them. It is important to understand whether the two methods are equivalent (that is, if they provide the same results) and otherwise to identify the reasons of the differences because this analysis allows us to highlight the advantages of using one method or the other. In effect, in Sect. 2 we prove the equivalence between the two methods, and in Sect. 3 we draw the conclusions.

2 Equivalence between data fusion with Kalman filter and CDF

In this section, we prove the equivalence of the method proposed in Schneider et al. (2021) and the CDF method proposed in Ceccherini et al. (2015).

We start from Eq. (A9) of Schneider et al. (2021) as follows:

$$\begin{array}{}\text{(1)}& {\widehat{\mathit{x}}}^{\mathrm{a}}={\widehat{\mathit{x}}}^{\mathrm{b}}+\mathbf{M}\left[{\widehat{\mathit{x}}}^{\mathrm{o}}-\mathbf{H}{\widehat{\mathit{x}}}^{\mathrm{b}}\right],\end{array}$$

where ${\widehat{\mathit{x}}}^{\mathrm{a}}$ is the analysis state, ${\widehat{\mathit{x}}}^{\mathrm{b}}$ is the background state, ${\widehat{\mathit{x}}}^{\mathrm{o}}$ is the new observation, M is the Kalman gain matrix, and H is the measurement forward operator. In our case, this equation determines the product of the fusion of two measurements (the analysis state) by combining the first measurement (the background state) with the information provided by the second measurement (the new observation).

We substitute in Eq. (1) the different quantities, as given in Appendix A2.3 of Schneider et al. (2021). From Eqs. (A14), (A15), and (A13) we have, respectively, the following:

$$\begin{array}{}\text{(2)}& {\displaystyle }{\widehat{\mathit{x}}}^{\mathrm{b}}={\widehat{\mathit{x}}}_{\mathrm{1}}-{\mathit{x}}_{\mathrm{a}},\text{(3)}& {\displaystyle }{\widehat{\mathit{x}}}^{\mathrm{o}}={\widehat{\mathit{x}}}_{\mathrm{2}}-{\mathit{x}}_{\mathrm{a}}\end{array}$$

and

$$\begin{array}{}\text{(4)}& \mathbf{H}={\mathbf{A}}_{\mathrm{2}},\end{array}$$

where ${\widehat{\mathit{x}}}_{\mathrm{1}}$ and ${\widehat{\mathit{x}}}_{\mathrm{2}}$ are the two individual retrieval products, x_a is the a priori state vector, and A₂ is the averaging kernel matrix of ${\widehat{\mathit{x}}}_{\mathrm{2}}$. The Kalman gain matrix M is given by Eq. (A17), which, using Eq. (A4), becomes the following:

$$\begin{array}{}\text{(5)}& \mathbf{M}={\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}\left({\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right),\end{array}$$

where S_a is the a priori CM, and we have introduced the Fisher information matrices (Fisher, 1935; Ceccherini et al., 2012) F₁ and F₂, which are given by the following:

$$\begin{array}{}\text{(6)}& {\mathbf{F}}_i={\mathbf{K}}_i^T{\mathbf{S}}_{{\mathit{y}}_i,\mathrm{n}}^{-\mathrm{1}}{\mathbf{K}}_i={\mathbf{A}}_i^T{\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}^{-\mathrm{1}}{\mathbf{A}}_i={\mathbf{S}}_{{\widehat{\mathit{x}}}_i}^{-\mathrm{1}}{\mathbf{A}}_{i}i=\mathrm{1},\mathrm{2},\end{array}$$

where K_i are the Jacobian matrices of the forward models, ${\mathbf{S}}_{{\mathit{y}}_i,\mathrm{n}}$ are the CMs for noise on the measured radiances y_i, ${\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}$ are the noise error CMs of the retrieved states, and ${\mathbf{S}}_{{\widehat{\mathit{x}}}_i}$ are the a posteriori CMs. The Fisher information matrices are useful quantities that fully describe the information provided by the observations on the state vectors and are easily obtained using either the quantities that characterize the observations or those that characterize the measurements. In the case that the matrices ${\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}$ are singular, Eq. (6) is still valid if we replace the matrices ${\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}^{-\mathrm{1}}$ with the generalized inverse matrices (Kalman, 1976) ${\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}^{\mathit{\#}}$, as shown in the Appendix of Ceccherini et al. (2012). Equations (5) and (6) show that the Kalman gain matrix takes into account the uncertainties of both measurements and the a priori state vector when we combine them using Eq. (1).

In analogy with Eqs. (2) and (3), we introduce the quantity ${\widehat{\mathit{x}}}_{\mathrm{f}}$, such as the analysis state ${\widehat{\mathit{x}}}^{\mathrm{a}}$ is the difference between ${\widehat{\mathit{x}}}_{\mathrm{f}}$ and x_a as follows:

$$\begin{array}{}\text{(7)}& {\widehat{\mathit{x}}}^{\mathrm{a}}={\widehat{\mathit{x}}}_{\mathrm{f}}-{\mathit{x}}_{\mathrm{a}}.\end{array}$$

Substituting Eqs. (2)–(5) and (7) in Eq. (1), we obtain the following:

$$\begin{array}{}\text{(8)}& \begin{array}{rl}{\widehat{\mathit{x}}}_{\mathrm{f}}-{\mathit{x}}_{\mathrm{a}}=& {\widehat{\mathit{x}}}_{\mathrm{1}}-{\mathit{x}}_{\mathrm{a}}+{\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}\left({\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)\\ & \left[{\widehat{\mathit{x}}}_{\mathrm{2}}-{\mathit{x}}_{\mathrm{a}}-{\mathbf{A}}_{\mathrm{2}}\left({\widehat{\mathit{x}}}_{\mathrm{1}}-{\mathit{x}}_{\mathrm{a}}\right)\right].\end{array}\end{array}$$

Using Eqs. (A2)–(A3) of Schneider et al. (2021) and Eq. (6), we have the following:

$$\begin{array}{}\text{(9)}& {\mathbf{A}}_i={\left({\mathbf{F}}_i+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}{\mathbf{F}}_{i}i=\mathrm{1},\mathrm{2}.\end{array}$$

Therefore, Eq. (8) can be written as follows:

$$\begin{array}{}\text{(10)}& \begin{array}{rl}{\widehat{\mathit{x}}}_{\mathrm{f}}-{\mathit{x}}_{\mathrm{a}}=[& \mathbf{I}-{\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}{\mathbf{F}}_{\mathrm{2}}]\left({\widehat{\mathit{x}}}_{\mathrm{1}}-{\mathit{x}}_{\mathrm{a}}\right)\\ & +{\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}\left({\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)\left({\widehat{\mathit{x}}}_{\mathrm{2}}-{\mathit{x}}_{\mathrm{a}}\right)\\ =(& {\mathbf{F}}_{\mathrm{1}}+{\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}{)}^{-\mathrm{1}}[\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)\left({\widehat{\mathit{x}}}_{\mathrm{1}}-{\mathit{x}}_{\mathrm{a}}\right)\\ & +\left({\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)\left({\widehat{\mathit{x}}}_{\mathrm{2}}-{\mathit{x}}_{\mathrm{a}}\right)].\end{array}\end{array}$$

Consistent with Eq. (3) of Ceccherini et al. (2015), we introduce the α_i quantities, as follows:

$$\begin{array}{}\text{(11)}& {\widehat{\mathit{x}}}_i-{\mathit{x}}_{\mathrm{a}}={\mathit{\alpha }}_i-{\mathbf{A}}_i{\mathit{x}}_{\mathrm{a}}i=\mathrm{1},\mathrm{2},\end{array}$$

and Eq. (10) becomes:

$$\begin{array}{}\text{(12)}& \begin{array}{rl}{\widehat{\mathit{x}}}_{\mathrm{f}}-{\mathit{x}}_{\mathrm{a}}=& {\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}[\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)\left({\mathit{\alpha }}_{\mathrm{1}}-{\mathbf{A}}_{\mathrm{1}}{\mathit{x}}_{\mathrm{a}}\right)\\ & +\left({\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)\left({\mathit{\alpha }}_{\mathrm{2}}-{\mathbf{A}}_{\mathrm{2}}{\mathit{x}}_{\mathrm{a}}\right)]\\ =& {\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}[\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right){\mathit{\alpha }}_{\mathrm{1}}\\ & +\left({\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right){\mathit{\alpha }}_{\mathrm{2}}-{\mathbf{F}}_{\mathrm{1}}{\mathit{x}}_{\mathrm{a}}-{\mathbf{F}}_{\mathrm{2}}{\mathit{x}}_{\mathrm{a}}]\\ =& {\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}[\left({\mathbf{F}}_{\mathrm{1}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right){\mathit{\alpha }}_{\mathrm{1}}\\ & +\left({\mathbf{F}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right){\mathit{\alpha }}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}{\mathit{x}}_{\mathrm{a}}]-{\mathit{x}}_{\mathrm{a}},\end{array}\end{array}$$

when we have used Eq. (9).

Using Eqs. (A2) and (A5) of Schneider et al. (2021), we can write the noise error CMs as follows:

$$\begin{array}{}\text{(13)}& {\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}={\left({\mathbf{F}}_i+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}{\mathbf{F}}_i{\left({\mathbf{F}}_i+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}i=\mathrm{1},\mathrm{2}.\end{array}$$

In the case that ${\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}$ are not singular matrices, from Eq. (9) the following results:

$$\begin{array}{}\text{(14)}& {\mathbf{A}}_i^T{\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}^{-\mathrm{1}}=\left({\mathbf{F}}_i+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)i=\mathrm{1},\mathrm{2},\end{array}$$

and, using Eqs. (6) and (14), from Eq. (12) we obtain the following:

$$\begin{array}{}\text{(15)}& \begin{array}{rl}{\widehat{\mathit{x}}}_{\mathrm{f}}=& {\left({\mathbf{A}}_{\mathrm{1}}^T{\mathbf{S}}_{{\widehat{\mathit{x}}}_{\mathrm{1}},\mathrm{n}}^{-\mathrm{1}}{\mathbf{A}}_{\mathrm{1}}+{\mathbf{A}}_{\mathrm{2}}^T{\mathbf{S}}_{{\widehat{\mathit{x}}}_{\mathrm{2}},\mathrm{n}}^{-\mathrm{1}}{\mathbf{A}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}\\ & \left[{\mathbf{A}}_{\mathrm{1}}^T{\mathbf{S}}_{{\widehat{\mathit{x}}}_{\mathrm{1}},\mathrm{n}}^{-\mathrm{1}}{\mathit{\alpha }}_{\mathrm{1}}+{\mathbf{A}}_{\mathrm{2}}^T{\mathbf{S}}_{{\widehat{\mathit{x}}}_{\mathrm{2}},\mathrm{n}}^{-\mathrm{1}}{\mathit{\alpha }}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}{\mathit{x}}_{\mathrm{a}}\right],\end{array}\end{array}$$

which is the equation of the CDF reported in Eq. (5) of Ceccherini et al. (2015), proving the equivalence of the Kalman filter method reported in Schneider et al. (2021) with the CDF method reported in Ceccherini et al. (2015). The form of Eq. (15) highlights that the combination of the two measurements is a generalization of the weighted mean, where in the weighting not only the CMs but also the averaging kernel matrices are taken into account. The expression coincides with the weighted mean in the case that the averaging kernel matrices are equal to the identity matrices.

In the case that ${\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}$ are singular matrices, we cannot write Eq. (15). However, using Eq. (A4) of Schneider et al. (2021) and Eq. (6), from Eq. (12) we obtain the following:

$$\begin{array}{}\text{(16)}& \begin{array}{rl}{\widehat{\mathit{x}}}_{\mathrm{f}}=& {\left({\mathbf{S}}_{{\widehat{\mathit{x}}}_{\mathrm{1}}}^{-\mathrm{1}}{\mathbf{A}}_{\mathrm{1}}+{\mathbf{S}}_{{\widehat{\mathit{x}}}_{\mathrm{2}}}^{-\mathrm{1}}{\mathbf{A}}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}\right)}^{-\mathrm{1}}\\ & \left[{\mathbf{S}}_{{\widehat{\mathit{x}}}_{\mathrm{1}}}^{-\mathrm{1}}{\mathit{\alpha }}_{\mathrm{1}}+{\mathbf{S}}_{{\widehat{\mathit{x}}}_{\mathrm{2}}}^{-\mathrm{1}}{\mathit{\alpha }}_{\mathrm{2}}+{\mathbf{S}}_{\mathrm{a}}^{-\mathrm{1}}{\mathit{x}}_{\mathrm{a}}\right].\end{array}\end{array}$$

Equation (16) is valid also when ${\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}$ are singular matrices and fully maintains the equivalence with the method proposed in Schneider et al. (2021). Furthermore, it is equivalent to Eq. (15) when ${\mathbf{S}}_{{\widehat{\mathit{x}}}_i,\mathrm{n}}$ are not singular matrices. Therefore, Eq. (16) is more general than Eq. (15) that was proposed for the CDF in Ceccherini et al. (2015).

3 Conclusions

The equivalence between the method for the synergetic use of data acquired by different instruments proposed in Schneider et al. (2021) and the CDF method proposed in Ceccherini et al. (2015) has been proved. However, the original CDF formula can only be used in the case that the noise error CMs of the fused products are not singular. The full equivalence of the two methods exists only in the case of the revised CDF formula that has been here derived and given in Eq. (16).

The CDF method was also proved (Ceccherini, 2016) to be equivalent to the measurement–space–solution data fusion method (Ceccherini et al., 2009) that explicitly considered the case of measurements made in an incomplete space. Therefore, the three methods are equivalent among themselves, and for linear and moderately nonlinear problems, they are all equivalent to the simultaneous retrieval. Consequently, we expect that the choice of which among these three methods can be more efficiently used in an operational data fusion depends only on the implementation advantages. Studies that assess the implementation convenience of the three methods for different data fusion problems would be very useful. A significant difference in the implementation between the method proposed in Schneider et al. (2021) and the other two methods occurs when we have to combine more than two measurements. In this case, while the Kalman filter method requires a sequential approach in which we add one measurement at a time, the CDF and the measurement–space–solution data fusion methods have the advantage that they can be implemented by fusing all the available measurements in a single step.