Interactive comment on “ A full-mission data set of H 2 O and HDO columns from SCIAMACHY 2 . 3 μ m reflectance measurements ” by Andreas Schneider

This paper evaluated the vertical column densities of H2O and HDO retrieved from SCIAMACHY for the whole mission (2003–2012) using the non-scattering retrieval mode of the Shortwave Infrared CO Retrieval (SICOR) algorithm. The SCIAMACHY results were compared against the ground-based FTIR measurements from MUSICA. Reasonably high correlation coefficients and relatively small biases for H2O and HDO were obtained at the stations. δD was calculated a posteriori from H2O and HDO, and


Introduction
Atmospheric water vapour is a trace gas which is important for the energy budget of the atmosphere. For instance, it is the strongest natural greenhouse gas and transports energy through latent heat (e. g. Kiehl and Trenberth, 1997;Harries, 1997).
However, the uncertainties regarding the effects of water vapour in the energy balance of the atmosphere and regarding the interaction between water vapour and the atmospheric circulation are still large (e. g. Stevens and Bony, 2013). Measurements 20 are expected to contribute to a better understanding and quantification of these processes. Especially observations of isotopologues of water are interesting, because the ratio provides information about the "history" of the sampled air parcel due to a temperature-dependent isotopic fractionation during evaporation and condensation caused by different vapour pressure and diffusion constants of the isotopologues (e. g. Dansgaard, 1964). For instance, when water is evaporated from a not too small reservoir, heavy isotopologues are depleted. Water vapour is also depleted by condensation during the formation of clouds and precipitation. In all these cases the amount of depletion depends on temperature.
The ratio of the isotopologues HDO and H 2 O is for column densities c HDO and c H 2 O defined by R D = c HDO /c H 2 O . The depletion is usually described by the relative difference of an observed ratio R D to a standard ratio R D,std , This is a column δ D, which some authors refer to as δ D to distinguish it from δ D profiles. The commonly used standard ratio is Vienna Standard Mean Ocean Water (VSMOW), R D,std = 3.1152 · 10 −4 .
Measurements of atmospheric water vapour isotopologues are rare. Observations are made in situ with aircraft and balloon (e. g. Rinsland et al., 1984;Stowasser et al., 1999;Ehhalt et al., 2005;Dyroff et al., 2010), in situ on ground (e. g. Wen et al., 2010;Bastrikov et al., 2014), and by (ground or space based) remote sensing techniques. Ground-based remote sensing of the  (Steinwagner et al., 2007), the Infrared Atmospheric Sounding Interferometer (IASI) abord the MetOP satellite (Herbin et al., 2009), and the Greenhouse Gases Observing Satellite (GOSAT) (Boesch et al., 2013). Frankenberg et al. (2009) used the short-wave infrared (SWIR) band of the Scanning Imaging Absorption Spectrometer for Atmospheric Chartography (SCIAMACHY) instrument (Bovensmann et al., 1999) (Veefkind et al., 2012) was launched, which includes the SWIR in the same spectral range and with the same spectral resolution as SCIAMACHY, but at much better signal to noise ratio (SNR) and spatial resolution. It is expected that retrievals from this new instrument will provide H 2 O/HDO data of unprecedented quality. The present study is also a preparation for the new mission, 30 apart from providing a data set for the whole mission period of SCIAMACHY.
The new data set is retrieved using the Shortwave Infrared CO Retrieval (SICOR) algorithm as discussed by Scheepmaker et al. (2016); Landgraf et al. (2016); Borsdorff et al. (2016Borsdorff et al. ( , 2017. SICOR is designed for the operational processing of carbon monoxide total column retrieval from TROPOMI measurements. The algorithm has two performance options. First, assuming a strict cloud filtering, the retrieval ignores atmospheric scattering in the SWIR spectral range, and the atmospheric total column of CH 4 , H 2 O, HDO, and CO is inferred from the measurement. This approach, called non-scattering retrieval in this paper, is proposed by Scheepmaker et al. (2016) to generate the H 2 O and HDO full mission data product. In this study, the retrieval is applied to SCIAMACHY observations, and the H 2 O and HDO data product is validated against ground based measurements from MUSICA. Alternatively, one can choose for a loose cloud clearing. Here, the optical depth and height of a scattering 5 layer is retrieved from the CH 4 absorptions of the measurement using appropriate a priori information from chemical transport models (CTMs). This approach, hereinafter referred to as scattering retrieval, is the processing baseline for CO retrievals from SCIAMACHY and TROPOMI as discussed by Landgraf et al. (2016); Borsdorff et al. (2017). In this paper, this approach is evaluated for the retrieval of water vapour isotopologues by comparing ground based observations at high-altitudes with collocated SCIAMACHY retrievals using observations with clouds at a height similar to that of the ground site.

10
The remainder of this article is structured as follows: In Sect. 2 the retrieval set-ups of both the non-scattering and scattering retrieval are given. Section 3 presents the validation of the non-scattering data product, whereas Sect. 4 discusses the potential added value of the scattering retrieval of the water isotopologues. Finally, a summary is given and conclusions are drawn in Sect. 5.

15
The SICOR algorithm and its application to CO and HDO/H 2 O retrievals is discussed in detailed by Scheepmaker et al. (2016); Landgraf et al. (2016); Borsdorff et al. (2016Borsdorff et al. ( , 2017. In this section its main features relevant in the context of this study are summarised. SICOR provides two options to perform the trace gas column retrieval from SWIR radiance measurements. First, assuming clear sky observations, atmospheric scattering is ignored in the forward simulation of the measurement. Subsequently the algo-20 rithm infers the total column of H 2 O, HDO, CH 4 and CO together with the Lambertian surface albedo from SCIAMACHY's channel 8 measurements between 2338.5 nm and 2382.5 nm. This spectral fit window is an extension of the one proposed by Scheepmaker et al. (2015) including more absorption lines of HDO, which is beneficial for retrievals using measurements at the end of SCIAMACHY's lifespan (which was not considered by Scheepmaker et al. (2015)) where more and more pixels  Figure 1. Simulation of atmospheric absorption in the spectral range of SCIAMACHY's channel 8 for the absorbers taken into account by the retrieval algorithm. The grey shading marks the retrieval window for CO used by Borsdorff et al. (2016Borsdorff et al. ( , 2017, the yellow shading the window for H 2 O/HDO used by Scheepmaker et al. (2015), the green shading the extension of that window used in this work. product satisfy with the retrieved column c CH 4 and c H 2 O and the corresponding model predication c CH 4 ,TM5 and c H 2 O,ECMWF , respectively.
Moreover, due to radiometric performance issues of SCIAMACHY in the SWIR spectral range, the data has to be filtered with respect to outliers. Since water vapour in the troposphere is log-normally distributed (Schneider et al., 2006) and (as seen in 5 our data) so is its uncertainty, the 15.9 and 84.1 percentiles P 15.9 and P 84.1 of the logarithmic H 2 O and HDO uncertainty data distributions log e are considered to define Only data within 5σ around the median µ of the logarithmic distribution are used. A similar filter is applied to the root mean square of the spectral fit residual, c rms , where data points within 6σ around the median µ are kept. Thus, µ rms − 6σ rms < log c rms < µ rms + 6σ rms (4) Furthermore, the data are screened with respect to the number of iterations, N iter ≤ 12, and the solar zenith angle (SZA), Alternatively, SICOR allows to account for atmospheric scattering in the retrieval and so enhances the data yield of the retrieval. Based on the numerically very efficient two-stream radiative transfer model , the retrieval uses prior model information on CH 4 to infer the height h cld and optical depth τ cld of a scattering layer together with the total column of CO, H 2 O and HDO and the Lambertian surface albedo. Using this approach, our study considers for the first time the retrieval of the water vapour isotopologues for cloudy atmospheres and demonstrates the added value of the approach for 10 the data product. The same quality filter as described above is used. The cloud filtering is detailed in Sect. 4.
The performance of SCIAMACHY suffered from an ice layer building up on the SWIR detector (e. g. Gloudemans et al., 2005). Among others, this caused additional stray light in the instrument, meaning in first approximation an additive radiometric bias to the measurement. To compensate this effect, a radiometric correction as described by Borsdorff et al. (2016) is applied. Hence, a radiometric offset with spectral dependence described by a third order polynomial is fitted for clear sky 15 scenes above the Sahara region with a high albedo and so a high signal to noise ratio while fixing methane to the prior TM5  Fig. 2. It shows an increasing offset with growing ice layer which resets to zero at so-called decontamination events during which the detector was heated to remove the ice. Since 2005 no regular decontamination procedures were carried out with one additional decontamination at the beginning of 2009. Due to that the offset eventually stabilised. To demonstrate that the calibration is globally applicable, the figure also shows the offset determined above Australia. The results are very 5 similar and the deviation between both radiometric biases is less than 10 % when disregarding three outliers in the vicinity of decontamination events and a short period in the beginning of 2011 with a deviation of ∼14 %.
The strategy for the data product presented in this paper is to provide H 2 O and HDO columns for the whole period of the SCIAMACHY mission. The ratio δ D may be computed a posteriori from these data but is not a primary product.
3 Validation of the non-scattering retrieval of H 2 O and HDO 10 To validate the SCIAMACHY H 2 O and HDO total column, the observations are compared to ground-based FTIR measurements from 9 NDACC/MUSICA stations (Barthlott et al., 2017;Schneider et al., 2016Schneider et al., , 2012, which are listed in Table 1  the measurements of both instruments agree well. The bias, which is defined as the average of the difference between SCIA-    MACHY and MUSICA results, is −5.6 · 10 20 molec cm −2 , −1.4 · 10 17 molec cm −2 , and +11 ‰, respectively. The time series also shows that the data set is homogeneous throughout the whole mission period. Figure 4 shows a clear correlation between MUSICA and SCIAMACHY measurements for all three quantities with a Pearson correlation coefficient is 0.81, 0.81, and 0.63 for H 2 O, HDO, and δ D. The statistics for all low-altitude MUSICA stations are presented in Fig. 5. For H 2 O and HDO, the correlation is good 5 (between 0.74 and 0.97) with small biases for all stations (on average −3.6 · 10 21 molec cm −2 and −1.0 · 10 18 molec cm −2 , respectively). Also for δ D a good correlations between SCIAMACHY and MUSICA data is achieved for most stations, but it is only mediocre for Kiruna, Wollongong, and Lauder. The reason is that the seasonality in δ D is weak for those stations. To demonstrate that the low correlation coefficient does not indicate bad results, the time series for Lauder (for which the correlation is lowest) is shown in Fig. 6. The polar stations Eureka and Ny Ålesund have a relatively high bias in δ D. Although the To demonstrate the coverage of the non-scattering retrieval data set, world plots of H 2 O, HDO, and δ D averaged over the 20 first and second half of the mission period are presented in the left and right panels of Fig. 7, respectively. In H 2 O and HDO the latitudinal gradient with more water vapour abundance in the tropics and dryer air in polar regions is clearly visible. Above mountain ranges such as the Andes and the Himalayas, the water vapour abundance is reduced. This effect is discussed in more detail in the next section in connection with high-altitude stations. A continental effect with dryer air inland can also be seen, e. g. in North America, North Africa and Asia. In δ D, the features described by Scheepmaker et al. (2015) and Frankenberg 25 et al. (2009) are mostly reproduced. On the large scale, δ D decreases from the equator to the poles, which has already been mentioned by Dansgaard (1964). High altitudes also show a strong depletion in δ D. The continental effect is visible, for example, in North and South America, Asia, and even Australia. Enhanced δ D above the Red Sea seen by Frankenberg et al. (2009) is also reproduced. More importantly, the comparison between the left and right panels shows that the data set is consistent throughout the mission time.  To demonstrate the benefit of accounting light scattering by clouds in the retrieval of H 2 O and HDO, the corresponding water vapour isotopologue product of SCIAMACHY is compared with MUSICA measurements at elevated sites above 2000 m a. s. l.
Here, significant differences between the mean surface height of the SCIAMACHY ground pixels and the surface elevation of the ground site hampers the verification of the measurements using clear sky observations. However, when retrieving cloud at Jungfraujoch, only the partial column above 3580 m is observed, while SCIAMACHY in a collocation radius of 800 km around the site observes the whole column above the ground pixel, which elevation is much lower than Jungfraujoch for most scenes. With the large vertical gradient of water vapour abundance in the lower troposphere, this leads to the observed bias.
Moreover, the column-averaged δ D has lower values over mountains due to the HDO depletion with height.
To correct the SCIAMACHY observations for this altitude difference, the collocated ECMWF water vapour profile is scaled 15 to the total column observed by SCIAMACHY, and subsequently the scaled profile is truncated at the height of Jungfraujoch.
The vertical integration of this truncated profile provides the altitude corrected H 2 O and HDO columns depicted in the centre panels of Fig. 8. Obviously, the correction eliminates the large biases for H 2 O and HDO for the most part; the biases are reduced by more than a factor of 400 to 1.0 · 10 20 molec cm −2 and by more than a factor of 36 to 3.0 · 10 17 molec cm −2 , respectively.
However, for δ D computed from the corrected columns the bias remains due to the assumption about the same relative vertical 20 distribution of both isotopologues.
Next, cloudy scenes with cloud height around the station altitude are evaluated, since for these the result for the whole column is dominated by the part above the cloud due to the measurement sensitivity. The sensitivity of the retrieved column Table 2. Selection criteria for clear sky and cloudy sky conditions using scattering retrieval. Here h s is the altitude of the MUSICA ground site and max(SNR) is the maximum of the signal-to-noise ratio (SNR) of a spectrum.

Quantity filter
Clear sky filter  to changes in the vertical distribution of the water vapour isotopologues is described by the column averaging kernel (e. g. Rodgers, 2004). An example for clear sky and cloudy measurements is shown in Fig. 9. A detailed discussion of the column averaging kernel is given by Borsdorff et al. (2014). In case of an optically thick cloud (in the example at 3.5 km altitude), the retrieval is sensitive above the cloud but insensitive below the cloud, which underlines the selection criteria for cloudy sky observations in Table 2. Furthermore, to compare MUSICA data with cloudy SCIAMACHY observations, it is important to 5 realise that cloudy conditions usually involve a higher humidity than clear sky conditions. The ground based measurements are performed at the latter and so MUSICA data are first interpolated to the time of the satellite observations using ECMWF, similar as described by Borsdorff et al. (2016, Sect. 4.1) (compare Fig. 8e and f). A height difference between SCIAMACHY ground pixel and MUSICA station is corrected as described above. The right panel of Fig. 8 depicts the result. The bias in δ D between SCIAMACHY and MUSICA is significantly reduced and the difference between panels k and l demonstrates the 10 altitude dependence of δ D. Only satellite measurements with similar altitude sensitivity as the ground based observation yield good agreement. Figure 10 presents the statistics for the cloud retrievals above both high-altitude stations Jungfraujoch and Izaña. The correlation in H 2 O and HDO is generally good for all evaluations, however it is small for δ D, although it increases somewhat for the cloud retrieval. This is due to the noise in δ D being nearly as large as its seasonality. When looking at the bias, the 15 behaviour already seen in the time series is confirmed. It is large for all three quantities for uncorrected clear sky observations. The altitude correction reduces significantly the bias in H 2 O and HDO, while no change occurs to δ D. The latter is significantly reduced by regarding scenes with clouds at an altitude near the station height. The bias in H 2 O and HDO increases slightly when going from clear sky to cloudy scenes, but is still much lower than for the uncorrected case. This may be explained by the differences in cloud height compared to the station height accepted by the data selection.  For cloud retrievals from SCIAMACHY observations at polar latitudes the data quality is reduced. Due to the limited radiometric performance of SCIAMACHY, the retrieval of cloud properties from methane absorption feature is hampered by the low SNR for these challenging observation geometries and the dominance of water vapour absorption in the considered spectral range.
Due to these instrumental problems at polar latitudes, the non-scattering retrieval is preferred for the global SCIAMACHY 5 data set. For the new TROPOMI instrument, however, the cloud retrieval is expected to work for all latitudes, because it has a much better signal to noise ratio and a much better radiometric calibration, and the size of the ground pixel (and thus the averaging over areas with potentially varying cloud cover) is much smaller. The last point is also important since water vapour is known to vary considerably both spatially and temporally. In this light, the current study is also a preparation for the future investigations with TROPOMI.

Summary and conclusions
In this paper an new data set of satellite observations of column densities of the water vapour isotopologues H 2 O and HDO from the SCIAMACHY instrument is presented, which spans the whole mission period from January 2003 to April 2012. This is an addition by more than 4 years compared to Scheepmaker et al. (2015). To this end, the degradation of the instrument has to be mitigated by enlarging the retrieval window compared to Scheepmaker et al. (2015) and by eliminating the radiometric offset 15 via a calibration with the Sahara region as natural target. Atmospheric light scattering is ignored in the forward simulation of the measurements. Additionally, the possibility to infer simultaneously effective cloud parameter and the water vapour abundance is considered in a dedicated case study for measurements around high-altitude stations.
Due to the high sensor noise of SCIAMACHY, an adequate averaging in time and/or space has to be performed to obtain reasonable results. For instance, monthly medians are meaningful when averaging over a circle with 800 km radius. More 20 spatial resolution such as 1 • × 1 • global coverage can be achieved when sacrificing temporal resolution by averaging over several years.
The satellite observations are validated against low-altitude ground-based FTIR measurements from the MUSICA project within the NDACC network. In general, the agreement is good. The Pearson correlation coefficient for H 2 O and HDO is between 0.74 and 0.97, and the average bias is only −3.6 · 10 21 molec cm −2 for H 2 O and −1.0 · 10 18 molec cm −2 for HDO. For 25 δ D, the correlation is also good for most stations (between 0.56 and 0.79) with three exceptions where the seasonal variation is small. The bias is low except at polar stations (Eureka and Ny Ålesund). The decreased performance at high latitudes stems from imprecise measurements caused by the large sensor noise of SCIAMACHY in combination with difficult measurement geometries. The average bias in δ D over all stations is −8 ‰, but when excluding Eureka and Ny Ålesund it is only +1 ‰.
High-altitude stations (Jungfraujoch and Izaña) are treated separately in a case study with a retrieval that takes scattering 30 into account and additionally infers cloud parameters. Since both instruments observe different altitude ranges, the bias is large for clear sky measurements with 3.8 · 10 22 molec cm −2 for H 2 O, 1.0 · 10 19 molec cm −2 for HDO and 125 ‰ for δ D.
To correct for the altitude effect, partial columns above the station height are computed for SCIAMACHY by cutting an ECMWF water vapour profile scaled to the retrieved column. This yields good agreement between SCIAMACHY and MU-SICA in H 2 O and HDO, the mean bias is reduced to −2.2 · 10 20 molec cm −2 and 3.4 · 10 17 molec cm −2 , respectively. The bias for the a posteriori δ D cannot be corrected that way, however, because the profiles for H 2 O and HDO have the same shape, neglecting the increasing depletion with height.

5
The average bias in δ D is drastically reduced to 44 ‰ by considering scenes with optically thick clouds between 1000 m below and 1000 m above the station height and applying the altitude correction, since for such scenes the retrieval is sensitive predominantly above the cloud. For this approach, the SCIAMACHY results in all three quantities H 2 O, HDO, and δ D agree well with MUSICA. The corresponding biases in H 2 O and HDO are 1.4 · 10 21 molec cm −2 and 4.5 · 10 17 molec cm −2 , respectively.

10
The scattering retrievals work well for low and mid-latitudes, as demonstrated in the case study for high-altitude stations, but are unreliable for polar latitudes. This is caused by the limited radiometric performance of SCIAMACHY in combination with difficult observation geometries. For the newly launched TROPOMI instrument, which measures in the SWIR range with similar resolution than SCIAMACHY but with a much better signal to noise ratio and a much higher spatial resolution, the method is expected to work for all latitudes. Moreover, the inferred columns are expected to have a higher precision, eliminating 15 the necessity to average over long times or large areas. Thus future investigations will concentrate on TROPOMI.
Data availability. The full-mission SCIAMACHY H 2 O/HDO data set from the non-scattering retrieval described in this paper is available for download at ftp://ftp.sron.nl/pub/pub/DataProducts/SCIAMACHY_HDO/. The MUSICA data was downloaded from ftp://ftp.cpc.ncep. noaa.gov/ndacc/MUSICA/.
Competing interests. The authors declare that they have no conflict of interest.