Water vapour total columns from SCIAMACHY spectra in the 2 . 36 μ m window

Abstract. The potential of the shortwave infrared channel of the atmospheric spectrometer SCIAMACHY on Envisat to provide accurate measurements of total atmospheric water vapour columns is explored. It is shown that good quality results can be obtained for cloud free scenes above the continents using the Iterative Maximum Likelihood Method. In addition to the standard cloud filter employed in this method, further cloud screening is obtained by comparing simultaneously retrieved methane columns with values expected from models. A novel method is used to correct for the scattering effects introduced in the spectra by the ice layer on the detector window. The retrieved water vapour total vertical columns for the period 2003–2007 are compared with spatially and temporally collocated values from the European Centre for Mid-Range Weather Forecast (ECMWF) data. The observed differences for individual measurements have standard deviations not higher than 0.3 g/cm2 and an absolute mean value smaller than 0.01 g/cm2 with some regional excursions. The use of recently published spectroscopic data for water vapour led to a significant improvement in the agreement of the retrieved water vapour total columns and the values derived from ECMWF data. This analysis thus supports the superior quality of the new spectroscopic information using atmospheric data.


Introduction
Water vapour is, after nitrogen and oxygen, the most abundant atmospheric constituent.Its presence is essential for Correspondence to: H. Schrijver (h.schrijver@sron.nl)life, and has a great impact on human society.Because water vapour is the most important greenhouse gas, contributing the major part of the greenhouse effect, it has a huge influence on climatic conditions.In the anthropogenic changes of the greenhouse effect it plays an important role through various feedback mechanisms, the size and influence of which is not well established.In comparison to other greenhouse gases, the distribution of water vapour is much more variable and inhomogeneous.Its highest concentrations are found in the lower troposphere, especially in tropical regions.
In order to better understand the hydrological cycle and its consequences for, e.g., climate, data on the water vapour global distribution are indispensable.Satellite remote sensing can provide these, and since the advent of the space age, many satellites have contributed to the gathering of these data.
Measurements in microwaves were made by the TOVS (Tiros Operational Vertical Sounder) systems and by the SSM/I (Special Sensor Microwave/Imager) series of satellites (Liu et al., 1992).Ocean topography missions like Topex/Poseidon and Jason feature microwave radiometers as part of their measurement system.These instruments provide total water vapour columns in two daily passes.A restriction of this type of measurements is that reliable results are only obtained over the oceans, and the sensitivity in the boundary layer, where most of the water vapour resides, is relatively low.
The Advanced Infrared Sounder (AIRS) on EOS-Aqua combines measurements in the thermal infrared with a microwave radiometer.It produces total water vapour columns and humidity profiles.Also for this system, sensitivity in the boundary layer is limited.
Published by Copernicus Publications on behalf of the European Geosciences Union.
H. Schrijver et al.: Water vapour columns from SCIAMACHY 2.36 µm observations The Moderate Resolution Imaging Spectrometer (MODIS) on the Terra and Aqua satellites, and the Medium Resolution Imaging Spectrometer (MERIS) on Envisat exploit various spectral windows near 900 nm to retrieve total columns of water vapour (Gao and Kaufman, 2003;Li et al., 2003).
The GOME (Global Ozone Monitoring Experiment, on ERS-2), SCIAMACHY (SCanning Imaging Absorption spectroMeter for Atmospheric CartograpHY, on Envisat), and GOME-2 (on METOP) instruments have allowed to use atmospheric absorption spectra with higher spectral resolution in the visible/near-infrared wavelength range to obtain total vertical water vapour columns.Various wavelength windows have been explored: around 650 nm (Wagner et al., 2003(Wagner et al., , 2005(Wagner et al., , 2006)), around 700 nm (Noël et al., 1999(Noël et al., , 2004(Noël et al., , 2005(Noël et al., , 2008;;Mieruch et al., 2008), and around 590 nm (Maurellis et al., 2000;Lang et al., 2003).The advantage of using these windows is that the measurements are also sensitive to the boundary layer, where most of the water vapour resides.Moreover, results are obtained both over land and over oceans.A disadvantage is that the measurements rely on backscattered solar radiation, so that results are only obtained on the day-side of the orbit.Also, clouds hide part of the column, and they occur rather frequently in the relatively large ground pixels of these instruments.
This paper describes another opportunity for gathering information on the distribution of atmospheric water vapour: exploiting the short-wave infrared measurements by SCIA-MACHY around 2.3 µm.
The SCIAMACHY instrument was launched in 2002 on board ESA's Envisat satellite (Bovensmann et al., 1999;Gottwald et al., 2006).It measures atmospheric backscatter spectra in the range 220-2380 nm.To date, it is the only nadir-viewing satellite instrument that measures shortwave infrared spectra at high spectral resolution.In the wavelength range of the longest-wavelength instrument channel, 2265-2380 nm, nadir measurements have a spatial resolution of 120×30 km 2 or 240×30 km 2 , the latter usually at higher geographic latitudes.In this channel, spectral lines from CH 4 , CO, H 2 O, and N 2 O are found.Total columns for these species can be retrieved from these spectra.Because the lines overlap to a great extent, the columns for all species present in the retrieval window have to be retrieved simultaneously (Gloudemans et al., 2008).
Retrieval results from measurements in this channel have been reported by Gloudemans et al. (2005Gloudemans et al. ( , 2006) ) and de Laat et al. (2006de Laat et al. ( , 2007) ) applying the methods also used in this paper, and by Buchwitz et al. (2004) and Frankenberg et al. (2005) using somewhat different retrieval approaches.
This wavelength window shares the advantages and disadvantages described above for the visible region with respect to sensitivity, clouds and restriction to day side, but here the sensitivity for the boundary layer is even stronger.Moreover, since the scattering in the atmosphere (in the absence of heavy aerosol loads) is low, the radiative transfer in this wavelength window is simple, and thus gives rise to small uncertainties only (Gloudemans et al., 2008).
One major disadvantage of this wavelength region is that the albedo of water surfaces is usually very low (<0.01),so that measurements over oceans have a low signal-to-noise ratio and consequently a large error.Only in exceptional cases, such as sun glint or floating ice, can ocean measurements be used.Gloudemans et al. (2009) obtain CO columns above low clouds over the oceans, but this method is not applicable for H 2 O since most of the water vapour is located below the clouds.
In this article, water vapour retrieval results from five years of SCIAMACHY shortwave-infrared measurements are presented.In Sect. 2 the retrieval procedure is described.The results are presented in Sect. 3 and compared with values computed from ECMWF data.Section 4 discusses the dependence of the retrieval results on the assumed water vapour profiles.Also, the improvements reached by using a new spectroscopic data set for water vapour are considered.Finally, Sect. 5 summarizes the conclusions.

Method
The retrieval method used is the Iterative Maximum Likelihood Method (IMLM, Gloudemans et al., 2008, and references therein) This method has been successfully applied to the short-wave infrared SCIAMACHY spectra between 1.5 and 2.4 µm (e.g.Houweling et al., 2005;Gloudemans et al., 2005Gloudemans et al., , 2006Gloudemans et al., , 2009;;de Laat et al., 2006de Laat et al., , 2007)).The retrievals are done including all the features described by Gloudemans et al. (2009), using version 7.4 of the retrieval software, but a more elaborate calibration procedure had to be adopted for the effects of the ice layer on the detector window (cf.Sect.2.2).All retrievals reported in this paper were carried out using spectra in the 2353-2368 nm window.
The algorithm fits a modelled detector signal to the measured signal by varying the total column amounts of the trace gases that play a role in the selected retrieval window.The height-resolved information in near-infrared nadir measurements is very poor, and the IMLM retrieval algorithm does not attempt to achieve vertical resolution.Given the fact that the algorithm is based on scaling a priori atmospheric trace gas profiles, it is important to use realistic temperature and water vapour profiles (Gloudemans et al., 2008) because the absorption depth and shape of the spectral lines depend on the temperature distribution (and thus indirectly on the height distribution) of the molecules, which has a direct impact on the derived water vapour columns.
The vertical profiles of water vapour are very variable (a few examples will be shown in Sect.4.1), and hence the water vapour as well as the temperature profile are computed by interpolating ECMWF three-hourly 0.5 • ×0.5 • data to the observation time and location of the SCIAMACHY measurement as described in Gloudemans et al. (2008).For the vertical profiles of CH 4 and CO, the US standard atmosphere profiles are employed (Anderson et al., 1986).The influence of the assumed vertical profiles on the total column retrieval results of CH 4 and CO has been shown in Gloudemans et al. (2008) to be relatively low; for the dependence of the water vapour total columns on the assumed profile, see also Sect.4.1.Note that the scaling procedure of a priori profiles yields total column results that are fully independent of the total columns associated originally with these profiles, i.e. the ECMWF total water vapour columns.
The water vapour measurements in this wavelength range are sensitive down to the atmospheric boundary layer, as has been shown by sensitivity calculations.This is also illustrated by the total column averaging kernel (following the definition in Buchwitz and Burrows, 2004) for IMLM water vapour retrievals as shown in Fig. 1 which have been computed by the procedure used by Gloudemans et al. (2008) for carbon monoxide and methane.
At first, spectroscopic data from the Hitran 2004 database (Rothman et al., 2005) were used.Later, new H 2 O line data published by Jenouvrier et al. (2007) were substituted and CH 4 line air broadening data were taken from Predoi-Cross et al. (2006) (cf. Gloudemans et al., 2009).The use of the new H 2 O data led to better agreement between the retrieved water vapour columns and the values derived from ECMWF data, as shown in Sect.4.2.The use of the updated CH 4 data did not result in a significant improvement of the retrieval results.
The most important issues complicating the retrieval are the increasing number of detector pixels affected by radiation damage (Kleipool et al., 2007) and the ice layer growing on the near-infrared detector windows (Gloudemans et al., 2005(Gloudemans et al., , 2008)).The detector pixels affected by radiation damage, causing higher noise levels or disfunctioning, must be excluded from the retrieval using a dynamic (observation time dependent) mask.Mildly affected pixels are included assigning them reduced weights corresponding to their enhanced noise level.
The ice layer is caused by residual water from the spacecraft structure that condenses on the coldest parts of the instrument, in casu the shortwave-infrared detector windows.The ice layer has been removed periodically from the detector windows by heating the detector units in a number of decontamination periods, after which the ice layer reappeared.The last decontamination in the period discussed in this paper took place in December 2004, and the ice layer has appeared practically stable since July 2005.In order to get reliable water vapour total columns, it was necessary to develop a more rigorous correction procedure for the effects of the ice layer (transmission loss and scattering) than that employed in earlier retrievals (Gloudemans et al., 2005).This procedure is described in the following section.ig. 1.Total column averaging kernel (as defined by Buchwitz and Burrows, 2004) for IMLM retrievals of water vapour from 2.36 µm SCIAMACHY spectra, for different solar zenith angles and ground albedo 0.3.Values do not differ significantly for other albedos.

Novel correction for effects of the ice layer
The ice layer on the detector window affects the measured signal in three ways.First, it leads to a transmission loss that varies with window position (in other words, wavelength).This effect can be corrected for by employing various available calibrations, the most important of which are solar measurements.
The second effect is a decrease of the dark signal in relation to the decrease of the window transmission.In this wavelength domain, the dark signal is dominated by the thermal radiation from the optical bench, that has to pass through the detector window.Because the dark signal is calibrated every orbit, the correction for this effect poses no major problem.
The third effect, scattering by the ice layer, is the hardest to correct for.Photons of a certain wavelength are thereby scattered to a broad range (at least ±50) of detector pixels.Although the scattering from a given wavelength position is small (0.2-0.4%), the added effect from all wavelengths can easily lead to a "background" of some 20% of the total signal.In passing it is noted that, since scattering can be assumed to occur in all directions, a significant part of the scattered photons will not finish at all on a detector pixel, a loss that can not be distinguished from the earlier mentioned transmission loss by extinction by the ice.The thickness of the ice layer can, therefore, not be estimated in a straightforward way from the transmission reduction.
In earlier retrievals it was assumed that the effect of scattering could be sufficiently well described by a constant background, the value of which was calibrated by comparing the retrieved CH 4 total columns over a Sahara region with results calculated by atmospheric chemistry models (Gloudemans et al., 2005).Comparison of the retrieved water vapour total columns with collocated ECMWF data (as described in Sect.3.2) showed trends that appeared to correlate with changes in the ice layer, suggesting that the correction procedure employed was not entirely adequate.Analysis showed that in the neighbourhood of strong absorptions the background caused by scattering is lower, breaking down the hypothesis of a constant background.This is also true if the strong absorptions occur outside the retrieval window.
Therefore, a more refined correction procedure was adopted.Information on scattering characteristics can be obtained from spectral light source (SLS) measurements.The SLS emits a few strong and narrow emission lines and its detector spectra yield information about the dispersion characteristics (the slit function) and the scattering.The latter can technically be considered as an addition to the slit function.
Examples can be found in Fig. 2.
The interpretation of the SLS measurements is hampered by defective detector pixels.Moreover, in the far wings the measured signal is of the same order of magnitude as the measurement noise.Both problems are solved by fitting cubic spline functions to the measurements in valid pixels, weighting each individual pixel measurement according to its noise characteristics.The noise is further reduced by modelling the time evolution of the extended slit function by cubic spline functions.The model for the scattering in the form of a time dependent extended slit function is then used in the retrieval process instead of a fixed nominal slit function.
A few examples of the modelled slit function are shown in Fig. 2.

Cloud filtering
Clouds hide the lower part of the atmosphere and consequently a part of the water vapour column is not observed.For water vapour this has a stronger impact than for many other trace gases since most of the water vapour is usually located in the lower troposphere.
The cloud filter described by Krijger et al. (2005) has been applied to all results shown in this paper.However, it is known (Schrijver et al., 2006) that, especially in tropical regions, scattered small clouds are not always detected by this method, leading to underestimation of the water vapour columns in such regions.Indeed, comparisons with ECMWF data show this effect clearly.
The methane total columns that are retrieved simultaneously can be used as an additional cloud filter.Comparing the retrieved methane columns with their expected values, measurements where the methane columns are underestimated by more than 10% are rejected.In this comparison Upper panel: measurements of two spectral line source (SLS) lines at 2326.66 and 2337.94nm, respectively.Black: one day after decontamination; red: during the stable phase after mid-2005; blue: during a period of strong scattering.Bad and dead detector pixels have been blanked out.In the inset the same at full scale.Note that the peaks of the red and blue curves are at 62% and 40% of the value measured just after decontamination.Lower panel: modelled slit functions for the three cases in the upper panel for the range ±50 detector pixels, corresponding to about ±6 nm.The functions are normalised such that their maximum is equal to 1. the surface pressure of the scene is taken into account, the latter being derived from the ECMWF data with a correction for the difference between the surface elevation assumed by ECMWF and the average surface elevation of the SCIA-MACHY ground pixel.
The methane columns can not be used as a proxy for correcting the water vapour columns since the vertical profiles of methane and water vapour are too different (Fig. 3).The latter also limits the usefulness of the methane cloud filter: the limit of 10% for methane column underestimation used in this paper can, in extreme cases, still lead to an underestimation of the water vapour column of 30%.The importance of using this filter will be shown in Sect.3.2.3 Results

SCIAMACHY water vapour columns
Retrievals have been performed using all relevant SCIA-MACHY nadir observations between 1 January 2003 and 31 December 2007.The most notable interruptions are the already mentioned decontamination periods of one to two weeks aimed at dislodging the ice layer by heating the detector modules.Instrument noise related errors (cf.Gloudemans et al., 2008) for individual measurements range, in 2003, from 0.003 g/cm 2 for desert-like scenes where the best signal-tonoise ratio is achieved, to about 0.2 g/cm 2 for scenes with low albedo, e.g., tropical vegetation.Due to detector degradation by radiation damage to individual detector pixels, these numbers gradually increase to values that are about 1.5 times higher in 2007.
Monthly averaged H 2 O columns using all cloud-filtered measurements above a certain signal level and solar zenith angle below 80 • have been computed on a 1 • ×1 • grid applying a weight based on the instrument noise errors of the individual water vapour total columns (cf.de Laat et al., 2007); Fig. 4 shows the results for 2005.A number of seasonal variations can be clearly distinguished, such as the annual cycles of Australia, India, the change in latitude of the maximum humidity in tropical Africa, and the variations in the Sahel region.Humidity variations in Northern America and Europe related to the variations in temperature are also obvious.In interpreting these maps it has to be kept in mind that large spatiotemporal differences exist in the instrument noise errors; for this reason, the distribution of these errors for the same monthly averages is shown in Fig. 5.The dynamical range of the errors in the monthly averages is even larger than that for the individual measurements since the scenes with the highest signal-to-noise ratio (deserts) usually have the largest number of cloud-free measurements.
Other years show similar patterns.The full time series of the results is represented in Fig. 6 averaged over monthly periods and 5-degree latitude intervals, displaying the seasonal variations in the water vapour global pattern reflecting the variations in temperature.The values in the tropics show a yearly maximum during northern summer.

Comparison with ECMWF data
The retrieved water vapour column results have been extensively compared with total columns derived from collocated ECMWF data.Humidity data with 0.5   resolution and 3-h temporal resolution were interpolated to the time and centre of the ground scene of each SCIA-MACHY measurement.Monthly averaged differences between the retrieved and the ECMWF values for all cloud-filtered measurements, both weighted as before with the instrument noise error, are shown in Fig. 7 for all months of 2005.The agreement is generally good, with some larger seasonal differences occurring in the Sahara-Sahel region and northern Australia, where the retrieved humidity is sometimes considerably larger than found in the ECMWF data.Noël et al. (2005) have noted comparable discrepancies for their SCIAMACHY 700 nm retrieval results.The ECMWF model assimilates data from SSM/I and some other satellites, that give good coverage over oceans and limited coverage over land (Flentje et al., 2007), and radio sonde data where coverage is limited, especially in desert areas.Satellite measurements in the visible or short-wave infrared region can give a valuable contribution to our knowledge of the global water vapour distribution.In a similar fashion as in Fig. 6, the time behaviour of these differences are displayed in Fig. 8.Besides the already mentioned differences at latitudes corresponding to Sahara and Australia, there are large differences in some months in 2003, where the number of observations was restricted due to frequent decontaminations (cf.Sect.2.1).The lower values occurring seasonally between latitude 15 • S and 15 • N are most probably due to imperfect cloud filtering (cf.Sect.2.3).Already, by introducing the methane filter, as explained in that section, the differences were brought back from values sometimes reaching −2 g/cm 2 to the current −0.3 g/cm 2 shown in Fig. 8. Figure 9 illustrates the importance of introducing this additional filtering.Besides the imperfection of the cloud filter (as noted in Sect.2.3), the overestimation, in convective areas, of the humidity by the ECMWF model as noted by Flentje et al. (2007) may be an additional explanation of the remaining differences.
It is also possible to correlate individual cloud-free measurements with the ECMWF values.An example, for a month of observations, is shown in Fig. 10.The slope of the linear regression is indicated in the figure, the intercept is very close to zero.Because differences occurring sometimes in the Sahara-Sahel region and northern Australia may disturb the analysis, measurements from these regions have been excluded in Fig. 11.The slopes are generally close to 1, as can be seen in Fig. 12 where the slopes obtained for all months of 2003-2007 are given.The standard deviation of all SCIAMACHY total water vapour column values corresponding to a given ECMWF value is not larger then 0.3 g/cm 2 for ECMWF values above 2 g/cm 2 and decreases for lower ECMWF values.The range of these variations is larger than expected from instrument noise.Presumably the variations are caused by natural variations in the humidity not represented in the ECMWF model underlying the comparison values.
An alternative way of comparing the retrieval results with ECMWF values was proposed by Noël et al. (2005).Differences for all relevant daily observations are averaged globally and their standard deviations computed.Results from such an analysis for the entire five-year period are given in Fig. 13, again both including and excluding the Sahara and Australia.Daily averaged biases are small (mostly within ±0.05 g/cm 2 ) with some seasonal excursions due to the above-mentioned  cloud effect.The standard deviation is generally below 0.3 g/cm 2 , except in 2003, reflecting the sometimes poor coverage caused by the frequent decontaminations in that year.These numbers are somewhat smaller than those shown by Noël et al. (2005).This does not necessarily mean that their results are less accurate, but is possibly related to the coarser ECMWF grid employed by these authors.

Dependence of results on assumed water vapour profile
In order to assess how much the retrieved water vapour total columns depend on the adopted water vapour vertical profiles, the retrievals have been repeated using a fixed water vapour profile.The standard profile defined by Ander-  (1986) was selected for this purpose.Differences between the global monthly averaged water vapour columns obtained this way and those retrieved earlier using the ECMWF profiles were computed.An example is shown in Fig. 14.Differences are generally less than 0.1 g/cm 2 and conclusions about the comparisons with ECMWF derived total columns are not altered (not shown).Larger differences occur where the water vapour columns differ to a large extent from the standard profiles: this is illustrated in Figs. 15.Where water vapour is over-abundant in the boundary layer (compared with the standard profile) retrievals with the standard profile lead to higher water vapour total columns (regions A and D) and vice versa (region C).This is in agreement with what is known about the dependence of water vapour absorption cross sections on temperature.

Use of recent spectroscopic data
As noted in Sect.2.1, the water vapour retrievals were originally performed using the spectroscopic data from the Hitran 2004 database (Rothman et al., 2005).Later, updated H 2 O data (in particular, air-and self-broadening parameters) were published by Jenouvrier et al. (2007).Retrieved water vapour columns obtained by using either data set can be compared with the ECMWF values as described before.A convenient way to describe the result of such comparisons is the slope of the regression of SCIAMACHY versus ECMWF values (as in Fig. 11).Figure 12 shows the monthly slopes for the two spectroscopic data sets.Use of the data by Jenouvrier et al. (2007) result in slopes closer to unity, suggesting that the new spectroscopic data constitute a substantial improvement.
Use of new CH 4 spectroscopic data (broadening coefficients) by Predoi-Cross et al. (2006) does not lead to a significant change in the results (cf.Gloudemans et al., 2008).

Conclusions
This paper presents, for the first time, an analysis of total water vapour columns retrieved from shortwave infrared spectra measured by SCIAMACHY.The conclusions are threefold.
SCIAMACHY measurements in the 2.36 µm window provide a worthwhile extra opportunity to measure global total water vapour columns over land, with high sensitivity near the surface where most of the water vapour resides.With some regional exceptions, the agreement with columns computed from ECMWF data is good.
The SCIAMACHY atmospheric measurements support the quality of the new spectroscopic water vapour data (Jenouvrier et al., 2007) since their use results in an improved agreement between the retrieved water vapour total columns and the values derived from ECMWF data, in particular a reduction in the bias.
The correction method for the scattering effects introduced by the ice layer on the SCIAMACHY shortwave-infrared detector, as used in earlier work for the retrieval of carbon monoxide, leads to unsatisfactory results for water vapour.By using a novel method, it is possible to correct in a consistent manner for these effects.
Fig. 2.Upper panel: measurements of two spectral line source (SLS) lines at 2326.66 and 2337.94nm, respectively.Black: one day after decontamination; red: during the stable phase after mid-2005; blue: during a period of strong scattering.Bad and dead detector pixels have been blanked out.In the inset the same at full scale.Note that the peaks of the red and blue curves are at 62% and 40% of the value measured just after decontamination.Lower panel: modelled slit functions for the three cases in the upper panel for the range ±50 detector pixels, corresponding to about ±6 nm.The functions are normalised such that their maximum is equal to 1.

Fig. 3 .
Fig. 3. Vertical profile of water vapour for a case of high humidity in the lower troposphere (solid line), and the standard methane profile from Anderson et al. (1986) (dashed line).

Fig. 7 .
Fig. 7. Monthly averaged differences between SCIAMACHY total water vapour columns and values derived from spatially and temporally collocated ECMWF data for 2005 on a 1 • ×1 • grid.

Fig. 8 .
Fig. 8. Monthly mean differences between total water vapour columns from SCIAMACHY and values derived from ECMWF data, averaged over 5 • latitude bins, for the period January 2003 to December 2007.

Fig. 9 .Fig. 10 .
Fig. 9. Monthly averaged differences between total water vapour columns from SCIAMACHY and values derived from ECMWF data averaged over six tropical latitude zones.Upper panel: using cloud filter following Krijger et al. (2005) (SPICI).Lower panel: additionally rejecting observations with CH 4 total column below 90% of expected value.

Fig. 13 .Fig. 14 .
Fig. 13.Daily mean (black) and standard deviation (blue) of difference between SCIAMACHY retrieved water vapour columns and values derived from collocated ECMWF humidity data.Upper panel: all observations.Lower panel: observations over Sahara and Australia excluded.Red dashed lines separate three-month intervals.

Fig. 15 .
Fig. 15.Standard atmospheric water vapour profile used in test retrievals and typical profiles for locations A-D indicated in Fig. 14.