The Spatial Heterodyne Observations of Water instrument (SHOW) is a
limb-sounding satellite prototype that utilizes the Spatial Heterodyne
Spectroscopy (SHS) technique, operating in a limb-viewing configuration, to
observe limb-scattered sunlight in a vibrational band of water vapour within
a spectral window from 1363 to 1366 nm. The goal is to retrieve high
vertical and horizontal resolution measurements of water vapour in the upper
troposphere and lower stratosphere. The prototype instrument has been
configured for observations from NASA's ER-2 high-altitude airborne remote
science airplane. Flying at a maximum altitude of
Water vapour is an extremely important trace species in the upper troposphere and lower stratosphere (UTLS) region of Earth's atmosphere. Indeed, it is well known that the abundance and distribution of water vapour in the UTLS is strongly linked to climate processes (Gettelman et al., 2011; Sherwood et al., 2010). However, research over the past couple of decades has indicated that the distribution of water vapour and its link to climate processes is not fully understood. For example, the impact of stratosphere–troposphere exchange (STE) and the formation of the tropopause inversion layer (TIL), as well as the role of water vapour as mechanism for radiative feedback still requires detailed study (Randel and Jensen, 2010). The primary factor limiting the ability to perform a detailed study of these processes is the lack of accurate long-term global measurements that have a high vertical resolution in the UTLS.
Remote sensing of water vapour can be achieved using many different
techniques and platforms (space-based, balloon, in situ, airplane, or
ground-based platforms). While each technique has its advantages, the best
combination of vertical resolution and global coverage of trace species in
the UTLS is achieved using a limb-viewing instrument operating from a
low-earth orbit satellite. Indeed, the current measurement record of water
vapour has been enriched with observations from limb-viewing instruments such
as the Michelson Interferometer for Passive Atmospheric Sounding (MIPAS)
(Fischer et al., 2008; Milz et al., 2005; Stiller et al.,
2012), the Microwave Limb Sounder (MLS) (Hurst et al., 2014,
2016; Sun et al., 2017), and the Scanning Imaging Absorption
Spectrometer for Atmospheric CHartographY (SCIAMACHY) (Rozanov et al., 2011).
The Atmospheric Chemistry Experiment (ACE) instrument, which performs
measurements in the solar occultation mode has also made significant
contributions to this data record (Carleer et al., 2008). These instruments
provide between 3 and 5 km vertical resolution in the UTLS with
<
The Spatial Heterodyne Measurements of Water (SHOW) instrument is a
prototype satellite instrument that is being developed in collaboration
between the Canadian Space Agency (CSA), the University of Saskatchewan
(USASK) and ABB Inc., to provide high spatial resolution measurements
(< 500 m) of the vertical distribution of water vapour in the UTLS
from a space-borne platform (Langille et al., 2017, 2018).
The instrument utilizes the Spatial Heterodyne Spectroscopy (SHS) technique
(Connes, 1958; Harlander, 1991), operating in the limb-viewing configuration
to observe limb-scattered radiation within a vibrational band of water
centred at 1364.5 nm. The prototype version of the instrument, discussed in
this paper, is configured for measurements from NASA's ER2 high-altitude
airborne science airplane. High-altitude measurements (
The SHOW measurement approach is unique in several ways. For example, it is the first demonstration of the measurement of atmospheric water vapour using the SHS technique, specifically, using limb-scattered sunlight. In addition, the majority of the SHS instruments that have been developed have been used to observe well-isolated emission features, such as in the case of SHIMMER (Harlander et al., 2002), and the DASH type instruments that are used to remotely detect motions in the upper atmosphere using well isolated airglow emissions (Englert et al., 2007, 2015, 2017). Therefore, SHOW is also one of only several SHS instruments (Englert et al., 2009) that has been developed to observe absorption features over a broader spectral range.
In two earlier publications, we presented the design of the prototype instrument (Langille et al., 2017) and demonstrated that the configuration and sensitivity of the instrument was suitable for airplane measurements of water vapour (Langille et al., 2018). This sensitivity study was performed assuming an ideal instrument configuration with realistic signal levels and Poisson noise added to the signals. A non-linear optimal estimation retrieval algorithm was developed to invert the spectral signatures and retrieve water vapour. Assuming the ideal configuration and an airplane altitude of 22 km, it was shown that SHOW is capable of providing vertically resolved measurements of water vapour with 1 ppm accuracy with 500 m–1 km vertical resolution in the 8–18 km altitude range.
In this paper, we focus on the level 0 to level 1 processing and characterization of the SHOW measurements that were obtained from NASA's ER-2 airplane during an engineering flight performed on 18 July 2017. During the flight, a radiosonde was launched from the Table Mountain Jet Propulsion Laboratory (JPL) facility located close to Wrightwood, CA, at the same time that SHOW observed the same approximate column of air. This provides a direct estimate of the in situ water vapour abundance for the coincident SHOW measurements.
In practice, it was found that non-ideal instrument effects associated with the instrument configuration introduced systematic variations in the spectra that must be appropriately characterized prior to performing water vapour retrievals. In this paper, we present the work that was performed to characterize these effects. It is shown that knowledge of the instrument configuration can be utilized to develop an instrument model that is optimized to predict the systematic variations that are observed in the SHOW spectra. The approach is validated by using the measured in situ water vapour abundance as input to a forward model to simulate the expected limb-scattered radiance profile for the coincident measurements. This radiance profile is then used as input to the SHOW instrument model to simulate the expected SHOW interferograms and spectra. The ability of the model to predict systematic variability in the level 1 spectra is examined. Preliminary water vapour retrievals are presented and compared to the in situ measurements. These measurements are used to demonstrate the SHOW measurement technique and examine the performance of the instrument.
The primary scientific goal of the SHOW instrument concept is the realization
of high vertical resolution sampling (< 500 m) of the water vapour
distribution in the UTLS region from a low-earth orbit satellite with
<
The NASA ER-2 airplane
The SHOW prototype instrument is shown mounted inside the wing pod of the ER-2
airplane in Fig. 1. The instrument observes the limb through a
forward-looking window that is located at the front of the wing pod (see
Fig. 1a). Limb-scattered sunlight enters the instrument through a
periscope assembly that is mounted at the input of the optics box. This
assembly consists of two mirrors that align the optical axis of the
instrument with the wing pod window, as shown in Fig. 1b. The aft optics
are configured to provide a 4
The vertical sampling at the limb is defined by the airplane viewing geometry from the ER-2, and the configuration of the optical system and the SHS. In general, a SHS produces an interferogram image that corresponds to a set of overlapping Fizeau fringes with spatial frequencies that depend on the separation of the signal wavelength from the heterodyne (Littrow) wavelength. For SHOW, the SHS and aft optics are arranged so that the interferograms are aligned horizontally. The fore-optics is configured to be anamorphic (Langille et al., 2017), so that the interferograms that are imaged conjugate to the limb in the vertical dimension at the detector, whereas each interferogram sample in the horizontal averages over the horizontal scene information. This ensures that the spatial information contained in the horizontal does not contaminate the spectral information recorded at a particular row.
Each SHOW measurement provides an interferogram image that has 295 vertical
rows and 494 interferogram samples. The geographic location of the range of
tangent altitudes within the field of view are determined from the airplane
attitude information (altitude, heading, pitch, roll, etc.). Variations in
the pitch of the airplane during flight result in different interferogram
rows observing different tangent altitudes during different parts of the
flight. Post-flight, the airplane attitude information is used to obtain a
one-to-one mapping of each interferogram row to tangent altitude. For the
measurements presented in this paper, the lowest observed tangent (set by
the pitch of the airplane) varies from
SHOW ER-2 instrument parameters.
During level 0 to level 1 processing, the raw interferogram signals are
converted to calibrated interferograms and these calibrated interferograms
are converted into corrected spectra. For each row in the image, a water
vapour spectrum is obtained with an unapodized spectral resolution of
Control and operation of the instrument from the ground is achieved using the flight control software developed by ABB Inc. and ground support equipment developed by USASK. Communication with the instrument is performed using the NASA Airborne Science Data and Telemetry system (NASDAT) and the airplane attitude information is also monitored and stored on board the airplane. The control software and communication system allows for near-real-time downloads of images, as well as the airplane attitude information (altitude, speed, pitch, roll, heading, etc.) during the flight. The system was used for configuring the science modes (acquisition times, frame rates, focal-plane array (FPA) temperature, etc.) and to monitor housekeeping data that are vital to the instrument survival, such as printed circuit board (PCB) temperatures and SHS temperatures.
A 29 GB data storage unit is used to store the SHOW measurements on board the
instrument. The SHOW detector array is a Raptor OWL 640 camera with
Post-flight, the raw data are downloaded and processed to generate a level 0 netCDF file for each specific science mode chosen during the flight. The netCDF file stores the raw image files, along with the measurement configuration, such as integration time, image size, UTC time, and housekeeping data. The corresponding airplane attitude information is stored in a separate data file provided by NASA. The raw data are processed using the level 0 to level 1 processing chain to produce level 1 calibrated interferograms and level 1 spectra. The level 1 data are stored as a netCDF file with each data record containing 1 min of data.
An important aspect of the design of the SHOW prototype is the thermal
stability of the instrument. The two most temperature sensitive components
are the narrow-band filter and the SHS, both of which were designed for
operation near room temperature. Since the SHS is not thermally compensated,
the heterodyne wavelength of the SHS shifts slightly with temperature. For
the SHOW instrument this shift occurs at a rate of approximately
In the worst case, the inside of the wing pod will reach the ambient
temperature where the extreme temperatures can easily reach 40
As noted in the introduction, non-ideal instrument effects associated with the instrument configuration result in systematic variations in the observed spectra that need to be characterized to perform water vapour retrievals. In order to understand and interpret these variations, it is important to review some key concepts regarding the SHS technique. The most rigorous theoretical background of SHS is provided by Harlander (1991, 1992). Some of the more recent applications of the SHS technique for the remote sensing of the atmosphere can be found in Englert et al. (2015). The framework and notation that is used here to describe the SHS technique follows directly from work that was performed at a US naval laboratory (Englert et al., 2004, 2006).
Conceptually, the SHOW instrument operates in a similar manner to a
traditional limb-imaging Fourier transform spectrometer (FTS) instrument such as MIPAS (Fischer et al.,
2008); however, in this case, a limb image of the water vapour absorption
spectrum is obtained from each frame without vertical scanning and without
moving parts in the interferometer. The basic configuration of a SHS is
similar to a Michelson interferometer that has its plane mirrors replaced by
fixed, tilted diffraction gratings. Field widening of the instrument is
achieved by placing appropriately selected prisms in the arms of the
interferometer. Collimated light enters the spectrometer and is incident on
a beam splitter that transmit and reflect 50 % of the
input radiation down the two arms of equal length of the interferometer, respectively. At the
end of each arm the radiation is incident on a diffraction grating that is
tilted by the Littrow angle
For the SHS, each pixel essentially behaves as a separate detector, making
the system extremely sensitive to variations in the intensity across the
interferogram. Non-ideal flat-field calibrations, bad (i.e. dead/hot pixels), and
pixel-to-pixel non-uniformities, etc. result in systematic variability in
the interferograms and subsequently errors in the retrieved spectra. The
presence of phase distortions in the interferograms can also be important.
In general, the basic form of a SHS interferogram, which includes these
effects, can be described (see Englert et al., 2006) using Eq. (2).
A robust theoretical framework has been developed by Englert et al. (2004, 2006) that provides a template to correct the interferogram
described by Eq. (2). This includes the flat-field correction and phase
distortion. Once the flat-field correction is performed, the corrected
interferogram can be written as in Eq. (3), where we have substituted
Observe that a SHS cannot distinguish between signals that are generated by
wavenumbers separated by the same amount on either side of Littrow (
In the prototype version of the SHOW instrument, the aliasing effect is
complicated by the presence of a linear phase variation:
On the other hand, taking the FFT in the presence of the linear phase term,
In this paper, we attempt to use an instrument model to predict the aliasing
behaviour with the assumption that the phase term is linear and given by
The primary source of uncertainty in the SHOW interferogram samples is
associated with photon counting statistics and the inter-pixel variability
of the detector response. Poisson statistics describe the measurement of the
incident photon signal from the atmosphere (
Generally speaking, flat-field calibrations are performed to remove the intensity variations associated with optical effects and a non-uniformity correction is performed to minimize the impact of the relative pixel response. Ideally, the relative pixel response is corrected separately from the flat-field variations by characterizing the PRNU prior to installing the camera in the instrument. For the SHOW prototype, this was not possible since the camera was installed by ABB prior to shipping the instrument for the calibration work. We characterize the approximate PRNU of the camera in the presence of optical effects by performing a two-point non-uniformity correction using white light flat-field measurements. While this approach cannot be used to completely separate the flat-field term from the non-uniformity correction term, it does give a good indication of the approximate relative pixel response of the detector. For the measurements presented in this paper, the flat-field correction is performed in combination with the non-uniformity correction by obtaining white light flat-field measurements at signal levels that closely match signal levels recorded during flight. Therefore, each flat-field frame corrects for the optically related variations as well as the inter-pixel response variations.
The SHOW flat-field calibrations are performed using the flat-field approach
developed by Englert et al. (2006) and described in Sect. 9.4. After
applying the flat-field corrections, the remaining inter-pixel variations
are associated with the residual photo-response non-uniformity of the
detector. Techniques have been developed to characterize these variations in
the interferogram images in the laboratory, as well as their influence on
the spectral measurements. In the interferogram dimension, these variations
add a relative error to the signal at each sample. Assuming a random
relative pixel variation
Although the relative pixel response behaves quasi-randomly (across several hundred interferogram samples) in the interferogram domain, the relative response is a fixed pattern and the observed variability is fixed across the interferogram row for a fixed intensity level. The effect can be viewed as adding or subtracting a small fixed perturbation to each sample (a similar effect is observed with digitization noise in FTS (Davis et al., 2001). The FFT of this distribution produces an associated pattern in spectral space that remains fixed for the same signal level. This fact is used during the level 0 to level 1 data processing in order to characterize this effect. It is shown that a multiplicative secondary correction is feasible for the SHOW ER-2 airplane measurements that effectively minimizes the impact of this variability.
Assuming a negligible inter-pixel variability, the total uncertainty
The SHOW level 0 data consist of the raw SHS interferograms and noise estimates, as well as the airplane attitude information (time, geographical location, speed, pitch, yaw, etc.). These raw data are processed in several stages, as shown diagrammatically in Fig. 2. In the first stage, raw interferograms are corrected for dark signal and bad pixels (of which there are very few) are removed by performing a nearest neighbour interpolation. Then the flat-field correction is applied. Finally, the DC bias is removed from the interferogram signal in order to obtain the final level 1A interferograms. In the second stage, corrected spectra are extracted from these interferograms. This consists of two primary steps. The first step is the application of Hanning apodization. The second step is the application of an FFT to each interferogram row to obtain the complex spectrum corresponding to each line of sight in the image. The amplitude of the complex spectrum is taken at each row to form a vertically resolved image of the water vapour absorption spectra.
SHOW level 0 to level 1 processing chain.
As noted earlier, the airplane pitch varies as a function of time and this variation results in each row of the detector observing a range of tangent altitudes at the limb. As part of the level 0 to level 1 processing, the airplane attitude information is used to obtain a mapping of the interferogram rows in each image to the corresponding line of sight (LOS) at the limb. Therefore, the level 1 data consist of the calibrated interferograms and spectra, as well as the geometrical information required to map each interferogram row in the image to a particular tangent altitude at the limb.
The pitch variation of the airplane is also used to obtain a secondary correction that can be applied to minimize the impact of the systematic variations in the spectra associated with the residual relative pixel response in the interferogram signals. In practice, over a short period of time, it can be assumed that the radiance level at a particular altitude remains constant. Therefore, as the airplane pitches up and down, the signal level crosses different rows and this variability can be tracked in the spectral images. This allows this detector related systematic variability to be characterized and corrected in the measurements. This correction, discussed in detail in Sect. 11.2, produces level 1C corrected spectra.
The SHOW retrieval algorithm utilized to process the flight data builds on
the SHOW sensitivity study presented in a previous publication (Langille et
al., 2018). The approach utilizes the non-linear optimal estimation formalism
described by Rodgers (2000) to extract water vapour profiles from the
vertically resolved spectral measurements. The retrieval performs an
iterative step given by Eq. (6) in order to find the next best estimate of
the state vector,
The retrieval of the water vapour profile requires a forward model that accurately predicts the instrument behaviour. For a typical FTS instrument, this is achieved by convolving the instrument line shape (ILS) with a high-resolution, forward modelled spectra. Generally, the ILS is obtained by taking measurements with a quasi-monochromatic calibration source. For the SHOW instrument, the convolution with the ILS at each row in the instrument produces a spectrum that does not capture the systematic variations associated with the aliasing effect discussed in Sect. 5. In the current work, we have performed a detailed instrument characterization to obtain appropriate calibrations that are utilized during level 0 to level 1 processing and we optimized an instrument model to accurately predict the systematic variations in the observed spectra. In Sect. 12, we utilize this model in the retrieval to extract water vapour profiles from measurements that were taken during a SHOW ER-2 engineering flight on 18 July 2017.
The SHOW instrument consists of several primary components. This includes the field widened SHS, the optical system (anamorphic fore optics and exit imaging optics), and the FPA. The configuration of these components introduces variations in the interferogram signals that must be characterized and calibrated in order to obtain the highest quality measurement. As noted in Sects. 5 and 6, treating each pixel in each interferogram row as a separate detector requires knowledge of the behaviour of each pixel in the array. Converting the raw level 0 interferograms to corrected level 1 spectra requires detailed knowledge of the SHS configuration and the imaging quality of the instrument. In addition, knowledge of the vertical field of view is essential to mapping the rows of the detector to the tangent altitudes at the limb. We also utilize the knowledge of the instrument configuration to develop the optimized instrument model that is applied to predict the systematic effects associated with aliasing.
The FPA utilized in the SHOW-ER2 instrument is a commercial off the shelf
(COTS) OWL 640 InGaAs detector manufactured by Raptor Photonics. The FPA has
SHOW OWL 640 detector parameters characterized in the laboratory.
Values with a
A total of 46 bad/defective pixels were identified within the imaged field
of view (FOV) of the SHOW FPA. A handful of pixels around the edge of the
FPA also exhibit odd behaviour and have been identified as bad pixels. During
data processing, a bad pixel map is utilized to identify these pixels and we
interpolate over these pixels using nearest neighbour interpolation. To
correct for dark signal we obtain average dark calibration frames for a
range of exposure times by averaging several hundred dark measurements at
each setting. The observed dark signal was found to vary non-linearly in the
SHOW operating range using less than 2 s exposures. For the 0.5 Hz
acquisition setting, the mean bias level of the detector was found to be
approximately 1974 DN and the average dark signal across the frame using the
1800 ms exposure setting was found to be
Propagation of error in the signal measurement using Eq. (4) requires
knowledge of the gain, readout noise and the magnitude of the relative pixel
response. The approximate gain (electron DN
The relative pixel response was characterized in the laboratory by
performing a two-point non-uniformity correction using flat-field
measurements. This approach inherently includes some variations from optical
effects and it was found that the PRNU associated with an uncorrected frame
results in a
The 4
An example raw image obtained by illuminating the entrance aperture with
white light from an 8 in. LabSphere integrating sphere is shown in Fig. 3a. This image corresponds to the full frame provided by the (512
pixel
An example dark corrected white light image
Scattered sunlight from the limb is collimated at the input to the instrument, therefore, a particular off-axis angle illuminates a single row on the SHOW detector. As the off-axis angle is varied, the illuminated row also changes. Knowledge of this mapping is critical in order to determine accurate lines of sight that are used to obtain accurately georeferenced spectra. The SHOW field of view only fills a portion of the detector frame – called the SHOW FOV from pixel 197 to 491 in the vertical and 9 to 502 in the horizontal. The outer edges of the red region in Fig. 3b have been removed from the FOV to ensure no edge effects are present. Later in the paper, only this FOV is shown, rather than the full frame that is shown in Fig. 3.
SHOW measured vertical FOV.
This vertical field of view was characterized in the laboratory at ABB Inc.
by directing a well-collimated beam into the instrument entrance at
different incident angles. For each incident angle, the position of the
illuminated row was determined. A plot of the illuminated row number vs.
incident angle is shown in Fig. 4. A LMS linear fit was performed to
determine the slope of
The flat-field calibration is performed using the approach developed by
(Englert et al., 2006) and takes into account the flat-field variations from
the optics, as well as slight differences between the two arms of the
interferometer. First, the entrance aperture of the instrument is uniformly
illuminated with white light. Then separate images are obtained with each
arm of the interferometer (arm A and arm B) blocked using an opaque
material. Hundreds of measurements are obtained, dark corrected, and then
averaged to minimize the Poisson noise. Taking these images to
be
Example SHOW flat-field correction images obtained using a 250 ms
exposure time to obtain a quarter full well signal. The FF1 term is shown in
Example FF1 and FF2 correction images (normalized) that were obtained in the
laboratory using an 8 inch aperture LabSphere integrating sphere are shown in
Fig. 5a and b, respectively. Optical variations of
For the SHOW data processing, we take the FF2 term to be equal to 1 and just apply the FF1 term which corrects for the optical variations. This also corrects for the inter-pixel variations associated with the relative pixel response since the flat-field images are obtained without performing a non-uniformity correction. However, the non-uniform pixel response of the detector depends on the intensity of the incident radiation. Therefore, we obtain flat-field correction images within the range of the intensity levels that are expected during flight. The flat-field correction terms shown in Fig. 5 were obtained with roughly a quarter of the full well and an exposure time of 250 ms. The flight measurements are corrected using the calibration frames that are obtained at the closest matching intensity level. This ensures that the flat-field correction also appropriately corrects for the relative pixel response.
The primary parameters that are used to characterize the configuration of
the SHOW SHS system are the Littrow wavelength and the spectral resolution.
The SHS is designed to have an unapodized spectral resolution of:
Characterization of the SHS configuration was performed in the laboratory by
uniformly illuminating the entrance aperture of SHOW with light from a
krypton calibration lamp. The krypton lamp contains a well-isolated line at
1363.422 nm (in air) that lies within the SHOW passband. Measurements with
the krypton lamp were used to measure the Littrow wavelength at the
operating temperature and determine the spectral resolution of the
instrument. Measurements of a white light source were utilized to
characterize the full instrument. The instrument was purged with dry
nitrogen for 24 h prior to the characterization work in order to ensure
a stable operating environment and a SHS temperature of 23
Krypton fringes observed with the SHOW SHS instrument at
23
An example image of the krypton fringes obtained with SHOW is shown in
Fig. 6a. The rotation of the fringes is due to the presence of a slight
tilt in the
The krypton spectrum, obtained by taking an FFT of an interferogram row close to the centre of the image is shown in Fig. 6b in blue. The theoretical spectrum is obtained by taking the FFT of a simulated interferogram that is modelled by assuming a perfectly monochromatic line at 1363.422 nm is shown in red. Hanning apodization was applied to both interferograms prior to taking the FFT. The spectral resolution was estimated by determining the FWHM from a Gaussian fitting to the spectral line. The FWHM of the measured spectrum is 0.0516 nm and the FWHM of the theoretical line is 0.0451 nm. The two values differ by roughly 0.0065 nm and the difference is likely due to slight distortions in the spectrum that are introduced by the imaging optics.
The performance of the SHOW flight instrument was characterized in the
laboratory by uniformly illuminating the instrument with white light from an
8 inch aperture LabSphere integrating sphere. The 150 interferogram images were obtained and then processed using the level 0 to
level 1 processing chain to obtain calibrated interferograms and corrected
spectra. An example calibrated interferogram image is shown in Fig. 7a
and an example raw and corrected interferogram row is shown in Fig. 7b.
The corresponding spectral image formed by taking the FFT of each row of the
interferogram image is shown in Fig. 7c. All rows of the spectral image
are plotted in Fig. 7d. The SHOW filter is a narrow-band filter (see
Langille et al., 2018) with a 2 nm bandwidth and a peak transmission of 0.77
that is centred at 1364.52 nm at 23
Example corrected white light interferogram
The goal here is to examine the quality of the corrected interferograms and spectra in order to estimate the remaining variability in the images that is not associated with photon noise. It is difficult to characterize the noise in each individual interferogram and spectral row since samples within each row contains variations associated with the source. However, the vertical dimension of the interferogram and spectral image is expected to be smooth since the illumination within the field of view is uniform and the optical configuration is anamorphic. Therefore, we examine the inter-sample variability in each vertical column in order to characterize the noise.
Ideally, we want to operate the detector with the minimum amount of
uncertainty in the measurements. Taking the variance on any particular
interferogram sample to be given by Eq. (4), we estimate the noise floor by
averaging the images to reduce the photon noise. For an average signal level
of
On the other hand, the observed variations in the spectral image are slightly more difficult to quantify. We expect that each row in the image will record the same spectrum; however, this is not what is observed. Instead, we observe a strong modulation of the spectra in the vertical dimension. This variation is systematic and is due to the presence of a small relative tilt between the two gratings in the SHS, combined with aliasing of spectral information from the left side of Littrow into the right-hand side (see Sect. 5). This systematic variation is characterized in the next section by optimizing an instrument model that predicts the effect. Here we focus on the propagation of error from the interferogram samples to the spectra.
Since each row of the interferogram should produce the same spectrum, the
presence of the inter-sample variability in the spectra is determined at
each wavelength (column) by subtracting off a high-order polynomial (
The aliasing effect that is shown Fig. 7c and d is extremely problematic
for the retrieval of the vertical distribution of water vapour. The observed
variation modulates the true spectrum and the amplitude of the effect
depends on the relative position of the filter centre peak and the Littrow
wavelength, as well as the amount of absorption due to water vapour. In
order to implement the retrieval approach described in Sect. 8, a forward
model is constructed that captures these perturbations. The model takes a
simulated high-resolution spectrum
As a first step prior to performing the optimization, the Littrow wavelength was determined by aligning the observed water absorption features in the white light spectra with known features in the high-resolution simulated spectrum. In order to model the correct amount of absorption, we need to estimate the water vapour abundance, as well as the path length. For this calculation, we assumed standard temperature and pressure and a relative humidity of 50 %. The path length was then manually adjusted in the model until the depth of the absorption features were closely matched.
The instrument model was then optimized by iteratively adjusting the centre
peak of the filter, the value of the grating rotation
Instrument model optimization using laboratory data. Comparison
between the modelled and measured interferograms and spectral
images
Figure 8a shows the final agreement between the instrument model and the
SHOW white light laboratory measurements as a result of this process. The
normalized interferograms are shown in the top panel and the normalized
spectra are shown in the bottom panel. Here, the image has been rotated so
the lower rows correspond to the lower altitudes when compared to the flight
measurements. For this set of measurements, the Littrow wavelength was found
to be 1363.62 nm (in vacuum), the filter centre was found to be 1364.52 nm,
the grating tilt was found to be
On 18 July, SHOW flew an engineering flight on the ER-2 airplane from the Air Force Flight Research Center (AFRC) located near Palmdale, California in order to test several aspects related to the performance of the instrument. The flight track of the ER-2 airplane for the entire engineering flight is shown in Fig. 9a overlaid on top of a map showing the geographic location of the aircraft above California. The airplane attitude information during the flight is shown in Fig. 9b, where the recorded airplane pitch, roll, heading, height, latitude, and longitude are shown as a function of time. Portions of stable flight correspond to regions where the pitch and roll are relatively close to zero; although small variations in the pitch are always recorded – even for the most stable portions of the flight. In practice, this airplane attitude information is used to map the interferogram rows to tangent altitude at the limb for each frame obtained with the SHOW instrument.
ER-2 flight track where the red dot indicates the location of the
JPL Table Mountain facility
SHOW SHS and opto-box temperatures during the course of the flight
Prior to the flight, the instrument was mounted in the wing pod (detached
from the airplane) and was purged with nitrogen for over 24 h in order to
remove moisture and to achieve a stable thermal environment (23
We can see that the temperature inside the wing pod at the beginning of the
flight is relatively warm, corresponding to the warm morning temperatures on
the tarmac at takeoff. Once the airplane reached its cruising altitude, the
temperature dropped steadily and reached close to
The thermal control is designed so that the heaters on the top and bottom of
the optics box keep the top and bottom centre area of the SHS close to
23
Measured in situ water vapour above Table Mountain using the JPL Vaisala RS41 radiosonde measurements.
Following from Fig. 9a and b, the ER-2 airplane took off from AFRC at
roughly 15:02 UTC and flew northward where the pilot performed a set of
flight patterns designed to examine the sensitivity of the instrument to the
observed scattering angle. Afterwards, the airplane flew towards the
southeast and then turned to fly in the direction towards Table Mountain
(identified by the red dot in Fig. 9a). During this time, SHOW was
configured to continuously record images with integration times of 1800 ms
at 0.5 Hz sampling. At 17:59 UTC, a radiosonde (model Vaisala RS41) was
launched from the JPL facility located close to Table Mountain to measure
relative humidity, temperature, and pressure as a function of altitude. The
in situ water vapour abundance was deduced from these measurements using the
Hyland and Wexler formulation (Hyland and Wexler, 1983). Roughly 12 min
of coincident measurements (over 300 samples) of the same approximate column
of air were obtained. The in situ water vapour abundance that was measured
by the radiosonde is shown in Fig. 11. The abundance increases from 5 ppm at 20 km to roughly 35 ppm at 13 km. In this figure, the error bars only
show the measurement uncertainties. Systematic errors are not included. In
general, the uncertainty on the radiosonde measurements is between
In the remainder of this paper, we focus on the set of measurements
performed between 17:59 and 18:09 UTC (bracketed by the red vertical
bars in Fig. 9b). During this period, the airplane remained stable with
pitch variations of <
Each image obtained by SHOW during the Table Mountain portion of the flight is corrected using the level 0 to level 1 processing chain illustrated in Fig. 2. An example of the correction of a raw interferogram is shown in Fig. 12. The raw interferogram is shown in Fig. 12a and the fully corrected interferogram is shown in Fig. 12b. In these figures, the SHOW image has been rotated so that the bottom part of the image corresponds to lower tangent altitudes. Figure 12c shows the application of the correction for the interferogram row – 150. Here the DC bias has been removed by subtracting off the mean of each interferogram row in order to isolate the modulated component of the interferogram. These interferograms represent the SHOW level 1A data product.
Example conversion of raw to level 1A interferograms from raw images
In this particular measurement, the presence of clouds is seen in both the raw, as well as the fully corrected modulated component. Observe the observed horizontal variability in lower part of the non-modulated component shown in Fig. 12b. Ideally, the anamorphic optics averages over the horizontal variability in the scene radiance and the flat-field correction assumes a uniform illumination of the input aperture. Here we observe that the variability associated with the cloud appears to be slightly different on either side of the interferogram centre burst. While not completely understood, the effect at these altitudes can partially be explained as a combination of non-uniform aperture illumination (which results in a poor flat-field correction) and non-ideal pixel response due to saturation effects at the interferogram centre burst. As the pixels approach saturation, their response is no longer linear and the subtraction of the DC bias in the presence of this effect, as well as non-uniform illumination will result in such variability. In the case of the Table Mountain measurements, the cloud effects were confined to a small region at altitudes below the lower altitude cutoff of the retrieval at 13.5 km. Clouds also have a large optical depth, making it difficult to see the tangent point at the limb. Therefore, we do not expect to be able to perform retrievals on this particular measurement below 13.5 km.
In some cases, scattering from clouds saturates pixels at the interferogram centre burst, resulting in corrupted pixels throughout the image. The presence of the saturation effect associated with clouds is detected by observing a small number of pixels off to the edge of the detector, far away from the SHOW FOV, as well as the average signal across the FOV. This region also acts as a dark current and bias monitor during the flight. In practice, it was found that spikes in the number of saturated pixels correlate with spikes in both the FOV average and the off-image average. For the measurements presented in this paper, images that have strong cloud effects are identified and removed from the analysis.
The quality of the corrected interferogram has been characterized by
examining the observed variability in column cuts of the image using the
same approach that was used in Sect. 9.6. We understand that this approach
is only approximate, since the variance of the measurement varies as a
function of signal level. From Fig. 12a, we can see that the raw
interferogram signal varies from
We estimate the measured variability in the image by subtracting a high-order polynomial (
The retrieval algorithm described in Sect. 8 utilizes a forward model
which requires accurate information regarding the viewing geometry of the
instrument. One must map the observed interferogram rows to the line of
sight angles which correspond to specific tangent altitudes at the limb.
This mapping is set by the absolute pointing, the field of view of the
instrument and the number of rows in the image. On the one hand, the maximum
vertical resolution of the measurements is fixed by the instantaneous field
of view (
Example of the correction for the airplane pitch variation. The
interferogram signal at a single pixel plotted with the pitch variation as
a function of time is shown in
The ER-2 airplane pitch variability and its impact on the SHOW measurements during the engineering flight on 18 July 2017 is demonstrated in Fig. 13a where we have plotted the signal measured by a single pixel as a function of time along with the ER-2 pitch variation. It is very clear that a strong correlation exists between the temporal variability of the interferogram signal and the pitch variation. In this particular case, the interferogram signal is anti-correlated with the pitch variation, corresponding to the location of detector pixel [150, 205] (see Fig. 12a) in the SHOW FOV, where we have a lower atmospheric signal above and a higher atmospheric signal below the corresponding tangent altitude. It is also clear that the pitch variation is slow relative to the 0.5 Hz sampling cadence of the SHOW measurements.
The pitch variation is further demonstrated in Fig. 13b, where the mean signal level of each interferogram row is plotted as a function of altitude and then stacked as a function of time. For each row in each frame, the signal at each vertical bin corresponds to the DC component of the corresponding interferogram row. Therefore, the vertical variation of these samples in each frame provides a rough approximation of the shape of the integrated atmospheric radiance profile. In the ideal case, without pitch variation, we expect this profile to be relatively smooth, both temporally and spatially over this short 12 min window. For the SHOW ER-2 measurements, we see that the airplane pitch variation shows up as vertical bands that correspond to the detector rows scanning up and down the atmospheric radiance profile. We also observe the presence of several cloud features below row 100 with the dominant cloud feature sitting between row 80 and row 120 and persisting through the full observation period. The impact of the pitch variation is the most obvious at the location of the clouds since the gradient in the limb radiance is largest at the cloud feature.
Since the pitch variation is slow relative to the 0.5 Hz sampling cadence,
we correct for the pitch variation by using the airplane attitude
information to map each interferogram row to the corresponding tangent
altitude at the limb as shown in Fig. 13c. This removes a significant
portion of the variability; however, a residual temporal variation on the
order of 1 % of the mean signal level remains. The primary impact of the
presence of this variability is an uncertainty in the absolute pointing of
an individual frame. This degradation is quantified by examining the
vertical perturbations in the two dominant thin cloud features located at
mean altitudes of 13.6 and 13.1 km, respectively. The black line shows the
location of the peak signal associated with these features. The standard
deviation from the mean altitude level is found to be 124.45 m for the
feature at 13.6 km and 175.03 m for the feature at 13.1 km. Assuming a mean
ER-2 altitude of 20.85 km, these perturbations correspond to an angular
uncertainty in the pointing of 0.023 and 0.032
SHOW level 1B spectra are generated from the level 1A interferograms by applying Hanning apodization to each of the interferogram rows, performing an FFT and calculating the amplitude spectrum. Each row of the spectrum is mapped to a tangent altitude at the limb using the pitch correction described in the previous section. Therefore, a single record consists of the level 1B spectrum and a mapping of spectral row to the tangent altitude at the limb. An example level 1B spectrum obtained by applying this procedure to the interferogram image obtained at 17:59:05 UTC is shown in Fig. 14a. The wavelength registration is set using knowledge of the instrument configuration and by manually adjusting Littrow wavelength to line up the absorption features with a high-resolution forward modelled spectrum. For this particular measurement, the Littrow wavelength was found to be 1363.76 nm. Over the course of the 12 min Table Mountain portion of the flight, the Littrow wavelength was not found to vary significantly as a function of time.
Example level 1B spectral image
Figure 14b shows a column cut at a wavelength of 1364.61 nm. Several sources of variability are apparent in the spectral samples. First, note the natural variability associated with the limb-scattered radiance profile. The shape of this profile depends heavily on the atmospheric state (aerosol content, the water vapour abundance, and clouds, etc.). This particular wavelength sits in the wing of a strong absorption feature that is centred near 1364.66 nm. In this case, the radiance profile is modulated by the systematic variations in the spectrum introduced due to aliasing effects. This effect was anticipated earlier in the paper and is characterized in the next section. Here we focus on characterizing the signal-to-noise ratio of the corrected spectral samples.
The two dominant sources of noise in the spectral measurements are Poisson noise, as well as, a systematic inter-sample variability, both of which are introduced by the propagation of error into the spectrum from the interferogram samples (see Sect. 6). The systematic inter-sample variability in the spectral measurements is minimized by utilizing the temporal series of samples obtained while the airplane pitches up and down to obtain a secondary correction factor that is applied to the spectral measurements. In this case, the noise on individual spectral samples is primarily due to Poisson statistics and is estimated using Eq. (5).
Observed systematic inter-sample variability in the measured spectra
for a single wavelength (1364.81 nm) and altitude (13.122 km)
To characterize the systematic inter-sample variability, we assume that the
shape of the true atmospheric profile is smooth and constant. This is a
justified assumption given the stability of the observed integrated
intensity profile shown in Fig. 13c. Therefore, as the airplane pitches
up and down, multiple rows in the SHOW FOV observe the same atmospheric
signal. For example, Fig. 15a shows the raw non-normalized L1B spectral
signal level at a wavelength of 1364.81 nm for each detector row that
observed the tangent altitude at 13.122 km (shown in black) within the 12 min observation window. The black data points form clusters at each
detector row that are approximately randomly distributed. This distribution
includes Poisson noise, as well as, noise due to the temporal variability in
the signal level associated with natural variability and the absolute
uncertainty in the pitch correction of
Ideally, each row will register the same radiance; however, we observe that each cluster exhibits a slight offset for each detector row. In this case, the systematic inter-sample variability is of the same magnitude as the combined temporal variability and Poisson noise. The observed systematic biases are isolated in several steps. First, each of the sample clusters is averaged to minimize the random noise (shown in red) and clusters that have fewer than 10 samples are discarded from further analysis. We then normalize the averaged values by the mean signal level across all of the detector rows that observed this altitude in order to isolate the deviations relative to the mean. This data is then merged with the normalized averaged clusters from all the other tangent altitudes in order to isolate the inter-row biases for this particular wavelength. In order to remove the larger-scale variation associated with aliasing, we divide out a high-order polynomial from the resulting data. This procedure is repeated for all of the measured wavelengths to obtain a correction for each spectral column in the image. Therefore, we form a secondary multiplicative correction factor that can be applied to the level 1B spectral images to minimize the systematic inter-sample variability.
The results of applying this correction to the L1 B spectral images is demonstrated in Fig. 15b where we plot the same data as shown in Fig. 15a, this time with the correction applied to the spectral measurements. In this case, the amplitude of inter-row sample variability has been reduced by a factor of roughly 5 and this variability is now significantly smaller than the noise due to Poisson statistics. Assuming only Poisson noise, the average SNR of this set of samples is 53.83.
Correction of the L1B spectra. Variability before and after the
secondary correction is applied for an example row spectrum at a fixed
altitude
Each level 1B spectral image obtained during the Table Mountain portion of the flight was corrected using this approach in order to obtain level 1C spectra. An example of the impact of the secondary correction on an individual spectral row and an individual spectral column within a single image is shown in Fig. 16a and b, respectively. This correction minimizes the impact of the systematic inter-sample variability; therefore, the noise on the corrected spectral samples is taken to be Poisson distributed (Eq. 5) and this noise level is included in the retrieval algorithm in order to compute the associated measurement error of the retrieved vertical water vapour distribution.
The systematic perturbations to the radiance profile associated with aliasing are characterized by applying the optimization approach that was used in Sect. 9.7. However, for the Table Mountain flight measurements, we use in situ water vapour measurements as the input to the forward model. Knowledge of the in situ water vapour abundance provides a better representation of the true atmospheric state and allows for a better quality optimization to be performed. Forward modelled interferograms and spectra are generated and compared to the flight measurements and the instrument model is iteratively adjusted to identify the instrument configuration that best predicts the measurements.
Instrument model optimization using flight data. Comparison between
the modelled and measured interferograms and spectral images
The results from this optimization are shown in Fig. 17. The optimized
configuration corresponds to a Littrow wavelength of 1363.76 nm (in vacuum),
a grating tilt of
For the water vapour retrieval, we focus on a 10-image measurement window from 17:57:29 to 17:57:49 UTC and average the frames to increase the SNR of the measurements. During this time, SHOW has good coincidence with the radiosonde measurements and the aircraft is pitched slightly upward above the cloud features that are present at the beginning of the Table Mountain portion of the flight. The retrieval is performed using the approach discussed in Sect. 8. For the retrieval, we take the upper and lower altitude cutoff to be 18 and 13.5 km, respectively. The upper cutoff was chosen to be a few kilometres below the airplane altitude. Above this altitude, the region is optically thin and produces weak spectral signatures that reduce the accuracy of the retrieval. The lower cutoff was chosen to be above the region with clouds and above the region where the atmosphere is too optically thick to observe the tangent point.
Example SHOW water vapour retrieval
SHOW water vapour retrieval plotted with the radiosonde measurements
transformed to the coarse retrieval grid
The SHOW measurements utilize the full spatial resolution that is provided
by the vertical sampling within the imaged FOV. This provides an
instantaneous vertical resolution of 0.0176
An example retrieval that was obtained by averaging the 10 successive
coincident measurements is shown in Fig. 18a. Here, we perform the
retrieval on a 250 m sampling grid that is larger than the instantaneous
vertical resolution of the measurements. We include regularization; however,
very little smoothing is applied and this term has a negligible impact on
the vertical resolution. In addition, a small amount of damping is applied
to ensure convergence of the retrieval. The figure shows 15 iterations;
however, the retrieval converges in approximately 10 iterations. At each
iteration, we produce a high-resolution forward modelled radiance for each
of the 10 viewing geometries. Averaging is performed on the georeferenced
forward modelled images and the georeferenced measurement images using the
exact same procedure. The associated averaging kernel is shown in Fig. 18b. Very little smoothing has been applied; therefore, the approximate
vertical resolution of the measurements, determined from the FWHM of the averaging kernel, is closely matched to the 250 m spacing of
the retrieval grid. The measurement error due to Poisson noise is less than
The retrieved water vapour profile is plotted along with the measured in situ water vapour profile in Fig. 19a. Since the in situ profile is measured on a finer grid than the retrieval, the radiosonde data are transformed to the 250 m retrieval grid using the method that is suggested by Rodgers (2000). In effect, this appropriately smooths the radiosonde measurements down to the vertical resolution of the retrieval grid. The measurement uncertainties for both instruments are less than 1 ppm and are not shown in the figure. There is good agreement between the overall shape of the retrieved water vapour profile (shown in red) and the smoothed in situ water profile (shown in blue) with the best agreement occurring at the high altitudes. The ppm difference between the radiosonde and SHOW measurement is shown in Fig. 19b. At higher altitudes, the difference is on the order of 1–2 ppm and the two measurements match quite well. Such discrepancies are anticipated due to the sensitivities to aerosols that are described in our earlier sensitivity study (Langille et al., 2018). For example, differences in the knowledge of the true aerosol profile utilized in the forward model can result in systematic uncertainties on the order of 1 ppm. However, it is clear that the SHOW measurements do not capture some of the small-scale variability that is observed with the radiosonde below 16 km where the difference is between 2 and 5 ppm. This type of difference is not uncommon when comparing limb-sounding water vapour measurements with radiosondes (Stiller et al., 2012).
There are several potential reasons for differences between the Vasaila RS41 radiosonde and SHOW. The most likely is differences in the viewing geometry between the two instruments, coupled with the fact that both instruments have unknown accuracies. The radiosonde samples the spatial and temporal variability of the water vapour field as a function of altitude, whereas SHOW measures the 1-dimensional water vapour abundance, which is heavily weighted to the tangent point. It is known that radiosonde accuracies vary as a function of relative humidity and temperature and from sensor to sensor (Miloshevich et al., 2009). The Vaisala model RS41, utilized in this paper, has been shown to have improved accuracy over earlier models; however, errors between sensors on the order of 2 %–5 % have been recorded (Jensen et al., 2016). On the other hand, incomplete knowledge regarding the true instrument configuration (see Sect. 5) and the lack of knowledge of the true state of the atmosphere limits our ability to perform a more detailed characterization of the systematic effects associated with aliasing. In addition, uncertainties in the absolute pointing knowledge of the SHOW measurements leads to potential uncertainties in the forward modelled radiances that are difficult to characterize when coupled with the aliasing effect. Therefore, more work is required to fully examine the expected biases between these two types of measurement.
The SHOW prototype instrument has performed successful demonstration flights from NASA's ER-2 airplane. In this paper, we have presented the characterization and the level 0 to level 1 processing of flight measurements that were obtained with SHOW during an engineering flight that was performed on 18 July 2017. Extracting water vapour profiles from the SHOW measurements required a significant amount of calibration and characterization work and the development of an instrument model that is optimized to capture systematic variations that are observed in the measured spectra.
We have applied this approach to the SHOW measurements that were obtained
during a period of stable flight where the SHOW instrument observed the same
approximate column of air as a radiosonde that was launched from a JPL
facility nearby Table Mountain. These coincident measurements were compared
to the SHOW measurements and were found to agree to within 1–5 ppm from 13.5 to 18 km. The work presented in this paper provides initial validation of
the SHOW measurement technique and demonstrates that high vertical
resolution (< 500 m) measurements with <
The data presented in this paper is available upon request via the contact author.
The authors declare that they have no conflict of interest.
The authors would like to acknowledge the significant support provided to this project by the Canadian Space Agency. Edited by: John Worden Reviewed by: three anonymous referees