We present a new method for retrieving temperature from pure rotational Raman (PRR) lidar measurements. Our optimal estimation method (OEM) used in this study uses the full physics of PRR scattering and does not require any assumption of the form for a calibration function nor does it require fitting of calibration factors over a large range of temperatures. The only calibration required is the estimation of the ratio of the lidar constants of the two PRR channels (coupling constant) that can be evaluated at a single or multiple height bins using a simple analytic expression. The uncertainty budget of our OEM retrieval includes both statistical and systematic uncertainties, including the uncertainty in the determination of the coupling constant on the temperature. We show that the error due to calibration can be reduced significantly using our method, in particular in the upper troposphere when calibration is only possible over a limited temperature range. Some other advantages of our OEM over the traditional Raman lidar temperature retrieval algorithm include not requiring correction or gluing to the raw lidar measurements, providing a cutoff height for the temperature retrievals that specifies the height to which the retrieved profile is independent of the a priori temperature profile, and the retrieval's vertical resolution as a function of height. The new method is tested on PRR temperature measurements from the MeteoSwiss RAman Lidar for Meteorological Observations system in clear and cloudy sky conditions, compared to temperature calculated using the traditional PRR calibration formulas, and validated with coincident radiosonde temperature measurements in clear and cloudy conditions during both daytime and nighttime.

High time and space resolution measurements of atmospheric temperature are necessary to improve our understanding of many atmospheric processes, both dynamical and chemical. Radiosounding is the most widely used method for temperature profiling in the troposphere and lower stratosphere and has the advantage of operation in most weather conditions, but is typically limited to two flights per day only at selected sites worldwide. Pure rotational Raman (PRR) lidars have excellent vertical and temporal resolution and can be combined with vibrational Raman channels to determine relative humidity

The traditional Raman lidar temperature retrieval method, introduced by

Of primary importance is the calibration of the lidar returns to allow for absolute temperature measurements. In the traditional method, the ratio of the corrected photocounts is fit to a set of corresponding temperature data points usually obtained from radiosondes.

The accuracy of the traditional Raman temperature estimations is highly sensitive to the estimation of the calibration function and calibration coefficients.

The application of the optimal estimation method (OEM) for temperature retrievals using pure rotational Raman (PRR) lidar measurements has several advantages over the traditional method, including the ability to find temperature without assuming an analytic form of the temperature or count ratio relation. Our OEM retrieval does not use the ratio of the counts. Rather we use a forward model which includes complete physics to describe the raw count profiles. For calibration, the ratio of the lidar constants, here referred to as coupling constant, needs to be determined. The coupling constant can in principle be estimated at a single point, such as a nearby flux tower or surface measurement. Our OEM retrieval has other important benefits over the traditional method as it can directly retrieve ancillary parameters (in addition) to temperature, such as geometrical overlap, particle extinction, and the lidar constant. Our OEM is also capable of providing a full uncertainty budget, including both random and systematic uncertainties on a profile-by-profile basis, including the systematic uncertainty introduced in the retrieved temperature by the estimation of the coupling constant. The OEM is an inverse method and is a standard tool in the retrieval of geophysical parameters from passive atmospheric remote sensing instruments. Recent studies including

In Sect.

The development of a forward model for application of the OEM to PRR temperature retrieval is given in Sect.

PRR measurements from the RAman Lidar for Meteorological Observations (RALMO), located in Payerne, Switzerland (46

The returns of the Raman-shifted backscatter arising from rotational energy state transitions of nitrogen and oxygen molecules due to the excitation at the laser wavelength at 354.7 nm are detected in analog or photon-counting mode. The high-quantum-number channel (JH) of RALMO is assigned to the backscattered signals from the energy exchange that occurs in the high-quantum-number states for both the Stokes (355.77–356.37 nm) and anti-Stokes (353.07–353.67 nm) branches. The low-quantum-number channel (JL) is assigned to the signals from the energy exchange occurring in the low-quantum-number states in the Stokes (355.17–355.76 nm) and anti-Stokes (353.67–354.25 nm) branches.

The backscattered PRR signal is given by the Raman lidar equation:

The response of the photomultiplier tubes operated in the photon-counting mode can become nonlinear at high count rates. In the case of RALMO, the true and observed counts are related by the equation for non-paralyzed systems:

The OEM is an inverse method that uses the measurements

The measurements are related to the forward model by

The uncertainty budget is determined from the measurement and model parameter covariance matrices

The forward model describes the measurement as a function of both the state of the atmosphere and instrumental parameters. The core of our forward model is the lidar equation as presented in Sect.

The pressure,

The atmospheric transmission,

For each channel the subscript RR is replaced by JL and JH, the high- and low-quantum-number PRR channels. Then

RALMO detects multiple Stokes and anti-Stokes lines from both nitrogen and oxygen in a PRR spectrum. Therefore, to determine the attenuated backscatter cross-section in the forward model, we require knowledge of the exact number of quantum states detected by each RALMO PRR channel.
From the JH and JL channel characteristics we can calculate the range of frequency shifts for each channel relative to the elastic wavelength 354.7 nm. Then using the equations given by

The respective quantum lines from both nitrogen and oxygen in a PRR spectrum detected by the RALMO temperature polychromator. Note the given quantum lines are valid for both the Stokes and anti-Stokes branches.

In order to establish absolute calibration, we define the coupling constant

Using

The retrieval parameters (Table 2) in our OEM algorithm are temperature, background signals (including photomultiplier shot noise, sky background, and offset for analog channels), the lidar constants, dead times of the photon-counting systems, geometrical overlap, and particle extinction as a function of height. In OEM we can retrieve parameters on a height grid where the resolution can be the same or different than the vertical resolution of the height grid that the measurements obtained. If the retrieval grid is coarser than the measurement grid, we use linear interpolation on retrieved quantities when they are required in the forward model.

Values and associated uncertainties for the OEM retrieval and forward model parameters.

The OEM solver in the Qpack software package is used for our retrieval

The lidar measurements from the photon-counting channels follow Poisson counting statistics in the range where the counts are linear. Thus, the measurement variance

The U.S. Standard Atmosphere

Molecular and particle extinction terms occur in the atmospheric transmission term of Eq. (

The lidar ratio LR is chosen based on the

The effect of geometrical overlap and particle extinction on the signals is coupled and hence retrieving both parameters simultaneously is not possible unless at least one of the effects is highly constrained. The particle extinction is indefinitely measured using the backscatter ratio outside clouds, and overlap is well known above the height of full overlap, i.e. above 6 km

The a priori lidar constants for the JL analog and JL photon-counting channels are estimated by fitting the measurements generated using the sonde temperature and pressure used in the forward model to the PRR measurements. For analog measurements, the fitting range is between 1.5 to 2 km height. For photon-counting measurements with clear conditions or cloud/aerosol presence above 8 km, 6 to 8 km is used as the fitting range to ensure the photocounts are linear. If the photon-counting measurements contain cloud or aerosols in the geometrical overlap region, a fitting range below this is used, typically 3.5–4 km height. The fitting uncertainty for each analog and photon-counting lidar constants is used as the variance of the a priori lidar constant.

The a priori dead times for the two photon-counting systems are considered to be 3.8 ns, consistent with estimations from previous studies for RALMO and with values specified by the manufacturer

We present four different measurement situations which demonstrate the robust nature of our OEM temperature retrieval. Details of each case study are given in Table

Details of the four cases in clear and cloudy sky conditions we present to demonstrate the flexibility of our OEM temperature retrieval.

The traditional temperature profiles that will be shown later in this section are calculated using count profiles consisting of glued analog and photon-counting measurements, which are corrected for nonlinearity and background before processing. The vertical resolution of the traditional temperature profiles is 30 m at the lowest heights, increasing to 400 m by the upper heights to decrease the magnitude of the statistical uncertainty. A calibration function linear in

Figure

Figure

Count rate for 30 min of coadded RALMO measurements from 23:00 UT on 9 September 2011, a clear night;

Difference between the forward model and clear nighttime RALMO measurements on 9 September 2011 for the four signals (blue). The red curves show the standard deviation of the measurements noise.

The averaging kernels of temperature and the vertical resolution of the retrievals are shown in Fig.

Averaging kernels

Figure

The OEM provides a complete uncertainty budget in both random and systematic uncertainties (Fig.

Random uncertainties (red curve) and systematic uncertainties due to the forward model parameters for the temperature retrievals from the clear nighttime RALMO measurements on 9 September 2011. The systematic uncertainties included are Rayleigh-scatter cross section (purple dot-dashed curve),

Retrieved geometrical overlap function (red curve) from the clear nighttime RALMO measurements on 9 September 2011 compared to the a priori geometrical overlap function (green curve).

Figure

The retrieval setup for the second case study, which is a daytime measurement (Table

Count rate for 30 min of clear coadded RALMO measurements from 11:00 UT on 10 September 2011;

The residuals are unbiased and fall within the limits of the measurement standard deviation (Fig.

Difference between the forward model and the clear daytime RALMO measurements on 10 September 2011 for the four signals (blue). The red curves show the standard deviation of the measurements.

Similar to the clear nighttime case, the OEM-retrieved temperature agrees with the sonde temperature within the statistical uncertainty (Fig.

Same as Fig.

The analog coupling constant

The retrieved geometrical overlap function for the clear daytime case (not shown) agrees with the geometrical overlap retrieved for the nighttime case within 10 % statistical uncertainty.

Same as Fig.

Same as Fig.

The third case (details are given in Table

The retrieval setup is identical to the previous cases, as the cloud base is above the height of full geometrical overlap of the transmitter and receiver. The a priori profile of the particle extinction coefficient is derived from the backscatter ratio assuming a lidar ratio of 15 sr.

Count rate for 30 min of coadded RALMO measurements from 23:00 UT on 5 July 2011;

The residuals (Fig.

Difference between the forward model and the nighttime RALMO measurements on 5 July 2011 with the presence of a cirrus cloud for the four signals (blue). The red curves show the standard deviation of the measurements.

Same as Fig.

Same as Fig.

The retrieved geometrical overlap function from the measurement with the cirrus cloud (not shown) agrees within 10 % uncertainty with the geometrical overlap functions retrieved from the measurement with clear sky conditions, as the cloud is above the region of complete geometrical overlap. The red curve in the first plot in Fig.

The uncertainty budget for this case (not shown) is similar to the previous two cases shown; the cloud has little effect on the uncertainty values. As before, the statistical uncertainty makes the largest contribution to the full uncertainty. We have also calculated the statistical uncertainties for the estimated lidar ratio profile by standard uncertainty propagation of the OEM particle extinction and backscatter coefficient statistical uncertainties.

A cloud at about 4 km is present in measurements used for the last case study (Table

Count rate for 30 min of coadded RALMO measurements from 23:00 UT on 21 June 2011, which has a cloud base at an height about 4 km;

Difference between the forward model and the nighttime RALMO measurements on 21 June 2011 with the presence of a lower level cloud for the four signals (blue). The red curves show the standard deviation of the measurements.

Figure

We use these measurements obtained during cloudy conditions as input to our OEM and obtain unbiased residuals which fall within the standard deviation of the measurements, meaning the forward model accurately retrieve temperatures in the presence of cloud.

The response function (Fig.

Same as Fig.

Same as Fig.

Same as Fig.

Below 5.25 km the retrieved particle extinction coefficient agrees well with the a priori values and the corresponding lidar ratio is between 15 and 20 sr indicating a liquid cloud. Above 5.25 km the retrieved particle extinction is smaller than the first guess yielding again a lidar ratio around 5 sr (Fig.

The four retrievals discussed in the previous section demonstrate that the OEM provides robust and accurate retrievals of temperature, geometrical overlap, and particle extinction coefficients during clear conditions in both daytime and nighttime and in high-cloud and low-cloud nighttime conditions. Unlike the traditional Raman lidar temperature analysis method

The calibration function plays a key role in the traditional temperature retrieval algorithm from the PRR backscattered signals, in particular if calibration is not done over the entire observed temperature range. Typically, a calibration function linear in

The only calibration required in our OEM scheme is the determination of the two coupling constants,

The OEM temperature retrievals of four very different sky conditions have been compared against coincident radiosonde temperature measurements. The case presented is the US Standard Model normalized to the surface temperature from the coincident sonde temperature. We successfully used other a priori temperature profiles, such as the smoothed sonde temperature measurements and temperature from the Mass Spectrometer and Incoherent Scatter (MSIS) radar model, to retrieve temperature using our OEM algorithm. All the retrieved temperature profiles using each a priori profile for heights where the response function is 0.9 or greater are identical within the statistical uncertainty.

In our study, we have successfully retrieved a geometrical overlap function for the RALMO system using the PRR measurements simultaneously with the temperature retrieval. Ray-tracking studies have concluded that the RALMO system reaches its full geometrical overlap by 5.0–5.5 km in height. These calculations are consistent with our geometrical overlap retrievals in both clear daytime and nighttime conditions. Measuring the geometrical overlap function and its uncertainty allows a more accurate estimation of the particle extinction coefficient when clouds or aerosols are present. The particle extinction profiles we retrieved in the two cloudy condition cases are consistent with measurements collected by

The particle extinction coefficient is retrieved in the full geometrical overlap region, i.e. above 6 km or above the cloud base. The extinction values and lidar ratios we obtained for high and mid-level clouds are in agreement with other publications. The two case studies featuring clouds suggest that both clouds consisted of a liquid and a ice part with lidar ratios at 18 and 5 sr.

For all the case studies we presented, the lidar constants for the lower quantum channels (analog and photon counting), the dead times for each photon-counting channel (JL and JH), and background for all four signals are also retrieved. The retrieved lidar constants for each channel agreed within 20 % uncertainty for all four cases. The retrieved dead times are about 3.8 ns and consistent with the dead times specified by the manufacturer and other independent estimates.

The uncertainty budget provided by the OEM contains both random and systematic uncertainties. Estimation of the uncertainty budget requires assignment of appropriate covariances to the model parameters. Using the standard deviations given in Table 2, the uncertainty budgets for all the case studies are estimated. The largest contribution towards the temperature uncertainty originates from the statistical uncertainty due to the measurement noise. Overall contribution from the coupling constants to the temperature uncertainty are less than 0.2 K for all heights. Given the fact that the measurement noise can be reduced with longer integration times, this result suggests that by the OEM method very precise temperature measurements are possible even if calibration is only possible over a small temperature range.

The systematic uncertainties of pressure and air density are on the order of 0.1 K and 0.1 mK, respectively. Understanding the full uncertainty budget of temperature is of particular importance for trend analysis and process studies. The observational basis for supersaturation studies in the upper troposphere is still unsatisfactory and the OEM framework allows us to combine different data sources to provide a high-quality data set including profile-by-profile uncertainty budgets.

We have demonstrated the ability of the OEM to retrieve multiple geophysical and instrumental parameters from PRR lidar measurements. The first-principle forward model adequately represents the raw PRR measurement and allows us to retrieve temperature, geometrical overlap, particle extinction, lidar constants, background counts, and dead time using multiple analog and photon-counting channels. We found the following results from our OEM temperature retrievals from PRR measurements:

The forward model presented, based on the lidar equation, contains the essential physics to reproduce the analog and photon-counting measurements, leading to unbiased residuals and robust estimates of temperature.

Our OEM retrieval does not require a calibration function as used in the traditional temperature retrieval method. It only requires determination of the two coupling constants,

The OEM provides a cutoff height for the temperature retrievals that specify up to which height the retrieved profile is primarily due to the measurements and not the a priori temperature profile.

Vertical resolution is determined at each height and is automatically adapted in the retrieval in response to increasing measurement noise with height.

The OEM provides a complete uncertainty budget, including random and systematic uncertainties due to model parameters, including the assumed pressure, air density, and the coupling constants.

Simultaneous analog, which are linear, and photon-counting measurements allow the dead time to be retrieved.

The OEM-retrieved geometrical overlap function for the RALMO using the measurements in clear conditions is determined and shown to be consistent with, but not the same as, that calculated by

The OEM is a computationally fast and practical method for routine temperature retrievals from lidar measurements as required for operational lidar systems.

We have demonstrated that the OEM allows for retrieval of temperature from pure rotational Raman lidar measurements that are consistent with the coincident sonde temperature. We discussed the advantages of the OEM over the traditional temperature retrieval algorithm. We can use the OEM-retrieved temperature to study temperature trends with the benefit of a full uncertainty budget provided by our OEM. Our OEM temperature retrieval can also be used for routine measurements in a wide variety of observing conditions, and is applicable to any similar PRR lidar system.

We are in the process of implementing the OEM for routine temperature measurements from the RALMO system. We are also combining the OEM PRR temperature retrieval with the OEM water vapour mixing ratio retrieval of

Measurements used in this paper may be requested from MeteoSwiss by contacting Alexander Haefele (alexander.haefele@meteoswiss.ch).

SMG was responsible for coding the Raman temperature OEM routine as well as the comparisons between the OEM and radiosonde measurements. She also wrote the initial draft of the paper. RJS and AH were SMG's PhD supervisors. They defined the project and participated in the development of the OEM code. AH and GM provided lidar and radiosonde measurements and performed the traditional method to estimate the temperature from the measurements.

The authors declare that they have no conflict of interest.

We thank Ghazal Farhani for her helpful comments and suggestions that were extremely useful to us. We also thank the Western University Writing Support Centre and Patricia Sica for their assistance in editing and proofreading this paper. This project has been funded in part by the National Science and Engineering Research Council of Canada and by the Canadian Space Agency under the Arctic Validation and Training for Atmospheric Research in Science (AVATARS) program.

This project has been funded in part by the National Science and Engineering Research Council of Canada through a Discovery Grant (Robert J. Sica) and a CREATE award for a Training Program in Arctic Atmospheric Science (Kimberly Strong, U Toronto, PI) and by MeteoSwiss (Switzerland).

This paper was edited by Gerd Baumgarten and reviewed by Christoph Ritter and one anonymous referee.