the Creative Commons Attribution 4.0 License.
Special issue: Tropospheric profiling (ISTP11) (AMT/ACP interjournal SI)
Research article 29 May 2020
Research article  29 May 2020
Methodology for deriving the telescope focus function and its uncertainty for a heterodyne pulsed Doppler lidar
 ^{1}Institute for Atmospheric and Earth System Research/Physics, Faculty of Science, University of Helsinki, Helsinki, Finland
 ^{2}Finnish Meteorological Institute, Helsinki, Finland
 ^{3}Department of Meteorology, University of Reading, Reading, UK
 ^{4}Andalusian Institute for Earth System Research (IISTACEAMA), 18006 Granada, Spain
 ^{5}Department of Applied Physics, University of Granada, 18071 Granada, Spain
 ^{1}Institute for Atmospheric and Earth System Research/Physics, Faculty of Science, University of Helsinki, Helsinki, Finland
 ^{2}Finnish Meteorological Institute, Helsinki, Finland
 ^{3}Department of Meteorology, University of Reading, Reading, UK
 ^{4}Andalusian Institute for Earth System Research (IISTACEAMA), 18006 Granada, Spain
 ^{5}Department of Applied Physics, University of Granada, 18071 Granada, Spain
Correspondence: Pyry Pentikäinen (pyry.pentikainen@helsinki.fi)
Hide author detailsCorrespondence: Pyry Pentikäinen (pyry.pentikainen@helsinki.fi)
Doppler lidars provide two measured parameters, radial velocity and signaltonoise ratio, from which winds and turbulent properties are routinely derived. Attenuated backscatter, which gives quantitative information on aerosols, clouds, and precipitation in the atmosphere, can be used in conjunction with the winds and turbulent properties to create a sophisticated classification of the state of the atmospheric boundary layer. Calculating attenuated backscatter from the signaltonoise ratio requires accurate knowledge of the telescope focus function, which is usually unavailable. Inaccurate assumptions of the telescope focus function can significantly deform attenuated backscatter profiles, even if the instrument is focused at infinity. Here, we present a methodology for deriving the telescope focus function using a colocated ceilometer for pulsed heterodyne Doppler lidars. The method was tested with Halo Photonics StreamLine and StreamLine XR Doppler lidars but should also be applicable to other pulsed heterodyne Doppler lidar systems. The method derives two parameters of the telescope focus function, the effective beam diameter and the effective focal length of the telescope. Additionally, the method provides uncertainty estimates for the retrieved attenuated backscatter profile arising from uncertainties in deriving the telescope function, together with standard measurement uncertainties from the signaltonoise ratio. The method is best suited for locations where the absolute difference in aerosol extinction at the ceilometer and Doppler lidar wavelengths is small.
Coherent Doppler lidar systems are capable of providing accurate radial Doppler velocities at high temporal and spatial resolution and have been employed across a wide range of scientific and operational fields. Meteorological applications include the retrieval of turbulent properties to determine the strength, location, and source of mixing in the atmospheric boundary layer and, with many systems having scanning capability, the retrieval of winds. Information on the targets responsible for the radial Doppler velocities measured by the Doppler lidar (e.g. aerosol, cloud, precipitation) would greatly aid the interpretation of both the velocities and the products derived from them, but this requires quantitative use of the signal power received by the instrument.
The performance of a Doppler lidar depends on the signaltonoise ratio, SNR, of the system, as SNR determines the radial velocity uncertainty (Rye and Hardesty, 1993; Pearson et al., 2009). The outgoing laser beam can be focused to improve the SNR at ranges close to the focal length (Pearson et al., 2002), and this is often used to improve the Doppler lidar velocity data quality and data availability, particularly in the atmospheric boundary layer. The optimal choice of focus will depend on the atmospheric conditions at the deployment location (Hirsikko et al., 2014).
Knowledge of how the choice of instrument parameters, such as the effective focal length of the telescope, impacts the SNR profile is necessary in order to obtain profiles of the attenuated backscatter coefficient (Zhao et al., 1990). A comprehensive overview of the theoretical considerations in determining the performance of coherent Doppler lidar systems was given by Frehlich and Kavaya (1991), who provided analytical expressions for deriving the expected signal measured by the coherent detector for a given target for a range of instrument configurations, including analytical expressions for the telescope focus function (also termed coherent responsivity). Most analytical expressions assume ideal Gaussian beams, which may not always be appropriate (Hill, 2018); hence experimental approaches have also been used to determine the impact of beam aberrations (Hu et al., 2013).
The profile of the attenuated backscatter coefficient has the potential to be used in real time by weather forecasters (Illingworth et al., 2019), as it can be used in the same manner as for ceilometers. This includes the detection of liquid, supercooledliquid, mixedphase, and ice clouds (Hogan et al., 2003; Van Tricht et al., 2014; Tonttila et al., 2015), aerosol layer and mixingheight determination (Flentje et al., 2010; Kotthaus and Grimmond, 2018), and the retrieval of precipitation parameters (Lolli et al., 2018).
In addition to providing velocity estimates for wind and turbulence, the inclusion of the profile of the attenuated backscatter coefficient is advantageous for Doppler lidar boundary layer classification schemes (Tucker et al., 2009; Harvey et al., 2013; Manninen et al., 2018) by enhancing the discrimination between aerosol, cloud, and precipitation, and it can be used for tracking elevated aerosol plumes (Hannon et al., 1999). The combination of attenuated backscatter profiles from coherent Doppler lidars with other profiling instruments permits additional retrievals; for example, together with a ceilometer (Westbrook et al., 2010b), or with a cloud radar (Träumner et al., 2010), it can yield drizzle drop size and precipitation rate. There is also an advantage to obtaining attenuated backscatter and Doppler velocity measurements in the same measurement volume, since this will simplify the calculation of cloud or aerosol mass fluxes (Engelmann et al., 2008).
Therefore, an accurate profile of the attenuated backscatter coefficient requires confidence in the parameters used to generate the telescope focus function. The parameters may not be known a priori or may differ from what is assumed, and incorrect values can result in artefacts and very large biases in the attenuated backscatter coefficient. We present a methodology for deriving the parameters of the telescope focus function experimentally from colocated Doppler lidar and ceilometer observations, together with the uncertainties in the function parameters. The ceilometer, for which the overlap function is known, provides our reference attenuated backscatter profiles. This methodology is relevant for coherent Doppler lidars designed for meteorological applications with maximum ranges suitable for observing the full extent of the boundary layer and beyond. Note that a calibration constant may still need to be determined and applied after implementing the calculated telescope focus function to retrieve the profile of the attenuated backscatter coefficient (Westbrook et al., 2010a; Chouza et al., 2015).
The theoretical description of the telescope focus function is outlined in Sect. 2. In Sect. 3, we introduce the instruments and the methodology for deriving the parameters of the telescope focus function experimentally. An iterative leastsquares regression using weighted mean square error (MSE) is used to find the best solution for the telescope focus function, where the weights represent the measurement uncertainties in both instruments. The use of long time periods (1 year or more) also provides an estimate of the uncertainties in the parameters for the telescope focus function, which can then be propagated through to uncertainties in the retrieved attenuated backscatter coefficients. The methodology is applied to different instruments in multiple locations in Sect. 4, and the validation of the method is presented in Sect. 5.
2.1 Telescope focus function
Following Frehlich and Kavaya (1991), the coherent Doppler lidar equation can be expressed as
where SNR is the signaltonoise ratio, varying as a function of range, R, from the instrument; β^{′} is the attenuated backscatter coefficient; η is the detector quantum efficiency; c is the speed of light; E is the beam energy; h is Planck's constant; ν is the optical frequency; B is the receiver bandwidth; and A_{e} is the effective receiver area.
For a monostatic system emitting a circular Gaussian beam, using a circular aperture, and having matched filters, the effective receiver area is given by Frehlich and Kavaya (1991) and Henderson et al. (2005):
where D is the 1∕e^{2} effective diameter of a Gaussian beam, λ is the laser wavelength, f is the effective focal length of the telescope for the transmitter and receiver, and ρ_{0} is a turbulent parameter, also termed transverse field coherence length.
Collecting the rangedependent terms, we obtain a unitless telescope focus function:
which is also termed the coherent responsivity (Frehlich and Kavaya, 1991).
The profile of the attenuated backscatter coefficient is then obtained by rearranging Eq. (1):
Figure 1a shows how T_{f}(R) depends on the telescope focal length, f, and Fig. 1b how T_{f}(R) depends on D. Both figures show that the apparent focus – i.e range to the T_{f}(R) maximum – is always closer than f and that decreasing D shortens the apparent focus. This makes estimation of the parameters by eye in T_{f}(R) prone to errors, since the apparent focus cannot be translated into f without knowledge of D.
Figure 1c shows that even if the telescope is focused at infinity, knowledge of the D is essential to derive attenuated backscatter coefficient profiles. While the gradient of T_{f}(R) may be independent of D at the near and far ranges, the relative magnitude is not, and the potential variation is high in the range of the profile that is commonly of most interest.
2.2 Uncertainty in the attenuated backscatter coefficient
Assuming that the parameters T_{f}(R) and SNR are independent, and have uncertainties that can be described as Gaussian, the relative random uncertainty in the attenuated backscatter coefficient is
where σ_{S} is the relative uncertainty in the Doppler lidar SNR, and ${\mathit{\sigma}}_{{T}_{\mathrm{f}}}$ is the relative uncertainty in T_{f}(R). An expression for deriving σ_{S} is given by Manninen et al. (2018), and we describe our method for obtaining ${\mathit{\sigma}}_{{T}_{\mathrm{f}}}$ in Sect. 4.2.
There are three rangedependent unknowns in Eq. (2): f, D, and ρ_{0}. We first assume that we can neglect ρ_{0} and then describe a method for estimating f and D, together with their uncertainties, which can then be propagated to obtain the uncertainty in the attenuated backscatter coefficient. The impact of the ρ_{0} parameter is discussed in Sect. 4.3.
3.1 Instruments
We used measurements taken from the U.S. Department of Energy Atmospheric Radiation Measurement (ARM, Mather et al., 2016) observatories. We selected five sites with colocated ceilometer and Doppler lidar instruments: Southern Great Plains, US (SGP); tropical west Pacific, Darwin, Australia (Darwin); Barrow, Alaska, US (North Slope of Alaska, NSA); Graciosa, Azores (Graciosa); and Ascension Island, Atlantic, UK (Ascension).
The Doppler lidars operated by ARM comprise both Halo Photonics StreamLine and StreamLine XR versions. These are commercially available heterodyne pulsed systems capable of full hemispheric scanning and operated at a temporal resolution of 1–2 s (see Table 1). The focus for the StreamLine version can be set by the operator, whereas the StreamLine XR has the focus set by the manufacturer; however ARM has had some instruments upgraded from their original specification.
The ceilometer at all sites was a Vaisala CL31 ceilometer, which has a coaxial design and full overlap before 100 m and a temporal resolution of 30 s (more specifications given in Table 2).
3.2 Methodology
3.2.1 Telescope focus function parameter estimation
The methodology for deriving the parameters of the telescope focus function compares profiles from a colocated Doppler lidar and ceilometer using an iterative leastsquares regression to find the best solution. The method follows the process diagram given in Fig. 2.
Before input, the Doppler lidar SNR data had a background correction applied to reduce bias (Manninen et al., 2016). Both ceilometer and Doppler lidar data were averaged to a common 30 min, 30 m vertical resolution grid. If the Doppler lidar vertical resolution was larger than 30 m (as was the case for one period from Darwin), linear interpolation was used to match resolutions. After averaging, data below a minimum threshold of −22.2 dB (Manninen et al., 2018) were discarded. The threshold is based on the expected noise floor for the instruments considered here (Halo StreamLine and StreamLine XR) and should probably be modified for different instruments.
The data were then filtered to select only those portions of the profiles that are considered reliable for comparison. Ceilometer data below 195 m were discarded to ensure that only data with full overlap were used.
Due to the wavelength difference between the Doppler lidar and the ceilometer, it cannot be assumed that the atmospheric backscattering properties are the same at both wavelengths. However, we are only interested in the profile shape, not the absolute values, so profiles from the Doppler lidar and the ceilometer can be compared as long as they contain only one type of scatterer, and one which can be assumed to be distributed homogeneously throughout the portion of the profile used for comparison. Hence, the portion of a profile selected for comparison should contain only one aerosol layer, no clouds, and no precipitating hydrometeors.
We removed clouds by identifying the range gate 150 m below the cloud base detected by the ceilometer and excluding all data beyond this. Elevated aerosol layers and precipitating hydrometeors were filtered out by identifying layers using a convolution of the ceilometer profile with a haarwavelet to detect changes in the gradient. The base of the second layer was identified where the gradient was increasing over two range gates, and all data above this were discarded. This process also eliminates noisy profiles with low SNR.
The T_{f}(R) parameter estimation is performed on a profilebyprofile basis for each profile where the filtering process leaves eight successive range gates present. From Eq. (4), dividing the Doppler lidar SNR profile with the appropriate T_{f}(R) will generate a Doppler lidar attenuated backscatter profile whose shape should match that of the ceilometer attenuated backscatter profile.
We use a bruteforce approach to iterate through a range of reasonable f and D values, generating a corresponding T_{f}(R) and Doppler lidar attenuated backscatter profile for each combination of values. The ceilometer profile and resulting Doppler lidar attenuated backscatter profiles are normalised so that the integral value of the unfiltered portion is unity. We then use a leastsquares regression using weighted MSE to find the best solution (smallest MSE), where the weights represent the measurement uncertainties in both instruments.
Collecting results over many profiles results in a bivariate distribution; the peak of this distribution is chosen as the best estimate of f and D, and hence the best estimate of T_{f}(R), using Eq. (3).
3.2.2 Outlier removal
Occasionally, data of poor quality pass the filtering step in Fig. 2. The most common issues are noisy ceilometer data and a bias in the Doppler lidar SNR profiles. If not screened, these occasional profiles result in significantly altered T_{f}(R) estimation. Any noise in the ceilometer data is magnified by the profile length often being relatively short, and hence large uncertainty in even a single range gate can skew the regression. Doppler lidar SNR bias will impact the normalisation process, changing the T_{f}(R) selected by the method due to the now incorrect profile shape. Due to the nonlinearity of the T_{f}(R) parameter estimation process, these issues result in regression solutions wildly inconsistent with the estimates based on good data. These outliers, which do not fall within the normal uncertainty observed in good data, are then removed from the bivariate distribution of solutions before calculating the uncertainty estimates.
We used the median absolute deviation, MAD (Huber and Ronchetti, 2009; Leys et al., 2013), to distinguish outliers in the bivariate distribution of estimated f and D. MAD can be calculated using
where b=1.4826 when the distribution excluding the outliers is normal. However, the distribution of f and D may not meet this criterion due to the nonlinearity of T_{f}(R) and the computational T_{f}(R) estimation process. We expect the distributions of D and f^{−2} to be close to normal and will use f^{−2} rather than f to determine outliers. Additionally, the peak of the bivariate distribution may not always coincide with the medians of the univariate D and f^{−2} distributions, and, hence, we use a modified form of Eq. (6),
We selected three MADs as the threshold for flagging outliers:
Assuming the distribution excluding the outliers to be normal, three MADs correspond to 3 standard deviations of the underlying distribution. In cases where f is at infinity, all estimates with a finite f will be flagged as outliers.
4.1 Parameter estimation
We applied the T_{f}(R) estimation method to Doppler lidars at five ARM atmospheric observatories. Figure 3a shows the distribution of f and D calculated for the Doppler lidar operating at Darwin in northern Australia between 21 June 2011 and 22 July 2012. This Doppler lidar is a StreamLine and the distribution of f is positively skewed, as explained in Sect. 3.2.2. The distribution displays a slightly wider peak than expected for a normal distribution.
Figure 3b shows the distribution of f and D for the StreamLine Doppler lidar operating at SGP from 1 January 2015 to 2 May 2016. The distribution close to the peak is really tight, while the outliers have substantial spread. Many of the poor estimates responsible for the outlier spread occur during January and February in both years, while for the rest of the period the estimates are remarkably consistent. On 3 May 2016 the Doppler lidar at SGP was changed to a StreamLine XR, and Fig. 3c shows the distribution of f and D from 3 May 2016 to 5 June 2017. The change in instrument version, from StreamLine to StreamLine XR, is clearly seen in the change in D, whereas the best estimate for f did not change. However, inspecting the data by eye would suggest a significantly shorter apparent focus, and the T_{f}(R) calculated using the best estimates for f and D also exhibits a significantly shorter apparent focus. Consequently, the StreamLine XR in SGP has been noted to suffer from poor SNR at the boundary layer top.
The bivariate distributions of f and D show notable variations in how tight they are around the peak, likely a result of differences in data quality between the instruments. The best estimates of f and D and their uncertainties for all sites are presented in Table 3. The Doppler lidar measurements at Darwin were split into two periods, as there was a 2month break in the measurements between these two periods. We performed the T_{f}(R) parameter estimation separately for both periods. The best estimates from these periods differ from each other, which is expected as some adjustments were made to the instrument. The telescope focal length for this instrument is directly adjustable by any operator, while the beam diameter is set by the manufacturer and is not modifiable by the operator. We note that the D estimates are the same for these two periods within the margin of error calculated.
For the sites and instruments selected here, only the Doppler lidar at NSA had f set to infinity. In fact, all StreamLine lidars have D in the vicinity of 25 mm, whereas D for the StreamLine XR versions is about half this. Nevertheless, the variation between instruments of the same version is not negligible and should be taken into account when calculating T_{f}(R) and then attenuated backscatter.
The final step to obtain attenuated backscatter profiles is to apply a calibration constant, which can be achieved using the liquid cloud calibration method (Westbrook et al., 2010a; O'Connor et al., 2004).
The parameters f and D calculated for period 1 in Darwin have been used to derive T_{f}(R) and the results applied in Fig. 4. This shows the utility of the method, able to provide reliable Doppler lidar attenuated backscatter profiles in Fig. 4b that show no overcorrection below 1 km and display similar incloud values to the ceilometer in Fig. 4c. It is expected that the aerosol attenuated backscatter coefficients will differ due to the different scattering properties of aerosol at the different wavelengths; the scattering properties of cloud droplets remain similar at the two wavelengths (O'Connor et al., 2004; Westbrook et al., 2010a).
4.2 Uncertainty
A computational method was used to calculate the uncertainty in the estimated T_{f}(R) as it is a nonlinear function of f and D. We used Monte Carlo simulation (MCS) (Morgan and Henrion, 1990), where a distribution of input values is fed into a model, here the effective receiver area Eq. (2), and the uncertainty is obtained from the distribution of the output. The input values can be created either from observed statistics or by bootstrapping, i.e. resampling the data. We created three different sets of input values for our MCS:

Resampling the individual estimates of f and D provided directly by the T_{f}(R) estimation method (i.e. those displayed in Fig. 3) after excluding outliers.

Generating the values from the statistics presented in Table 3, assuming that D and f^{−2} are normally distributed and independent, $N({f}^{\mathrm{2}},D)$.

Generating the values from statistics presented in Table 3, assuming D and f to be normally distributed and independent, N(f,D).
For each set of input values, the relative uncertainty in T_{f}(R) is calculated as
where ${\mathit{\sigma}}_{{T}_{\mathit{\varphi}}}\left(R\right)$ is expressed in terms of the meansquared deviation of the MCSsimulated telescope focus function, T_{ϕ}(R), from the best estimate of T_{f}(R),
to avoid underestimating the uncertainty resulting from the asymmetry in T_{ϕ}(R). This also allows us to estimate the impact of refractive turbulence on the uncertainty estimate.
Examples of the three input parameter distributions are presented in Fig. 5. Resampling (Fig. 5a) is the most accurate method as it does not require assumptions about the parameter distributions and their independence. We recommend resampling as the primary method for uncertainty calculation. Using the $N({f}^{\mathrm{2}},D)$ distribution (Fig. 5b) produces a set of input values that appear to be a reasonable approximation, except that the distribution is not as tight around the peak. Using the N(f,D) distribution (Fig. 5c) produces a set of input values that tend to overemphasise shorter values of f and underemphasise higher values. We also note that the central bin of the resampled distribution contains 50 % more samples than the central bin of the statistically generated distributions do. We presume that this is a consequence of the variation in SNR not being necessarily normally distributed.
Figure 6a displays ${\mathit{\sigma}}_{{T}_{\mathrm{f}}}\left(R\right)$ for Darwin showing the range dependence of the uncertainty, with much larger uncertainties for ranges close to either side of the focus (f=590 m). The profile of uncertainties obtained with each set of MCS input values exhibit a similar shape, with $N({f}^{\mathrm{2}},D)$ being closer to resampling than N(f,D) in the near field.
Figure 6b displays ${\mathit{\sigma}}_{{T}_{\mathrm{f}}}\left(R\right)$ for the Doppler lidar at NSA, which has f set to infinity; therefore ${\mathit{\sigma}}_{{T}_{\mathrm{f}}}\left(R\right)$ is only dependent on the uncertainty in D. Note the reduced uncertainties around 200–400 m, which are expected when examining Fig. 1c.
The largest value of ${\mathit{\sigma}}_{{T}_{\mathrm{f}}}\left(R\right)$ provides the uncertainty envelope value for each site, which is presented in Table 4. Resampling provides values ranging from 0.12 for the updated instrument at SGP to 0.32 at NSA. MCS values created using $N({f}^{\mathrm{2}},D)$ provided similar values, whereas MCS using N(f,D) often provided much larger uncertainties.
4.3 Impact of refractive turbulence
So far we have neglected the potential impact of turbulence on T_{f}(R) arising from the refractive turbulent parameter, ρ_{0}, in Eq. (2). An expression for ρ_{0} is given in Frehlich and Kavaya (1991),
where H=2.914383, $k=\mathrm{2}\mathit{\pi}/\mathit{\lambda}$, and ${{C}_{n}}^{\mathrm{2}}\left(z\right)$ is the refractive turbulence at range z. We chose three profiles with constant ${{C}_{n}}^{\mathrm{2}}\left(z\right)$ and a realistic vertical profile based on the most turbulent case presented by Roadcap and Tracy (2009). Figure 7 shows the impact that these different profiles have on T_{f}(R) and the resulting resampling calculation of ${\mathit{\sigma}}_{{T}_{\mathrm{f}}}\left(R\right)$ for two Doppler lidar instruments with different T_{f}(R). Values of ${{C}_{n}}^{\mathrm{2}}$ up to ${\mathrm{10}}^{\mathrm{14}}\phantom{\rule{0.125em}{0ex}}{\mathrm{m}}^{\mathrm{2}/\mathrm{3}}$ have negligible impact on T_{f}(R), and even the realistic profile only showed a slight increase in ${\mathit{\sigma}}_{{T}_{\mathrm{f}}}\left(R\right)$ for the instrument with a focus set closer than infinity. This suggests that the impact of turbulence can be safely neglected for low values of ${{C}_{n}}^{\mathrm{2}}$, and for most applications, it can also be neglected when operating in the vertical. Hence, turbulence has no significant impact on the methodology described here for deriving the parameters f and D and their uncertainties from vertical profiles but can be included for completeness.
However, it is clear that the turbulent impact should not be ignored when measuring at low elevation angles close to the horizon, where a profile similar to ${{C}_{n}}^{\mathrm{2}}={\mathrm{10}}^{\mathrm{13}}\phantom{\rule{0.125em}{0ex}}{\mathrm{m}}^{\mathrm{2}/\mathrm{3}}$ may be possible. Figure 7 shows that such a profile has a major impact on T_{f}(R), especially in the far range. In these cases, the parameters f and D obtained from vertical measurements are still applicable, but ρ_{0}(R) must also be calculated or estimated in order to derive the profile of attenuated backscatter, β^{′}(R).
The liquid cloud calibration method (O'Connor et al., 2004; Westbrook et al., 2010a) is used to determine a calibration constant by integrating attenuated backscatter profiles containing fully attenuating liquid clouds, which have a wellconstrained apparent lidar ratio, ηS, where η is a multiple scattering factor and S is the lidar ratio. In the absence of multiple scattering, ηS can be assumed to be independent of the height of the cloud.
This calibration method can be used to evaluate the estimated T_{f}(R) for Doppler lidar by checking whether the attenuated backscatter profiles obtained for the Doppler lidar after applying T_{f}(R) indeed provide similar ηS values for liquid clouds at different heights.
Figure 8 shows examples of Doppler lidar attenuated backscatter profiles after calibration and the derived apparent lidar ratio at two sites, Darwin and NSA. These sites have different values of f: Darwin has f=590 m and NSA has f set to infinity. For both cases, liquid clouds are present throughout the day, with altitudes varying from 2 to 6 km. When fully attenuating liquid clouds are present, the apparent lidar ratio is close to the expected value of 20 sr, regardless of the height of the cloud, thus confirming that the method of estimating T_{f}(R) is valid.
Limitations
Table 3 shows that the proportion of data that can be used for the T_{f}(R) parameter estimation varies significantly from site to site. Over a third of the available profiles from SGP are used, whereas only 0.3 % pass the filtering for Ascension. The lack of suitable profiles at Ascension is explained by the almost constant low cloud cover at this site, with very few profiles having a sufficient number of successive range gates.
Data quality is also a limiting factor, so at sites with very low aerosol optical depth, AOD, such as NSA, the Doppler lidar SNR decreases so rapidly that again there are few profiles having a sufficient number of successive range gates. Low AOD also impacts the performance of the ceilometer, with 48 % of the estimates at NSA discarded as outliers even after the initial filtering was performed. While the outlier removal can separate the good and the poor estimates, the largest uncertainty in D was at NSA. We attempted to perform the T_{f}(R) parameter estimation on a Doppler lidar from an ARM campaign in Cape Cod but could not obtain reliable estimates due to the low SNR of the ceilometer data.
The T_{f}(R) parameter estimation method is suitable only in situations where there is minimal difference in atmospheric extinction within the aerosol layer between the two instrument wavelengths of 910 and 1500 nm. Using AERONET (Aerosol Robotic Network) AOD measurements colocated at the ARM atmospheric observatories, the median difference in AOD at 870 and 1640 nm varied between 0.016 and 0.027, which should correspond closely to what might be expected for the difference between the ceilometer and Doppler lidar. Very occasional periods of notable AOD differences were observed at some sites, but including these profiles in time series extending beyond a year will have negligible impact on the T_{f}(R) parameter best estimate. However, there were breaks in the AOD measurements, and some periods experiencing a significant differential extinction may have gone unnoticed. An additional filter using AERONET AOD measurements to remove profiles experiencing significant differential extinction could be included in Fig. 2 for those sites where this may be an issue.
We have developed a method for deriving the telescope focus function and its uncertainty for pulsed heterodyne Doppler lidars, and we applied the method to Halo Photonics StreamLine and XR Doppler lidars. The method compares profiles of the Doppler lidar SNR to profiles of the attenuated backscatter coefficient from a colocated ceilometer, producing estimates for two parameters of the T_{f}(R): the effective focal length for the telescope, f; and the 1∕e^{2} effective diameter of a Gaussian beam, D. This method was developed because it also provides uncertainties in f, D, and T_{f}(R), which are necessary for quantitative use of the subsequently derived attenuated backscatter profiles. The method can be used to check the manufacturer specifications for these parameters, calculate them if not known, and also check their stability over time. The method does not necessarily require a permanently colocated ceilometer as the estimates of f and D can be made from a short time series with a colocated ceilometer; the Doppler lidar can then be moved to another location without a colocated ceilometer using the T_{f}(R) estimated previously, assuming that there has been no modification to the instrument parameters.
The method was applied to data from Doppler lidars with different configurations deployed at five ARM sites. Relative uncertainties in f for these instruments ranged from 6 % to 17 %, with the median uncertainty being 10 %; the relative uncertainty in D ranged from 2 % to 12 %, with a median of 3 %. The uncertainty in T_{f}(R) was calculated using Monte Carlo simulation, using three methods to prepare the input values. We recommend the direct resampling method, but reasonable results were obtained using statistically derived input values assuming a normal distribution. The envelope of relative uncertainties in T_{f}(R) ranged from 13 % to 32 % and depended on both the instrument configuration and the instrument location. We also show that, even for a Doppler lidar with the focus set at infinity, uncertainty remains in estimating T_{f}(R) arising from the uncertainty in D. The method was validated by calculating the apparent lidar ratio of fully attenuating liquid clouds from T_{f}(R) corrected profiles of Doppler lidar attenuated backscatter.
The impact of turbulence on T_{f}(R) was also investigated and was found to have no significant impact on the methodology described here for deriving the parameters f and D and their uncertainties from vertical profiles. However, the turbulent impact should not be ignored when measuring at low elevation angles close to the horizon, as it can modify T_{f}(R) considerably, especially in the far range. In these cases, the parameters f and D obtained from vertical measurements are still applicable, but the turbulent contribution to T_{f}(R) should be included when deriving the attenuated backscatter coefficient.
The T_{f}(R) estimation method is suitable only for conditions where the differential extinction at the two wavelengths of the Doppler lidar and the ceilometer is small, which can be identified, for example, using AOD from colocated AERONET observations.
The groundbased data used in this article are generated by the Atmospheric Radiation Measurement (ARM) user facility and are made available from the ARM Data Discovery website as follows: ceilometer data (CEIL, 2002) from https://doi.org/10.5439/1181954 and Doppler lidar data (DLFPT, 2010) from https://doi.org/10.5439/10251850.
EJO supervised and conceptualised the project. PP created the methodology and performed the analysis. AJM provided the processed input data. POA and AJM contributed to the development of the methodology. PP and EJO wrote the paper.
The authors declare that they have no conflict of interest.
This article is part of the special issue “Tropospheric profiling (ISTP11) (AMT/ACP interjournal SI)”. It is a result of the 11th edition of the International Symposium on Tropospheric Profiling (ISTP), Toulouse, France, 20–24 May 2019.
The Doppler lidar and ceilometer data used in this study were obtained from the Atmospheric Radiation Measurement (ARM) user facility, managed by the Office of Biological and Environmental Research for the U.S. Department of Energy Office of Science.
This research has been supported by the U.S. Department of Energy's Atmospheric System Research (ASR), an Office of Science, Office of Biological and Environmental Research (BER) programme (grant no. DESC0017338).
Open access funding provided by Helsinki University Library.
This paper was edited by Pauline Martinet and reviewed by two anonymous referees.
CEIL: Atmospheric Radiation Measurement (ARM) user facility, updated hourly. Ceilometer (CEIL). 20110621 to 20171221, ARM Mobile Facility (ASI) Ascension Island, South Atlantic Ocean; AMF1 (M1), Eastern North Atlantic (ENA) Graciosa Island, Azores, Portugal (C1), North Slope Alaska (NSA) Central Facility, Barrow AK (C1), Southern Great Plains (SGP) Central Facility, Lamont, OK (C1), Tropical Western Pacific (TWP) Central Facility, Darwin, Australia (C3), compiled by: Morris, V., Flynn, C., and Ermold, B., ARM Data Center, https://doi.org/10.5439/1181954, 2002. a
Chouza, F., Reitebuch, O., Groß, S., Rahm, S., Freudenthaler, V., Toledano, C., and Weinzierl, B.: Retrieval of aerosol backscatter and extinction from airborne coherent Doppler wind lidar measurements, Atmos. Meas. Tech., 8, 2909–2926, https://doi.org/10.5194/amt829092015, 2015. a
DLFPT: Atmospheric Radiation Measurement (ARM) user facility, updated hourly. Doppler lidar fixedpointing (DLFPT). 20110621 to 20171221, ARM Mobile Facility (ASI) Ascension Island, South Atlantic Ocean; AMF1 (M1), Eastern North Atlantic (ENA) Graciosa Island, Azores, Portugal (C1), North Slope Alaska (NSA) Central Facility, Barrow AK (C1), Southern Great Plains (SGP) Central Facility, Lamont, OK (C1), Tropical Western Pacific (TWP) Central Facility, Darwin, Australia (C3), compiled by: Newsom, R. and Krishnamurthy, R., ARM Data Center, https://doi.org/10.5439/1025185, 2010. a
Engelmann, R., Wandinger, U., Ansmann, A., Müller, D., Z̆eromskis, E., Althausen, D., and Wehner, B.: Lidar observations of the vertical aerosol flux in the planetary boundary layer, J. Atmos. Ocean. Tech., 25, 1296–1306, https://doi.org/10.1175/2007JTECHA967.1, 2008. a
Flentje, H., Heese, B., Reichardt, J., and Thomas, W.: Aerosol profiling using the ceilometer network of the German Meteorological Service, Atmos. Meas. Tech. Discuss., 3, 3643–3673, https://doi.org/10.5194/amtd336432010, 2010. a
Frehlich, R. G. and Kavaya, M. J.: Coherent laser radar performance for general atmospheric refractive turbulence, Appl. Optics, 30, 5325–5352, https://doi.org/10.1364/AO.30.005325, 1991. a, b, c, d, e
Hannon, S. M., Thomson, J. A. L., and Smith, D. D.: Plume detection and tracking using Doppler lidar aerosol and wind data, in: Application of Lidar to Current Atmospheric Topics III, edited by: Sedlacek III, A. J. and Fischer, K. W., 3757, 28–39, International Society for Optics and Photonics, SPIE, https://doi.org/10.1117/12.366435, 1999. a
Harvey, N. J., Hogan, R. J., and Dacre, H. F.: A method to diagnose boundary‐layer type using Doppler lidar, Q. J. Roy. Meteor. Soc., 139, 1681–1693, https://doi.org/10.1002/qj.2068, 2013. a
Henderson, S. W., Gatt, P., Rees, D., and Huffaker, R. M.: Wind Lidar, in: Laser Remote Sensing, edited by: Fujii, T. and Fukuchi, T., 469–722, CRC Taylor and Francis, 2005. a
Hill, C.: Coherent focused lidars for Doppler sensing of aerosols and wind, Remote Sens., 10, 466, https://doi.org/10.3390/rs10030466, 2018. a
Hirsikko, A., O'Connor, E. J., Komppula, M., Korhonen, K., Pfüller, A., Giannakaki, E., Wood, C. R., BauerPfundstein, M., Poikonen, A., Karppinen, T., Lonka, H., Kurri, M., Heinonen, J., Moisseev, D., Asmi, E., Aaltonen, V., Nordbo, A., Rodriguez, E., Lihavainen, H., Laaksonen, A., Lehtinen, K. E. J., Laurila, T., Petäjä, T., Kulmala, M., and Viisanen, Y.: Observing wind, aerosol particles, cloud and precipitation: Finland's new groundbased remotesensing network, Atmos. Meas. Tech., 7, 1351–1375, https://doi.org/10.5194/amt713512014, 2014. a
Hogan, R. J., Illingworth, A. J., O'Connor, E. J., and Polares Baptista, J. P. V.: Characteristics of mixedphase clouds. II: A climatology from groundbased lidar, Q. J. Roy. Meteor. Soc., 129, 2117–2134, https://doi.org/10.1256/qj.01.209, 2003. a
Hu, Q., Rodrigo, P. J., Iversen, T. F. Q., and Pedersen, C.: Investigation of spherical aberration effects on coherent lidar performance, Opt. Express, 21, 25670–25676, https://doi.org/10.1364/OE.21.025670, 2013. a
Huber, P. J. and Ronchetti, E.: Robust Statistics, chap. 5, 105–123, John Wiley & Sons, Ltd, https://doi.org/10.1002/9780470434697.ch5, 2009. a
Illingworth, A. J., Cimini, D., Haefele, A., Haeffelin, M., Hervo, M., Kotthaus, S., Löhnert, U., Martinet, P., Mattis, I., O’Connor, E. J., and Potthast, R.: How can existing groundbased profiling instruments improve European weather forecasts?, B. Am. Meteorol. Soc., 100, 605–619, https://doi.org/10.1175/BAMSD170231.1, 2019. a
Kotthaus, S. and Grimmond, C. S. B.: Atmospheric boundarylayer characteristics from ceilometer measurements. Part 1: A new method to track mixed layer height and classify clouds, Q. J. Roy. Meteor. Soc., 144, 1525–1538, https://doi.org/10.1002/qj.3299, 2018. a
Leys, C., Ley, C., Klein, O., Bernard, P., and Licata, L.: Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median, J. Exp. Soc. Psychol., 49, 764–766, https://doi.org/10.1016/j.jesp.2013.03.013, 2013. a
Lolli, S., D’Adderio, L. P., Campbell, J. R., Sicard, M., Welton, E. J., Binci, A., Rea, A., Tokay, A., Comerón, A., Barragan, R., Baldasano, J. M., Gonzalez, S., Bech, J., Afflitto, N., Lewis, J., and Madonna, F.: Vertically resolved precipitation intensity retrieved through a synergy between the groundBased NASA MPLNET lidar network measurements, surface disdrometer datasets and an analytical model solution, Remote Sens., 10, 1102, https://doi.org/10.3390/rs10071102, 2018. a
Manninen, A. J., O'Connor, E. J., Vakkari, V., and Petäjä, T.: A generalised background correction algorithm for a Halo Doppler lidar and its application to data from Finland, Atmos. Meas. Tech., 9, 817–827, https://doi.org/10.5194/amt98172016, 2016. a
Manninen, A. J., Marke, T., Tuononen, M., and O'Connor, E. J.: Atmospheric boundary layer classification with Doppler lidar, J. Geophys. Res.Atmos., 123, 8172–8189, https://doi.org/10.1029/2017JD028169, 2018. a, b, c
Mather, J. H., Turner, D. D., and Ackerman, T. P.: Scientific maturation of the ARM Program, Meteorol. Monogr., 57, 4.1–4.19, https://doi.org/10.1175/AMSMONOGRAPHSD150053.1, 2016. a
Morgan, M. G. and Henrion, M.: Uncertainty: A guide to dealing with uncertainty in quantitative risk and policy analysis, 172–219, Cambridge University Press, https://doi.org/10.1017/CBO9780511840609.009, 1990. a
O'Connor, E. J., Illingworth, A. J., and Hogan, R. J.: A technique for autocalibration of cloud lidar, J. Atmos. Ocean. Tech., 21, 777–786, https://doi.org/10.1175/15200426(2004)021<0777:ATFAOC>2.0.CO;2, 2004. a, b, c
Pearson, G., Davies, F., and Collier, C.: An analysis of the performance of the UFAM pulsed Doppler lidar for observing the boundary layer, J. Atmos. Ocean. Tech., 26, 240–250, 2009. a
Pearson, G. N., Roberts, P. J., Eacock, J. R., and Harris, M.: Analysis of the performance of a coherent pulsed fiber lidar for aerosol backscatter applications, Appl. Optics, 41, 6442–6450, https://doi.org/10.1364/AO.41.006442, 2002. a
Roadcap, J. R. and Tracy, P.: A preliminary comparison of daylit and night ${C}_{n}^{\mathrm{2}}$ profiles measured by thermosonde, Radio Sci., 44, https://doi.org/10.1029/2008RS003921, 2009. a
Rye, B. J. and Hardesty, R. M.: Discrete spectral peak estimation in incoherent backscatter heterodyne lidar. I. Spectral accumulation and the CramerRao lower bound, IEEE T. Geosci. Remote, 31, 16–27, https://doi.org/10.1109/36.210440, 1993. a
Tonttila, J., O'Connor, E. J., Hellsten, A., Hirsikko, A., O'Dowd, C., Järvinen, H., and Räisänen, P.: Turbulent structure and scaling of the inertial subrange in a stratocumulustopped boundary layer observed by a Doppler lidar, Atmos. Chem. Phys., 15, 5873–5885, https://doi.org/10.5194/acp1558732015, 2015. a
Träumner, K., Handwerker, J., Wieser, A., and Grenzhäuser, J.: A Synergy Approach to Estimate Properties of Raindrop Size Distributions Using a Doppler Lidar and Cloud Radar, J. Atmos. Ocean. Tech., 27, 1095–1100, https://doi.org/10.1175/2010JTECHA1377.1, 2010. a
Tucker, S. C., Senff, C. J., Weickmann, A. M., Brewer, W. A., Banta, R. M., Sandberg, S. P., Law, D. C., and Hardesty, R. M.: Doppler lidar estimation of mixing height using turbulence, shear, and aerosol Profiles, J. Atmos. Ocean. Tech., 26, 673–688, https://doi.org/10.1175/2008JTECHA1157.1, 2009. a
Van Tricht, K., Gorodetskaya, I. V., Lhermitte, S., Turner, D. D., Schween, J. H., and Van Lipzig, N. P. M.: An improved algorithm for polar cloudbase detection by ceilometer over the ice sheets, Atmos. Meas. Tech., 7, 1153–1167, https://doi.org/10.5194/amt711532014, 2014. a
Westbrook, C., Illingworth, A., O'Connor, E., and Hogan, R.: Doppler lidar measurements of oriented planar ice crystals falling from supercooled and glaciated layer clouds, Q. J. Roy. Meteor. Soc., 136, 260–276, https://doi.org/10.1002/qj.528, 2010a. a, b, c, d
Westbrook, C. D., Hogan, R. J., O'Connor, E. J., and Illingworth, A. J.: Estimating drizzle drop size and precipitation rate using twocolour lidar measurements, Atmos. Meas. Tech., 3, 671–681, https://doi.org/10.5194/amt36712010, 2010b. a
Zhao, Y., Post, M. J., and Hardesty, R. M.: Receiving efficiency of monostatic pulsed coherent lidars. 1: Theory, Appl. Optics, 29, 4111–4119, https://doi.org/10.1364/AO.29.004111, 1990. a