the Creative Commons Attribution 4.0 License.

the Creative Commons Attribution 4.0 License.

# An explicit formulation for the retrieval of the overlap function in an elastic and Raman aerosol lidar

### Adolfo Comerón

### Constantino Muñoz-Porcar

### Alejandro Rodríguez-Gómez

### Michaël Sicard

### Federico Dios

### Cristina Gil-Díaz

### Daniel Camilo Fortunato dos Santos Oliveira

### Francesc Rocadenbosch

We derive an explicit (i.e., non-iterative) formula for the retrieval of the overlap function in an aerosol lidar with both elastic and Raman N_{2} and/or O_{2} channels used for independent measurements of aerosol backscatter and extinction coefficients. The formula requires only the measured, range-corrected elastic and the corresponding Raman signals, plus an assumed lidar ratio. We assess the influence of the lidar ratio error in the overlap function retrieval and present retrieval examples.

At near ranges, lidar signals suffer from a varying overlap between the emitted laser beam and the field of view of the receiving optical assembly. The overlap function of a lidar system can be defined as the ratio between the power scattered by a scattering volume at a given range that reaches the photodetector (excluding transmission losses) and the power scattered by the same scattering volume that reaches the telescope aperture (Comeron et al., 2011). This ratio is a function of range, especially at short ranges, and depends on the optical and geometrical arrangement of the transmitting and receiving optics of the instrument. The key parameters determining the overlap function are those related to the laser beam features (diameter, shape and divergence), receiver optical properties (telescope diameter, focal length and field stop diameter), and relative location and alignment between transmitter and receiver optical axes (Halldórsson and Langerholc, 1978; Lefrère, 1982). In a perfectly aligned system, the overlap function is 0 at the telescope aperture level and progressively grows up to a constant value, where all the backscattered radiation collected by the telescope aperture, or at least a constant proportion of it, reaches the photodetector. In practical cases, misalignments may make the overlap function dependence on range depart from the ideal behavior just described.

The existence of a varying overlap prevents the system from trustfully providing
lidar signals for ranges below the altitude at which the overlap attains a
constant value, thus limiting the minimum operational range of the lidar
instrument. To reduce these overlap issues, some systems duplicate their
receivers, enabling both far- and near-range telescopes and detectors and
combining their respective signals for reconstructing a lidar signal with an
extended (towards the lower end) constant overlap range. For example,
Polly^{XT} systems (Engelmann et al., 2016) use
this type of solution, and their full overlap altitude is reduced down to
∼100 m. Alternatives when such a hardware-based extension of
the operational range is not possible rely on the calculation or estimation
of the overlap function and on the correction of the detected signals from
the effect of the varying overlap. Several authors have developed
theoretical calculations of the overlap function using the transmitter and
the receiver optical parameters, both on an analytical basis (Sassen and Dodd, 1982; Ancellet et al., 1986; Kuze et al., 1998; Stelmaszczyk et al., 2005; Comeron et al., 2011) and by relying on ray-tracing procedures (e.g., Kumar and Rocadenbosch, 2013). However, such theoretical approaches are in many cases not practical because most of the system parameters in which they are based on are not easily measurable (Kokkalis, 2017), and they change, sometimes unpredictably and unnoticeably, with time. Alternatives to theoretical calculations are based on experimental estimations relying on practical field lidar measurements and inversions. A first proposal, presented by Sasano et al. (1979), is based on the assumption of a homogeneous atmosphere up to distances above
the full overlap altitude. In many cases, this method is not practical,
first, because its applicability depends on the state of the atmosphere and,
second, because in order to assure the required atmospheric homogeneity, it
demands a horizontal alignment of the lidar line of sight that is not
always possible. Further contributions, making different assumptions about the
atmospheric conditions, were proposed by Tomine et al. (1989), Dho et al. (1997) and Vande Hey et al. (2011).

A comparison with a reference system not affected (or less affected) by the varying overlap has been proposed by Guerrero-Rascado et al. (2010). Hu et al. (2005) proposed retrieving the overlap profile by comparing Raman signals with radiosonde profiles; Povey et al. (2012) performed a nonlinear regression using optical analysis combined with measured aerosol optical thickness; Mahagammulla Gamage et al. (2019) obtained the overlap profile as a by-product of a retrieval of temperature profiles with multiple pure rotational Raman channels, using an optimal estimation method. For motor-controlled lidars, a beam-mapping procedure has been proposed by Di Paolantonio et al. (2022).

Up to date, one of the best-established and widely accepted methods was presented by Wandinger and Ansmann (2002). This approach assumes that the lidar system has a Raman channel to independently retrieve the aerosol extinction coefficient and relies on the fact that, under the assumption of the same overlap function for the elastic and the Raman channels, the Raman inversion of the backscatter coefficient is not affected by the incomplete overlap. Further contributions, including an analysis of the effect of the lidar ratio (LR) used, were reported by Li et al. (2016).

In this paper, we present an alternative formulation for the retrieval of the overlap function based on the same principles as the one discussed in Wandinger and Ansmann (2002), i.e., the fact that the backscatter coefficient retrieved by the Raman method is not affected by the incomplete overlap. However, unlike in the Wandinger and Ansmann method, our formulation results in an explicit formula that does not require iterative inversions of the backscatter coefficient by both the Raman (Ansmann et al., 1992) and Klett (Klett, 1985; Sasano et al., 1985) methods. Section 2 develops the proposed formulation. In Sect. 3 we assess the effect of an erroneous lidar ratio on the retrieved overlap function. Examples based on real measurements are presented in Sect. 4. Conclusions and outlook are summed up in Sect. 5.

The proposed method uses, like Wandinger and Ansmann (2002), the elastic and Raman signals backscattered by an air volume under the excitation of one of the emitted wavelengths of an aerosol lidar. First, let us consider the expression of the range-corrected elastic lidar signal, *X*(*R*), affected by an overlap function, *O*(*R*), *R* being the range to the lidar, where the aerosol and molecular components of the extinction coefficient are written using the corresponding lidar ratios at the elastic wavelength, *S*_{a0}(*R*) and *S*_{m0} (Bucholtz, 1995; D'Amico et al., 2016) respectively:

where *A* is an instrument constant, and *β*_{a0}(*R*) and
*β*_{m0}(*R*) are respectively the aerosol and molecular components of the backscatter coefficient at the emitted wavelength *λ*_{0}. To avoid using the instrument constant, we look for an aerosol-free range, *R*_{m}, at which the aerosol backscatter coefficient can be assumed
to be 0 and where *β*_{m0}(*R*_{m}) can be estimated from the pressure and the temperature provided by a radiosonde or by using a standard model of the atmosphere. We assume as well that at that range the overlap function has attained a constant value that we set conventionally to *O*(*R*_{m})=1; we have at that range

Dividing Eq. (1) by Eq. (2) and reordering terms, we obtain

Now we follow steps similar to those leading to the well-known Klett's formula (Klett, 1985; Gimmestad and Roberts, 2010) but explicitly keeping the overlap function in the equations. Multiplying both members of Eq. (3) by

we obtain

In the left-hand member of Eq. (4) we recognize that

with which Eq. (4) can be rewritten as

Integrating both members of Eq. (6) between *R*_{m} and *R* and rearranging terms one obtains

Substituting the right member of Eq. (7) for $\mathrm{exp}\left\{-\mathrm{2}{\int}_{{R}_{\mathrm{m}}}^{R}{S}_{\mathrm{a}\mathrm{0}}\left(x\right)\left[{\mathit{\beta}}_{\mathrm{a}\mathrm{0}}\right(x)+{\mathit{\beta}}_{\mathrm{m}\mathrm{0}}(x\left)\right]\mathrm{d}x\right\}$ in the left-hand member of Eq. (4) and rearranging terms we arrive at

Note that Eq. (8) is Klett's solution of the lidar equation (Klett, 1985; Sasano et al., 1985), except for the overlap function appearing in its left-hand member and in the integral in the denominator in its right-hand member.

Now, from the Raman inversion method we obtain, assuming that the overlap functions of both the elastic and the Raman channels are the same, a backscatter coefficient not affected by the varying overlap (Ansmann et al., 1992):

with *X*_{R}(*R*) being the range-corrected Raman signal; *α*_{aR}(*R*) and *β*_{mR}(*R*) the aerosol extinction and the molecular backscatter coefficients respectively, both at the Raman-shifted wavelength *λ*_{R}; and *S*_{mR} the molecular lidar ratio at *λ*_{R} (D'Amico et al., 2016). If we divide Eq. (8) by Eq. (9), we finally arrive at the formula

If we knew the aerosol differential transmission term $\mathrm{exp}\left\{{\int}_{R}^{{R}_{\mathrm{m}}}\left[{\mathit{\alpha}}_{\mathrm{a}\mathrm{0}}\right(x)-{\mathit{\alpha}}_{\mathrm{a}R}(x\left)\right]\mathrm{d}x\right\}$ and the aerosol lidar ratio *S*_{a0} (the other terms are assumed to be known because they are either measured or derived from radiosonde measurements), Eq. (10) could be solved iteratively for *O*(*R*) by assuming an initial *O*(*R*) in the
right-hand member of Eq. (10) (e.g., *O*(*R*)=1, or the immediately previous overlap function assumed as valid for the system). This will give a new *O*(*R*) estimate that would be substituted again in the right hand of Eq. (10), and the procedure will continue until *O*(*R*) converges.

However, it is also possible to obtain an explicit expression for *O*(*R*) by casting Eq. (9) into the form of a Volterra integral equation (Mathews and Walker, 1970; Sect. 11-5), which, in turn, can be converted into a first-degree differential equation that can be integrated using standard techniques (Mathews and Walker, 1970; Sect. 1-1; see Appendix A for details). To do that, we call

and define the functions *g*(*R*), *ϕ*(*R*) and *ψ*(*R*) as

and

Then, following the steps detailed in Appendix A, one arrives at the explicit form of the overlap function

Note that every term in Eqs. (10) and (15), except the aerosol lidar ratio
profile *S*_{a0}(*R*) and the aerosol extinction coefficients,
can either be obtained directly from the elastic and Raman lidar signals
(*X*(*R*) and *X*_{R}(*R*)) or be calculated from the pressure and temperature provided by a radiosonde or by using a standard model of the atmosphere (*β*_{m0}(*R*) and *β*_{mR}(*R*)). Note as well that if a purely rotational Raman channel is used, the differential molecular and aerosol transmission terms respectively $\mathrm{exp}\left\{{\int}_{R}^{{R}_{\mathrm{m}}}\left[{S}_{\mathrm{m}\mathrm{0}}{\mathit{\beta}}_{\mathrm{m}\mathrm{0}}\right(x)-{S}_{\mathrm{m}R}{\mathit{\beta}}_{\mathrm{m}R}(x\left)\right]\mathrm{d}x\right\}$
and $\mathrm{exp}\left\{{\int}_{R}^{{R}_{\mathrm{m}}}\left[{\mathit{\alpha}}_{\mathrm{a}\mathrm{0}}\right(x)-{\mathit{\alpha}}_{\mathrm{a}R}(x\left)\right]\mathrm{d}x\right\}$ can safely be ignored in Eqs. (10) and (15). In Appendix B we assess the error committed when a vibro-rotational Raman channel is used and the wavelength differences can no longer be neglected.

Although based on the same principles as the iterative method proposed in Wandinger and Ansmann (2002), the formulation of Eq. (15) has the advantages of not requiring iterations (admittedly, not a decisive issue with the current computing technology) and, more importantly, providing insight into the effect of the assumed aerosol lidar ratio on the retrieved overlap function (see Sect. 3) and the systematic error incurred when the differential aerosol transmission at the emitted and Raman wavelengths cannot be neglected (see Appendix B).

To assess the influence of the assumed lidar ratio on the overlap function
retrieval we substitute in Eq. (10) the expressions of *X*(*R*) and *X*_{R}(*R*) that would correspond to a given aerosol distribution

where *A* and *B* are instrument constants. We also assume that we may use
an erroneous lidar ratio

where *S*_{a0}(*R*) is the “true” lidar ratio (actually unknown) and Δ*S*_{a0}(*R*) the deviation from it. Using ${S}_{\mathrm{a}\mathrm{0}}^{\prime}\left(R\right)$ in Eq. (15) and replacing in it the expressions
of *X*(*R*) and *X*_{R}(*R*) given by Eqs. (16), we find, after some boring and cumbersome but otherwise straightforward algebraic developments, the surprisingly simple result

where *O*^{′}(*R*) is the overlap function found, different from the true one, *O*(*R*), because of the error Δ*S*_{a0} in the lidar ratio.

One reaches the following conclusions from Eq. (18):

- a.
If the atmosphere measured to retrieve the overlap function was aerosol-free, i.e.,

*β*_{a0}(*R*)=0 for all ranges, the assumed lidar ratio (hence Δ*S*_{a0}) would be irrelevant, since Eq. (18) would lead to ${O}^{\prime}\left(R\right)=O\left(R\right)$. - b.
Likewise, if there is no aerosol for any range ${R}_{T}<R<{R}_{\mathrm{m}}$, in that range ${O}^{\prime}\left(R\right)=O\left(R\right)$ regardless of the assumed lidar ratio.

- c.
If Δ

*S*_{a0}(*x*)>0, then ${O}^{\prime}\left(R\right)<O\left(R\right)$ in the range with aerosol. - d.
If Δ

*S*_{a0}(*x*)<0, then ${O}^{\prime}\left(R\right)>O\left(R\right)$ in the range with aerosol.

Note that because *β*_{a0} tends to be larger at shorter wavelengths, the sensitivity of the retrieved overlap function to an error in the assumed lidar ratio is expected to be larger at shorter wavelengths.

We have used Eq. (15) to obtain estimates of the overlap function at 355
and 532 nm of the lidar of the Universitat Politècnica de Catalunya
(UPC), a combined eight-channel multispectral Raman–elastic backscatter
lidar that is described in Kumar et al. (2011), with the modification in the UV branch of the wavelength separation unit described in Zenteno-Hernández et al. (2021) to implement a ${\mathrm{N}}_{\mathrm{2}}/{\mathrm{O}}_{\mathrm{2}}$ purely rotational Raman channel at 354 nm. This instrument belongs to the European Aerosol Research Lidar Network (EARLINET), currently integrated into the Aerosol, Clouds and Trace Gases Research Infrastructure (ACTRIS). To retrieve the overlap function at 355 nm, we have used the purely rotational Raman channel, which provides a higher signal-to-noise ratio than the vibro-rotational one (Zenteno-Hernández et al., 2021). For the overlap function at 532 nm we used the elastic signal return and the signal of the N_{2} vibro-rotational Raman channel at 607 nm.

To illustrate the effect of the assumed aerosol lidar ratio, we have chosen two nighttime measurements (60 min measurement on 11 November 2021 starting at 20:41 UTC and 60 min measurement on 1 December 2021 starting at 01:44 UTC) corresponding to situations with a relatively low aerosol load.

Figure 1 presents the range-corrected, Rayleigh-fitted lidar signals used for computing the overlap profiles. The 11 November signals were fitted to the Rayleigh profile between 4 and 6 km because of the presence of clouds (partially visible in the plot) from 6.3 km upwards. The 1 December signals were fitted between 7 and 11 km. In both cases, the lidar signals fit to the Rayleigh profile with great accuracy in the interval from 4 to 6 km (to 8 km in the case of 1 December), indicating an aerosol-free atmosphere.

Figure 2 presents the backscatter coefficients obtained with the Raman method (Eq. 9; no smoothing applied to the signals) at 355 and 532 nm, neglecting the difference between the aerosol extinction coefficients at the emitted and Raman wavelengths. Note that this approximation is very well justified when the Raman channel is a purely rotational one, as in the case of the backscatter coefficient at 355 nm, since the two signals employed are at almost the same wavelength. Figure 2 shows that the aerosol backscatter coefficient at both wavelengths was much lower for the 1 December measurement than for the 11 November one. It also shows that the backscatter coefficient for the same day is higher at the shorter wavelength. Figure 2 also warns of a possible breakdown of the equal-overlap function hypothesis for the elastic and Raman channels, more clearly seen examining the profiles of 1 December: while the 532 nm aerosol backscatter coefficient shows a reasonable behavior until very low altitudes, the 355 nm one has a sudden fall below approximately 400 m. For this reason, in this particular case of optical alignment we should distrust the overlap function retrieval below that height for all cases.

Figure 3 shows the results of the overlap function retrieval with our formulation (Eq. 15; neglecting the differential aerosol transmission terms $\mathrm{exp}\left\{{\int}_{R}^{{R}_{\mathrm{m}}}\left[{\mathit{\alpha}}_{\mathrm{a}\mathrm{0}}\right(x)-{\mathit{\alpha}}_{\mathrm{a}R}(x\left)\right]\mathrm{d}x\right\}$ assumed 1; see Appendix B for the assessment of the error bound entailed by this assumption) for three “reasonable” lidar ratios (25, 50 and 75 sr) from the 11 November 2021 measurement. The reference height is taken at 6 km, where the Rayleigh fit of the signals indicates the absence of aerosol (in agreement with the profiles of Figs. 1 and 2). The detected lidar signal sequences are noisy, especially the Raman ones, whereas the overlap function cannot have steep or sudden variations at far ranges; therefore, a smoothing procedure, coupled with a Monte Carlo routine to assess the residual error bars, has been employed. An overlap profile retrieved with the original noisy sequences (only for 50 sr lidar ratio) is plotted as well.

The raw elastic and Raman signal sequences detected by our lidar were fitted
to a Rayleigh reference profile obtained from a nearby radiosonde. The
sequences were corrected in range as well, being all the processes common
in lidar inversion techniques. The result of this process leads to *X*(*n*) and *X*_{R}(*n*), standing for elastic and Raman signal sequences. Previous (noisy) estimates of the overlap profiles were calculated with these sequences.

These sequences were then smoothed to reduce the remaining noise, especially in the segments corresponding to high altitudes. This smoothing uses an adaptive sliding average approach. Each sample of the smoothed sequence was calculated as

where the sub-index *X* stands for either elastic or Raman. The averaging
window length *L* varies from 1 to 150 (3.75 to 562.5 m considering the raw range resolution of our lidar) as *n* grows. For low altitudes (low *n*) the
noise is not significant, and the expected lidar signals show relevant variations, so *L* must be short, while it can be made longer for sequence
segments corresponding to farther ranges, especially in the molecular zone.

The noise of these signals is estimated by comparing the non-smoothed
sequences with the smoothed one. The estimation of this noise is necessary
to create the different realizations in a Monte Carlo strategy to compute
the error bars of the overlap estimation. Considering that the sequences
have been smoothed by performing a (*L*+1)-long average, the
standard deviation of the *n*th sample is estimated as (Papoulis and Pillai, 2002)

The uncertainty of the calculated overlap profiles is estimated by using a
common Monte Carlo approach. With the statistics obtained with Eqs. (19) and (20), *N*_{MC} (usually *N*_{MC}≈100) pairs of statistically independent elastic and Raman signal sequences are synthesized. Each of these synthesized sequences are generated as

where each *e*_{X_k}(*n*) is a realization of a Gaussian random variable with 0 average and standard deviation Δ*X*_{X}(*n*).

With these *N*_{MC} sequence pairs, *N*_{MC} overlap profiles Ov_{k}(*n*) are calculated. The average overlap profiles Ov(*n*) and error bars ΔOv(*n*) presented in the next figures have been calculated as (Papoulis and Pillai, 2002)

In Fig. 4 the retrieved overlap functions from data of 1 December 2021 are represented for the same assumed lidar ratios as in Fig. 3. Because we have arbitrarily normalized the profile to the reference height, where the overlap function has reached a stable value, values greater than 1, as shown in Figs. 2 and 3, at lower ranges are possible and reveal a non-perfect alignment, in particular, a slight crossing between the laser beam and the receiver field-of-view axes, leading to a loss of energy from the far range (see for example Fig. 1a in Kokkalis, 2017, with laser tilt *A*_{tilt}, half-width laser beam divergence (LBD) and receiver field of view (RFOV) fulfilling the conditions ${A}_{\mathrm{tilt}}+\text{LBD}>\text{RFOV}$ and ${A}_{\mathrm{tilt}}-\text{LBD}<\text{RFOV}$). As
expected (Sect. 3), being the aerosol backscatter coefficients at both wavelengths lower in this measurement, the difference between the overlaps obtained with different lidar ratios is lower than for 11 November. Also, because the backscatter coefficient at 532 nm is lower than at 355 nm, the differences in the retrieved overlap functions are less sensitive to the guessed lidar ratio at the former wavelength, being in fact almost negligible. An overlap profile retrieved with the original noisy sequences (for LR =50 sr) is plotted as well. Although using a different, explicit non-iterative formulation, the method presented in this paper relies on the same basis as the one given by Ulla Wandinger and Albert Ansmann. The reader can check that for the same measured data and assumed lidar ratio both methods for a sufficient number of iterations in Wandinger and Ansmann (2002) yield indistinguishable results.

Based on the same principle as in Wandinger and Ansmann (2002), i.e., that the aerosol backscatter coefficient derived by the Raman method (Ansmann et al., 1992) is not affected by the lidar range-varying overlap (under the assumption of the same overlap function for the elastic and the Raman channels), a new formulation for deriving the overlap function of an aerosol lidar system equipped with Raman channels has been presented. As input data, the method uses the elastic and Raman signals and a guess of the lidar ratio corresponding to the emitted wavelength of interest. The novelty of our approach consists of the derivation of an explicit formula in which no iterations have to be performed.

Results of the formula are illustrated with two examples, both with a low aerosol load but one of them with a much lower load than the other, showing the effect of the guessed lidar ratio on the overlap function retrievals.

The explicit formula allows one to assess the errors committed when an erroneous lidar ratio is used (Sect. 3), showing, as already stated by Wandinger and Ansmann (2002), that the retrieval of the overlap function is less prone to errors when performed in clear atmospheres. It also makes it possible to find systematic error bounds associated with the uncertainty in the different aerosol transmissions at the elastic and the Raman wavelengths when Raman vibro-rotational channels are used (Appendix B). Section 3 also cautions against trying to derive a lidar ratio using the corrected-for-overlap signal. Actually, one could be tempted to think of the following procedure: an overlap function is retrieved using a guessed aerosol lidar ratio; with that overlap function, the Raman signal is corrected, and an aerosol extinction coefficient is calculated, which, divided by the aerosol backscatter, gives a new lidar ratio, which is in turn used to retrieve a new overlap function, and so on. However, Eq. (18) shows that this procedure does not converge, for if a too low lidar ratio is used as the first guess, the overlap function will be enhanced in the range with aerosol; when correcting with this enhanced overlap function, the Raman signal will be suppressed, which will give rise to an aerosol extinction coefficient lower than due and, consequently, to a lower new lidar ratio. A similar reasoning goes on if the guessed aerosol lidar ratio is too high. The determination of the required lidar ratio from Raman inversions needs atmospheric regions with both significant aerosol load and stable overlap. However, in cases with regions where both conditions are fulfilled, using the retrieved lidar ratio for overlap estimations requires assuming that the type of aerosol is uniform down to the ground. Moreover, as seen in Sect. 3, in aerosol-loaded scenarios, errors in the lidar ratio determination yield greater errors in the estimation of the overlap profile. A more conservative approach is to stay with situations with a low aerosol load at low altitudes and use the aerosol backscatter profiles derived with the Raman method (e.g., Fig. 2) together with a sun- or lunar-photometer aerosol optical depth (AOD) measurement and find the aerosol lidar ratio that, multiplied by the integrated aerosol backscatter coefficient, would yield the AOD measured by the photometer. However, these techniques are out of the scope of this paper, which aims only at presenting the explicit formulation of the overlap function and discussing the effect of the assumed lidar ratio on the retrieved profiles.

We outline here the mathematical details to obtain Eq. (15). Using the definitions of Eqs. (12), (13) and (14), Eq. (10) can be written as the Volterra integral equation

which is amenable to a differential equation. In order to do that, we define the function

which, substituting into Eq. (A1), yields

We next take the derivative Eq. (A2)

and substitute Eq. (A3) on it to obtain, after reordering terms,

To integrate that equation, we define an integrating factor $\mathrm{exp}\left[{\int}_{R}^{{R}_{\mathrm{m}}}\mathit{\varphi}\right(x\left)\mathit{\psi}\right(x\left)\mathrm{d}x\right]$ and multiply both members of Eq. (A5) by it, which allows us to recast the equation as

Integrating both members of Eq. (A6) between *R* and *R*_{m} and noting that, by construction, *u*(*R*_{m})=0, leads to

Finally, taking the derivatives of both members of Eq. (A7) and considering Eq. (A4) one obtains

In our case, $g\left(R\right)=\frac{\mathrm{1}}{\mathrm{2}}\frac{X\left({R}_{\mathrm{m}}\right)}{{\mathit{\beta}}_{\mathrm{m}\mathrm{0}}\left({R}_{\mathrm{m}}\right)}\mathit{\varphi}\left(R\right)$ (see Eqs. 12 and 13), which makes

We assess the error incurred in the estimation of *O*(*R*) (Eq. 15) when neglecting the difference in molecular lidar ratios and the differential aerosol transmission term.

We start by noting that (Bucholtz, 1995; D'Amico et al., 2016) the molecular lidar ratio at a wavelength *λ* can be written as

with ${S}_{\mathrm{m}}=\frac{\mathrm{8}\mathit{\pi}}{\mathrm{3}}$, and *δ*_{nλ} is the depolarization factor that takes into account the anisotropy of the air
molecules (Bucholtz, 1995). We can then write the terms
*S*_{m0}*β*_{m0}(*x*)−*S*_{mR}*β*_{mR}(*x*) in
Eq. (15) as

where, for a vibro-rotational Raman channel, $\frac{{S}_{\mathrm{m}}}{\mathrm{2}}\left[{\mathit{\delta}}_{n\mathrm{0}}{\mathit{\beta}}_{\mathrm{m}\mathrm{0}}\right(x)-{\mathit{\delta}}_{nR}{\mathit{\beta}}_{\mathrm{m}R}(x\left)\right]>\mathrm{0}$.

The terms *α*_{a0}(*x*)−*α*_{aR}(*x*) can be written as

with *a*(*x*) being the Ångström exponent, which is in general positive.

By examining Eqs. (15), (B2) and (B3), it is seen that

with *O*_{0}(*R*) defined as

i.e., ignoring the difference in the aerosol transmissions at the elastic and Raman wavelengths and

with AOD_{0} being the aerosol optical depth at the wavelength *λ*_{0} and *a*_{max} the maximum Ångström exponent found along the lidar line of sight.

As an example, the upper-bound factor $O\left(R\right)/{O}_{\mathrm{0}}\left(R\right)$ according to Eq. (B6), with the data used to obtain the lower panel of Fig. 3 and *S*_{a}=50 sr, AOD_{0}=0.05 and ${a}_{max}=\mathrm{1.3}$, is given in Fig. B1. We see as well that considering ${S}_{\mathrm{m}\mathrm{0}}={S}_{\mathrm{m}R}={S}_{\mathrm{m}}$ has only a small impact on the bound.

The range-corrected signals used for the overlap function retrievals can be downloaded from https://doi.org/10.34810/data704 (Comerón Tejero et al., 2023).

AC developed the new formulation, the software needed and the overall concept, and he wrote part of the text and generated Figs. 2 and B1. CMP contributed with data analysis and wrote some parts of the text. ARG developed the Monte Carlo approach to smooth the overlap profiles and to calculate the error bars, and he generated Figs. 1, 3 and 4. The contributions of MS, FD, CGD, DCFDSO and FR have been invaluable in the inception, development and consolidation of our lidar system, besides the contribution in measurements, data analysis and selection of the most appropriate cases for the retrieval of the overlap profiles.

The contact author has declared that none of the authors has any competing interests.

Publisher’s note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The authors acknowledge the support of the Ministry for Science and Innovation to ACTRIS ERIC.

This research has been supported by the Agencia Estatal de Investigación (grant no. PID2019-103886RB-I00) as well as the H2020 Environment (grant nos. 871115 and 101008004) and the H2020 Excellent Science (grant no. 778349) programs.

This paper was edited by Vassilis Amiridis and reviewed by four anonymous referees.

Ancellet, G. M., Kavaya, M. J., Menzies, R. T., and Brothers, A. M.: Lidar telescope overlap function and effects of misalignment for unstable resonator transmitter and coherent receiver, Appl. Optics, 25, 2886–2890, https://doi.org/10.1364/AO.25.002886, 1986.

Ansmann, A., Wandinger, U., Riebesell, M., Weitkamp, C., and Michaelis, W.: Independent measurement of extinction and backscatter profiles in cirrus clouds by using a combined Raman elastic-backscatter lidar, Appl. Optics, 31, 7113–7131, https://doi.org/10.1364/AO.31.007113, 1992.

Bucholtz, A.: Rayleigh-scattering calculations for the terrestrial atmosphere, Appl. Optics, 34, 2765–2773, https://doi.org/10.1364/ao.34.002765, 1995.

Comeron, A., Sicard, M., Kumar, D., and Rocadenbosch, F.: Use of a field lens for improving the overlap function of a lidar system employing an optical fiber in the receiver assembly, Appl. Optics, 50, 5538–5544, https://doi.org/10.1364/AO.50.005538, 2011.

Comerón Tejero, A., Muñoz Porcar, C., Rodríguez Gómez, A. A., Sicard, M., Dios Otín, V. F., Gil Díaz, C., Oliveira, D. C. F. d. S., and Rocadenbosch Burillo, F.: Calibrated signals used for overlap retrievals, Version 1.0, CORA.Repositori de Dades de Recerca [data set], https://doi.org/10.34810/data704, 2023.

D'Amico, G., Amodeo, A., Mattis, I., Freudenthaler, V., and Pappalardo, G.: EARLINET Single Calculus Chain – technical – Part 1: Pre-processing of raw lidar data, Atmos. Meas. Tech., 9, 491–507, https://doi.org/10.5194/amt-9-491-2016, 2016.

Dho, S. W., Park, Y. J., and Kong, H. J.: Experimental determination of a geometric form factor in a lidar equation for an inhomogeneous atmosphere, Appl. Optics, 36, 6009–6010, https://doi.org/10.1364/AO.36.006009, 1997.

Di Paolantonio, M., Dionisi, D., and Liberti, G. L.: A semi-automated procedure for the emitter–receiver geometry characterization of motor-controlled lidars, Atmos. Meas. Tech., 15, 1217–1231, https://doi.org/10.5194/amt-15-1217-2022, 2022.

Engelmann, R., Kanitz, T., Baars, H., Heese, B., Althausen, D., Skupin, A., Wandinger, U., Komppula, M., Stachlewska, I. S., Amiridis, V., Marinou, E., Mattis, I., Linné, H., and Ansmann, A.: The automated multiwavelength Raman polarization and water-vapor lidar Polly^{XT}: the neXT generation, Atmos. Meas. Tech., 9, 1767–1784, https://doi.org/10.5194/amt-9-1767-2016,
2016.

Gimmestad, G. G. and Roberts, D. W.: Teaching lidar inversions, in: Proceedings of the 25th International Laser Radar Conference, St. Petersburg, Russia, 5–9 July 2010, 190–191, ISBN: 978-1-61782-614-6, 2010.

Guerrero-Rascado, J. L., Costa, M. J., Bortoli, D., Silva, A. M., Lyamani, H., and Alados-Arboledas, L.: Infrared lidar overlap function: an experimental determination, Opt. Express, 18, 20350–20369, https://doi.org/10.1364/OE.18.020350, 2010.

Halldórsson, T. and Langerholc, J.: Geometrical form factors for the lidar function, Appl. Optics, 17, 240–244, https://doi.org/10.1364/ao.17.000240, 1978.

Hu, S., Wang, X., Wu, Y., Li, C., and Hu, H.: Geometrical form factor determination with Raman backscattering signals, Opt. Lett., 30, 1879–1881, https://doi.org/10.1364/OL.30.001879, 2005.

Klett, J. D.: Lidar inversion with variable backscatter/extinction ratios, Appl. Optics, 24, 1638–1643, https://doi.org/10.1364/AO.24.001638, 1985.

Kokkalis, P.: Using paraxial approximation to describe the optical setup of a typical EARLINET lidar system, Atmos. Meas. Tech., 10, 3103–3115, https://doi.org/10.5194/amt-10-3103-2017, 2017.

Kumar, D. and Rocadenbosch, F.: Determination of the overlap factor and its enhancement for medium-size tropospheric lidar systems: a ray-tracing approach, J. Appl. Remote Sens., 7, 1–15, https://doi.org/10.1117/1.JRS.7.073591, 2013.

Kumar, D., Rocadenbosch, F., Sicard, M., Comeron, A., Muñoz, C., Lange, D., Tomás, S., and Gregorio, E.: Six-channel polychromator design and implementation for the UPC elastic/Raman lidar, in: Proceedings of the SPIE, Lidar Technologies, Techniques, and Measurements for Atmospheric Remote Sensing VII, Prague, Czech Republic, 19–22 September 2011, Vol, 8182, https://doi.org/10.1117/12.896305, 2011.

Kuze, H., Kinjo, H., Sakurada, Y., and Takeuchi, N.: Field-of-view dependence of lidar signals by use of Newtonian and Cassegrainian telescopes, Appl. Optics, 37, 3128–3132, https://doi.org/10.1364/AO.37.003128, 1998.

Lefrère, J.: Etude par sondage laser de la basse atmosphere, Thèse de 3ème cycle, University of Paris, 1982.

Li, J., Li, C., Zhao, Y., Li, J., and Chu, Y.: Geometrical constraint experimental determination of Raman lidar overlap profile, Appl. Optics, 55, 4924–4928, https://doi.org/10.1364/ao.55.004924, 2016.

Mahagammulla Gamage, S., Sica, R. J., Martucci, G., and Haefele, A.: Retrieval of temperature from a multiple channel pure rotational Raman backscatter lidar using an optimal estimation method, Atmos. Meas. Tech., 12, 5801–5816, https://doi.org/10.5194/amt-12-5801-2019, 2019.

Mathews, J. and Walker, R. L.: Mathematical Methods of Physics, 2nd edn., The Benjamin/Cummings Publishing Company, ISBN: 8053-7002-1, 1970.

Papoulis, A. and Pillai, S. U.: Probability, random variables, and stochastic processes, 4th edn., McGraw-Hill, ISBN 0-07-112256-7, 2002.

Povey, A. C., Grainger, R. G., Peters, D. M., Agnew, J. L., and Rees, D.: Estimation of a lidar's overlap function and its calibration by nonlinear regression, Appl. Optics, 51, 5130–5143, https://doi.org/10.1364/AO.51.005130, 2012.

Sasano, Y., Shimizu, H., Takeuchi, N., and Okuda, M.: Geometrical form factor in the laser radar equation: an experimental determination, Appl. Optics, 18, 3908–3910, https://doi.org/10.1364/ao.18.003908, 1979.

Sasano, Y., Browell, E. V., and Ismail, S.: Error caused by using a constant extinction/backscattering ratio in the lidar solution, Appl. Optics, 24, 3929–3932, https://doi.org/10.1364/ao.24.003929, 1985.

Sassen, K. and Dodd, G. C.: Lidar crossover function and misalignment effects, Appl. Optics, 21, 3162–3165, https://doi.org/10.1364/AO.21.003162, 1982.

Stelmaszczyk, K., Dell'Aglio, M., Chudzyński, S., Stacewicz, T., and Wöste, L.: Analytical function for lidar geometrical compression form-factor calculations, Appl. Optics, 44, 1323–1331, https://doi.org/10.1364/AO.44.001323, 2005.

Tomine, K., Hirayama, C., Michimoto, K., and Takeuchi, N.: Experimental determination of the crossover function in the laser radar equation for days with a light mist, Appl. Optics, 28, 2194–2195, https://doi.org/10.1364/AO.28.002194, 1989.

Vande Hey, J., Coupland, J., Foo, M. H., Richards, J., and Sandford, A.: Determination of overlap in lidar systems, Appl. Optics, 50, 5791–5797, https://doi.org/10.1364/AO.50.005791, 2011.

Wandinger, U. and Ansmann, A.: Experimental determination of the lidar overlap profile with Raman lidar, Appl. Optics, 41, 511–514, https://doi.org/10.1364/AO.41.000511, 2002.

Zenteno-Hernández, J. A., Comerón, A., Rodríguez-Gómez, A., Muñoz-Porcar, C., D'amico, G., and Sicard, M.: A comparative analysis of aerosol optical coefficients and their associated errors retrieved from pure-rotational and vibro-rotational raman lidar signals, Sensors-Basel, 21, 1–21, https://doi.org/10.3390/S21041277, 2021.

- Abstract
- Introduction
- Overlap retrieval
- Influence of the lidar ratio
- Example results
- Conclusions
- Appendix A: Derivation of the explicit form of the overlap function
- Appendix B: Systematic error bounds
- Data availability
- Author contributions
- Competing interests
- Disclaimer
- Acknowledgements
- Financial support
- Review statement
- References

_{2}and/or O

_{2}channels used for independent measurements of aerosol backscatter and extinction coefficients. The formula requires only the measured, range-corrected elastic and the corresponding Raman signals, plus an assumed lidar ratio. We assess the influence of the lidar ratio error in the overlap function retrieval and present retrieval examples.

- Abstract
- Introduction
- Overlap retrieval
- Influence of the lidar ratio
- Example results
- Conclusions
- Appendix A: Derivation of the explicit form of the overlap function
- Appendix B: Systematic error bounds
- Data availability
- Author contributions
- Competing interests
- Disclaimer
- Acknowledgements
- Financial support
- Review statement
- References