Articles | Volume 19, issue 22
https://doi.org/10.5194/amt-19-7505-2026
https://doi.org/10.5194/amt-19-7505-2026
Research article
 | 
17 Nov 2026
Research article |  | 17 Nov 2026

On the impact of a misestimation of the lidar ratio in the retrieval of the overlap function of an elastic/Raman lidar and a two-step explicit formulation of the overlap function retrieval

Adolfo Comerón, Constantino Muñoz-Porcar, Alejandro Rodríguez-Gómez, Michaël Sicard, Federico Dios, Cristina Gil-Díaz, Daniel Camilo Fortunato dos Santos Oliveira, and Francesc Rocadenbosch
Abstract

In a relatively recent paper by the authors, an explicit (i.e. non iterative) formula was provided for recovering the overlap factor in an aerosol lidar equipped with a Raman channel of wavelength close to the elastic one, i.e. either a O2 or a N2 vibro-rotational Raman channel, or a purely rotational Raman channel collecting lines of the N2 and O2 rotational Raman spectra, to retrieve the aerosol extinction coefficient. One of the advantages of that formula is that it allows for the estimation of the impact of an erroneous lidar ratio in the retrieved overlap function. In the cited paper, the equation that relates the retrieved overlap function to the true one and the error on the assumed lidar ratio was given with almost no proof, on the grounds that its derivation was “boring and cumbersome, but otherwise straightforward”. After having discussed that equation with some interested researchers, it turns out that its derivation is not perhaps that straightforward. Therefore, we provide here its derivation. As a subproduct of that derivation, we propose a streamlined, yet equivalent to the one already published, two-step explicit formulation to retrieve the overlap function.

Share
1 Introduction

Comerón et al. (2023) derived an explicit formulation for the retrieval of the overlap function in an elastic and Raman aerosol lidar (Comerón et al., 2023), which is equivalent to the iterative method derived by Wandinger and Ansmann (2002). The accuracy of the retrieval in these two methods is affected by how good is the estimation of the lidar ratio of the aerosol in presence. The explicit formulation permits to assess clearly the impact of an erroneous assumption of the lidar ratio on the retrieved overlap function. The retrieved overlap function, O′(R) , where R is the range from the lidar, relates to the “true” overlap function, O(R), through (Eq. 18 of Comerón et al., 2023)

(1) O ′ ( R ) = O ( R ) exp - 2 ∫ R R m Δ S a 0 ( x ) β a 0 ( x ) d x ,

where βa0(R) is the aerosol backscatter coefficient at range R, Rm is a reference range where βa0(Rm)=0, and ΔSa0(R) is the difference at range R between the assumed lidar ratio at the emitted wavelength, S′a0(R), and the “true” lidar ratio at the same wavelength Sa0(R), i. e.

(2) Δ S a 0 ( R ) = S ′ a 0 ( R ) - S a 0 ( R ) .

From Eq. (1) it is clear that O′(R)=O(R) if the aerosol lidar ratio is estimated correctly, i.e.,ΔSa0=0, or if there is no aerosol, i.e. βa0(R)=0, in which case there is no aerosol lidar ratio to be considered.

In Sect. 2, we derive Eq. (1). In Sect. 3 we propose a streamlined, explicit method to retrieve the overlap function, alternative to that derived in Comerón et al. (2023). Section 4 contains the conclusions.

2 Derivation of the relationship between the true overlap function and the estimated one

We start, as indicated in Comerón et al. (2023), by noting that, under the assumption that the overlap function is the same for the elastic and the Raman signals (Wandinger and Ansmann, 2002), the range-corrected elastic and Raman signals can be expressed, respectively as

(3a)X(R)=AO(R)[βm0(R)+βa0(R)]×exp-2∫0R[αa0(x)+Sm0βm0(x)]dx,(3b)XR(R)=BO(R)βmR(R)×exp{-∫0R[αa0(x)+αaR(x)+Sm0βm0(x)+SmRβmR(x)]dx},

where A and B are instrument constants, αa0(R) and αaR(R) are the extinction coefficients at the excitation and Raman wavelengths, respectively, βm0(R) and βmR(R) are the molecular backscatter coefficients at the excitation and Raman wavelengths, respectively, and Sm0 and SmR are the molecular lidar ratios at the excitation and Raman wavelengths, respectively (D'Amico et al., 2016; Bucholtz, 1995). The assumption of equal overlap function in Eqs. (3), although usually satisfied at medium and long ranges for a well-aligned system, may fail at short ranges for a number of reasons depending on the lidar optical and mechanical architecture, for example, varying response of the narrow-band filters in the elastic and Raman receivers due to varying angles of incidence of incoming radiation affecting differently to either filter, imperfect mechanical alignment of elements (detectors, irises, lenses…) in the receiving paths, which can also depend on environmental conditions like temperature, etc. In fact, the failure of the identical overlap function assumption at short ranges is acknowledged in Comerón et al. (2023) and made evident by the unrealistic aerosol backscatter coefficient behavior (sudden drop) at short ranges visible in Fig. 2 of Comerón et al. (2023). A formal treatment of errors in the retrieval of non-equal overlap functions might be possible, but it is beyond the scope of the present paper, in which we keep the assumption of the same overlap function for the elastic and the Raman channels.

It could also be argued that the molecular backscatter terms βm0(R) and βmR(R) appearing outside the exponentials in Eq. (3) should rather be effective backscatter coefficients, since they may be affected by the widths of the narrow-band filters in the elastic and Raman channels, respectively. This effect can be absorbed in the instrument constants; however, these effective backscatter coefficients, and their ratios appearing in subsequent equations of this paper, can still be subject to temperature variations because the intensity of individual lines in the Raman spectrum depend on temperature and, if the corresponding receiver filter is very narrow, the balance of power passing through it will change with the temperature of the atmospheric volume the backscattered power comes from Adam (2009). Temperature effects can nevertheless be neglected for vibro-rotational N2 or O2 Raman channels with receiving filters passing most of the spectrum lines. Likewise, the effect of the N2 and O2 Raman rotational spectrum lines that can pass through the filter in the elastic channel is also negligible (Adam, 2012). As for a purely rotational Raman channel, the corresponding interference filter can be tailored for that channel to exhibit minimal temperature dependence (Veselovskii et al., 2015; Zenteno-Hernández et al., 2021). We therefore dismiss in our analysis the temperature dependence of βm0(R), βmR(R) and their ratio.

We turn now to the explicit expression of the overlap function given in Eq. (15) of Comerón et al. (2023), where, by the way, we note that two typos slipped into. One affects the argument of the fourth exponential function, which should be raised to −1. The other typo is found in the initial factor in that equation, βm0(Rm)XR(R)βm(R)XR(Rm), which should be βm0(Rm)XR(R)βm0(R)XR(Rm), Rm being an aerosol-free, sufficiently far reference range, where the aerosol backscatter coefficient can be assumed to be 0 and the overlap function has stabilized to a constant value that we make arbitrarily 1, i.e. O(Rm)=1. We repeat here that expression for convenience and, because it uses an aerosol lidar ratio at the excitation wavelength that might not be the true one, we denote that lidar ratio as S′a0(R) (see Eq. 2) and the resulting overlap function as O′(R):

(4) O ′ ( R ) = β m 0 ( R m ) X R ( R ) β m 0 ( R ) X R ( R m ) × exp 2 ∫ R R m S ′ a 0 ( x ) - S m 0 β m 0 ( x ) d x × exp ∫ R R m [ S m 0 β m 0 ( x ) - S m R β m R ( x ) ] d x × exp ∫ R R m [ α a 0 ( x ) - α a R ( x ) ] d x × exp - 2 X R ( R m ) X ( R m ) ∫ R R m S ′ a 0 ( x ) β m 0 ( x ) X ( x ) X R ( x ) × exp - ∫ x R m S m 0 β m 0 ( x ′ ) - S m R β m R ( x ′ ) d x ′ × exp - ∫ x R m α a 0 ( x ′ ) - α a R ( x ′ ) d x ′ d x .

Let us focus on the fourth exponential in Eq. (4)

(5) f ( R ) = exp - 2 X R ( R m ) X ( R m ) ∫ R R m S ′ a 0 ( x ) β m 0 ( x ) X ( x ) X R ( x ) × exp - ∫ x R m S m 0 β m 0 ( x ′ ) - S m R β m R ( x ′ ) d x ′ × exp - ∫ x R m α a 0 ( x ′ ) - α a R ( x ′ ) d x ′ d x ,

where the constant term XR(Rm)X(Rm) can pass inside the integration operator:

(6) f ( R ) = exp - 2 ∫ R R m S ′ a 0 ( x ) β m 0 ( x ) X ( x ) X R ( R m ) X R ( x ) X ( R m ) × exp - ∫ x R m S m 0 β m 0 ( x ′ ) - S m R β m R ( x ′ ) d x ′ × exp - ∫ x R m α a 0 ( x ′ ) - α a R ( x ′ ) d x ′ d x .

But, using Eqs. (3), the ratio X(x)XR(Rm)XR(x)X(Rm) can be written as

(7) X ( x ) X R ( R m ) X R ( x ) X ( R m ) = [ β m 0 ( x ) + β a 0 ( x ) ] β m R ( R m ) β m 0 ( R m ) β m R ( x ) × exp ∫ x R m α a 0 ( x ′ ) + S m 0 β m 0 ( x ′ ) d x ′ × exp - ∫ x R m α a R ( x ′ ) + S m R β m R ( x ′ ) d x ′ ,

and taking into account that the ratio βmR(R)βm0(R) is independent of the range, hence βmR(Rm)βm0(x)βm0(Rm)βmR(x)=1, one arrives at

(8) X ( x ) X R ( R m ) X R ( x ) X ( R m ) = β m 0 ( x ) + β a 0 ( x ) β m 0 ( x ) × exp ∫ x R m α a 0 ( x ′ ) + S m 0 β m 0 ( x ′ ) d x ′ × exp - ∫ x R m α a R ( x ′ ) + S m R β m R ( x ′ ) d x ′ .

By replacing Eq. (8) in Eq. (6), f(R) simplifies to

(9) f ( R ) = exp - 2 ∫ R R m S ′ a 0 ( x ) [ β m 0 ( x ) + β a 0 ( x ) ] d x ,

which, after reinstating in Eq. (4) and rearranging terms, yields

(10) O ′ ( R ) = β m 0 ( R m ) X R ( R ) β m 0 ( R ) X R ( R m ) × exp - 2 ∫ R R m S ′ a 0 ( x ) β a 0 ( x ) d x × exp - ∫ R R m S m 0 β m 0 ( x ) + S m R β m R ( x ) d x × exp ∫ R R m α a 0 ( x ) - α a R ( x ) d x .

Likewise, by substituting Eq. (3b) in the factor βm0(Rm)XR(R)βm0(R)XR(Rm) appearing in Eq. (10), and, taking into account again the independence with range of the ratio βmR(R)βm0(R), we obtain

(11) β m 0 ( R m ) X R ( R ) β m 0 ( R ) X R ( R m ) = O ( R ) exp { ∫ R R m [ α a 0 ( x ) + α a R ( x ) + S m 0 β m 0 ( x ) + S m R β m R ( x ) ] d x } .

Note that in Eq. (11) there is no assumption on the aerosol lidar ratio and, therefore, the overlap function appearing is the “true” one.

The substitution of Eq. (11) in Eq. (10) yields therefore

(12) O ′ ( R ) = O ( R ) exp { 2 ∫ R R m [ α a 0 ( x ) - S ′ a 0 ( x ) β a 0 ( x ) ] d x } ,

which, knowing that, by definition, αa0(x)=Sa0(x)βa0(x), and substituting, according to Eq. (2), S′a0(x)=Sa0(x)+ΔSa0(x), results finally in Eq. (1).

3 Alternative retrieval of the overlap function

In our way to deriving Eq. (1), we have found a way to compute explicitly (i.e. non iteratively) the overlap function, other than the daunting Eq. (4) (Eq. 15 of Comerón et al., 2023, corrected for typos), namely Eq. (10) of the present paper. All the variables intervening in it are found in Eq. (4), except the aerosol backscatter coefficient at the emitted wavelength βa0. But this coefficient can be retrieved, assuming, as we have done since the beginning, that the overlap functions of the elastic and the Raman channels are identical (Eqs. 3), as (Ansmann et al., 1992)

(13) β a 0 ( R ) = X ( R ) X R ( R m ) β m 0 ( R ) X ( R m ) X R ( R ) × exp - ∫ R R m [ α a 0 ( x ) + S m 0 β m 0 ( x ) ] d x exp - ∫ R R m [ α a R ( x ) + S m R β m R ( x ) ] d x - β m 0 ( R ) .

Substituting the result of Eq. (13) for βa0(x) in Eq. (10) yields the same result as Eq. (4).

4 Conclusion

We have shown the details of the derivation of the ratio between the retrieved overlap function – which needs the assumption of an aerosol lidar ratio that may in principle differ from the actual, unknown one – and the true one in terms of the difference between the assumed lidar ratio and the true one. Obviously, this difference is also unknown, but helps in detecting too high or too low assumptions and the expression clearly shows the interest of doing the retrieval of the overlap function in clear atmospheres, with βa0(R) as low as possible to minimize the error.

At the same time, we propose a two-step explicit formulation for the retrieval of the overlap function, easier to apply than that proposed in Comerón et al. (2023), but otherwise equivalent: 1st step: compute βa0(R) according to Eq. (13); 2nd step: compute the overlap function after Eq. (10).

We should stress that the formalisms discussed in this paper assume equal overlap functions for both the elastic and the Raman channel of the lidar. For a well-aligned system this is, to a good approximation, the case after a certain range. The range below which the assumption is no longer met can be estimated by identifying non-realistic behaviors of the retrieved aerosol backscatter coefficient by Eq. (13), which also assumes the equality of overlap functions. The formalism is nevertheless valid above this minimum range.

Data availability

The present paper does not use experimental data. However, it relies heavily in a previous paper (Comerón et al., 2023), and the interested reader may wish to verify that the two-step method to retrieve the overlap function (Eqs. 13 and 10) yields the same result as Eq. (4) by downloading the data available from https://doi.org/10.34810/data704 (Comerón Tejero et al., 2023).

Author contributions

AC developed the new formulation, wrote the text. CMP and ARG contributed with data analysis. The contributions of MS, FD, CGD, DCFDSO and FR have been invaluable in the inception, development and consolidation of our lidar system, besides the contribution in measurements and data analysis.

Competing interests

The contact author has declared that none of the authors has any competing interests.

Disclaimer

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. The authors bear the ultimate responsibility for providing appropriate place names. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.

Acknowledgements

The authors acknowledge the support of the Spanish Ministry for Science, Innovation and Universities to ACTRIS ERIC. They also thank the colleagues Juan Antonio Bravo-Aranda, Sol Fernández-Carvelo and Samuel Villalba-Torres from University of Granada for fruitful discussions.

Financial support

This research has been partially supported by the Spanish Ministry of Science, Innovation and Universities' Agencia Estatal de Investigación (MICIU/AEI/10.13039/501100011033) project SYRSAC (no. PID2023-149747NB-I00), by the European Union under Horizon Europe Grant Agreement 101086690 (REALISTIC project) and by the COST Action EARLICOST (CA24135), supported by COST (European Cooperation in Science and Technology).

Review statement

This paper was edited by Ulla Wandinger and reviewed by two anonymous referees.

References

Adam, M.: Notes on temperature-dependent lidar equations, J. Atmos. Ocean. Technol., 26, 1021–1039, https://doi.org/10.1175/2008JTECHA1206.1, 2009. 

Adam, M.: Notes on Rayleigh scattering in lidar signals, Appl. Optics, 51, 2135–2149, https://doi.org/10.1364/AO.51.002135, 2012. 

Ansmann, A., Wandinger, U., Riebesell, M., Weitkamp, C., and Michaelis, W.: Independent measurement of extinction and backscatter profiles in cirrus clouds by using a combined Raman elastic-backscatter lidar, Appl. Optics, 31, 7113–7131, https://doi.org/10.1364/AO.31.007113, 1992. 

Bucholtz, A.: Rayleigh-scattering calculations for the terrestrial atmosphere, Appl. Optics, 34, 2765–2773, https://doi.org/10.1364/ao.34.002765, 1995. 

Comerón, A., Muñoz-Porcar, C., Rodríguez-Gómez, A. A., Sicard, M., Dios Otín, V. F., Gil-Díaz, C., dos Santos Oliveira, D. C. F., and Rocadenbosch Burillo, F.: An explicit formulation for the retrieval of the overlap function in an elastic and Raman aerosol lidar, Atmos. Meas. Tech., 16, 3015–3025, https://doi.org/10.5194/amt-16-3015-2023, 2023. 

Comerón Tejero, A., Muñoz Porcar, C., Rodríguez Gómez, A., Sicard, M., Dios Otín, V. F., Gil Díaz, C., dos Santos Oliveira, D. C. F., and Rocadenbosch, F.: Calibrated signals used for overlap retrievals, V1, CORA.Repositori de Dades de Recerca [data set], https://doi.org/10.34810/DATA704, 2023. 

D'Amico, G., Amodeo, A., Mattis, I., Freudenthaler, V., and Pappalardo, G.: EARLINET Single Calculus Chain – technical – Part 1: Pre-processing of raw lidar data, Atmos. Meas. Tech., 9, 491–507, https://doi.org/10.5194/amt-9-491-2016, 2016. 

Veselovskii, I., Whiteman, D. N., Korenskiy, M., Suvorina, A., and Pérez-Ramírez, D.: Use of rotational Raman measurements in multiwavelength aerosol lidar for evaluation of particle backscattering and extinction, Atmos. Meas. Tech., 8, 4111–4122, https://doi.org/10.5194/amt-8-4111-2015, 2015. 

Wandinger, U. and Ansmann, A.: Experimental determination of the lidar overlap profile with Raman lidar, Appl. Optics, 41, 511, https://doi.org/10.1364/AO.41.000511, 2002. 

Zenteno-Hernández, J. A., Comerón, A., Rodríguez-Gómez, A., Muñoz-Porcar, C., D'Amico, G., and Sicard, M.: A comparative analysis of aerosol optical coefficients and their associated errors retrieved from pure-rotational and vibro-rotational Raman lidar signals, Sensors (Switzerland), 21, https://doi.org/10.3390/s21041277, 2021. 

Download
Short summary
The overlap function of an aerosol lidar is a key feature determining the minimum range from which the lidar can perform reliable measurements. This article develops the relationship between the retrieved overlap function, which requires an assumption on the relationship between the extinction and backscatter coefficients of the aerosols present, and the “true” overlap function, and proposes a simple, two-step, non-iterative formulation for the retrieval of the lidar overlap function.
Share
Altmetrics
Final-revised paper
Preprint