In the following text we use the notation and formulation in . The observables, log⁡10(1/τ) and height, are represented as di and hi respectively and the corresponding unobserved true values as ξi and ηi respectively, where the index i represents the ith meteor detection. For consistency we also assume that log⁡10(1/τ) is presented in abscissa, and height is in the ordinate in the respective scattered plot (as shown in Fig. ). Suppose we are assuming a linear relation in variables ξi and ηi as 9ηi=α+βξi,i=1,2,…,N. Due to measurement errors and various geophysical processes, the true values ξi and ηi will be subject to random errors, and hence the observable di and hi will have scatter around the linear model in Eq. (): 10di=ξi+δi and 11hi=ηi+εi=α+βξi+εi, where δi and εi are errors in the measured log⁡10(1/τ) and height respectively and are assumed to be mutually uncorrelated, have zero mean and be independent of the suffix i. This implies that the measurement error variances, sδ and sε, are constant with respect to the suffix i. Classical regression analysis or ordinary least-squares (OLS) treats di as an independent variable without intrinsic errors (or, di=ξi) and then minimises the sum of squared residuals along the ordinate. The slope from the OLS method can be represented in terms of the covariance (Cov) and variance (Var) as e.g. 12βOLSd=Cov(di,hi)Var(di)≡rshsd, where βOLSd is the OLS slope estimated by considering log⁡10(1/τ) as an independent variable, and r is the Pearson product-moment correlation coefficient between d and h. Likewise, by reversing the arguments above, it is trivial to show that the reciprocal value of the OLS slope estimate with height as an independent variable is e.g. 13βOLSh=Var(hi)Cov(di,hi)≡1rshsd. To see the effect of errors in the independent variable on the OLS slope estimate, e.g. for βOLSd, Eqs. () and () are used in Eq. (): 14βOLSd=Cov(ξi+δi,α+βξi+εi)Var(ξi+δi). Since δi, εi and ξi are mutually independent, Eq. () simplifies to 15βOLSd=βCov(ξi,ξi)Var(ξi+δi)=11+Var(δi)Var(ξi)β=ζβ, where ζ is known as the attenuation or regression dilution bias. Since variances are always positive by definition, Eq. () shows that in the presence of measurement error in the so-called independent variable (in abscissa), the OLS slope estimate (βOLSd) will always be smaller than the unbiased slope β. Likewise, βOLSh is greater than β if there is error in the measured height (specific example presented in Fig. a). By substituting di=ξi+δi, Eq. () can be rearranged as 16βOLSd=(1-Var(δi)Var(di))β. Equation () is a well known identity in statistical literature e.g., that was re-derived by in Eq. () in the context of SCT correction. Equation () reveals that an absolute knowledge of error variances, sδ ( or sε), is required to obtain the bias-corrected slope (β) if we choose OLS fitting for the slope estimate. A common practice is to calibrate the biased slope with optical or satellite data by arbitrarily choosing a value of sδ or sε e.g.. In the remaining part of this section, we demonstrate how to obtain an average value of the error variances in these data following a revised calibration procedure.

For each 24 h of the data set, we performed two OLS fittings to estimate βOLSd and βOLSh by using Eqs. () and (). The corresponding biased temperatures, TMRd and TMRh respectively, are estimated using Eq. (). Experimental values of the parameters sδ and sε can be obtained by comparing these estimated biased temperatures with the co-located lidar temperatures (Tlidar) as the reference values. Using Eqs. () and (), and noting that the slope is proportional to the estimated temperatures from Eq. (), we obtain 17sδ≈(Tlidar-TMRdTlidar)sdandsε≈(TMRh-TlidarTMRh)sh, where TMRd and TMRh are MR temperatures estimated using OLS fitting with log⁡10(1/τ) and height as an independent variable respectively. Furthermore, if the measurements are normalised with the mean and standard deviation (SD) as 18di′=di-mean(di)sdandhi′=hi-mean(hi)sh, then Var(di′)=Var(hi′)=1, and the OLS estimate of the ratio of the measurement error variance is (from Eqs.  and ) 19λeffOLS=sε′sδ′≈TMRh-TlidarTlidar-TMRd⋅TlidarTMRh, where the error variances, sε′ and sδ′, are in the dimension-free system defined by Eq. (). In essence, λeffOLS is a measure of all sources of errors in the normalised heights and decay times that cause the real data to deviate from the idealised physical model of Eq. (), thereby producing a typical scatter as seen in Fig. .

λeffOLS, sδ and sε were estimated by Eqs. () and () using 24 h of MR data and co-located lidar temperatures at 88, 89 and 90 km for the dates for which lidar data were available. The biasing effect on the OLS estimate of MR temperatures with log⁡10(1/τ) and height as an independent variable respectively are presented in Fig.  (green and blue lines and histogram). As expected from Eq. () or Eq. (), a mean offset of -75 and +335 K occurs depending on whether log⁡10(1/τ) or height respectively is considered as an independent variable. In practice, the magnitude of these biases is related to the total errors in heights and log⁡10(1/τ) from Eq. (), which was not taken into account by the OLS regression model in Eqs. () and (). Furthermore, the relative density contour lines for each data point is obtained by counting the number of detections in a circle of unit area in the normalised height–log⁡10(1/τ) plane relative to the number of detections in unit area at the peak meteor region (Fig. b). In addition, all the error estimates from the SCT method are obtained at contour levels 0, 0.2 and 0.4, and the results are presented in Table . Several key features of these data are reflected in Table . Despite the data transformation via Eq. (), the average error variances for the height data are more than those for the log⁡10(1/τ) data in these coordinates. This asymmetry in the error variances implies that the bivariate distribution is slightly skewed away from a perfectly normal distribution along the y direction. However, this effect of asymmetric error variances is less pronounced near the core of this distribution. This implies that the parameter λeffOLS is closer to 1 near the core of this distribution and further away from 1 near the tail of the distribution. The data near the outer contour area are subject to larger parameter estimation error due to observational limitation. For example, the zenith-angle-dependent error in height can easily skew the distribution in this direction. On the other hand, natural geophysical variability in these data contributes significantly at all altitudes. As seen in Table , the average error in height for contour levels 0, 0.2 and 0.4 are 5.0, 3.0 and 2.2 km respectively. This error is significantly higher than what is expected from purely parameter estimation error (for zenith angle less than 50∘, the error in height is 1 km or less), indicating that geophysical variability dominates the total error variance in these data at all altitudes. The average error variances estimated at the contour level 0.2 (Table ) correlate very well with the values reported in and . estimated △(height) =3.25 km using a numerical model for a pulse length equivalent to 2 km and a meteor at an altitude of 90 km and zenith angle 50∘. Likewise, their estimation of △log⁡10(1/τ)=0.14 was based on simulation studies for meteors below 95 km and confirmed that “decay times' variability” arise due to 27 % variability in Kamb and 8 % variability in temperatures over the meteor region. The argument for choosing 95 km as the maximum height is that above this altitude, meteor decay rates are substantially affected by processes other than ambipolar diffusion. p. 5 applied similar data rejection criteria and found out that △log⁡10(1/τ)=0.14 is required to calibrate the slope if log⁡10(1/τ) is used as an independent variable in the OLS regression model. Likewise, performed the comparison between the parameter estimation error and the geophysical variability for estimating the mean wind field in the middle atmosphere and found that the geophysical variability dominated at all heights.

It is worth noting that instead of directly using the individual observation between biased MR temperatures and lidar measurements from Eq. (), we have used the statistical mean of differences for calibration. This is because lidar data are not available for all days during the 6 months of data used in this work. Moreover, both MR and lidar data have their own intrinsic errors and technical differences in the observation time and volume of sky. MR temperatures are daily averages over 24 h of observation, whereas lidar data are just the nightly mean profile. The lidar probes a small volume limited to the diameter of the lidar beam, while the radar illuminates a large part of the sky. For a single observation, the lidar may see the phase structure of large-scale gravity waves, while the MR averages over the gravity wave structure due to different spatial resolutions. As a result, the radar averages over gravity waves with horizontal wavelengths smaller than few hundred kilometres. On the other hand, the lidar may resolve these gravity waves if the runtime is shorter than the period of these waves. As gravity wave amplitudes can be up to 10–15 K at these altitudes , we cannot expect perfect agreement between radar and lidar temperature due to geophysical variation which shows up differently in the two data sets as a result of the different observational volumes.

While such calibration routine may prevent large offsets in the estimated temperatures, the day-to-day variation in these error variances due to natural geophysical processes will persistently introduce artefacts in the estimated temperatures. Moreover, due to the continually changing atmospheric dynamic, these calibration parameters need to be updated at time intervals. This in turn requires availability of optical or satellite data throughout the year for the given location. In all generality, it is desirable to avoid any kind of calibration process and instead formulate an independent estimate of temperatures using MR data alone. As an alternative to the OLS method, errors-in-variables (EIV) regression analysis provides a way to incorporate the error variances in both height and log⁡10(1/τ) data, thereby reducing the biasing effect in the estimated slope parameter.

For fitting a straight line model, such as 20y(x)=a+bx to a set of N data points (xi, yi) measured with errors, the corresponding χ2 merit function is p. 660 21χ2=∑i=1N(yi-a-bxi)2σyi2+b2σxi2, where σyi and σxi are the standard deviation of the ith data point, and the weighted sum in the denominator of Eq. () can be interpreted as the weighted error of the ith data point. The regression coefficients, a and b, can be found by minimising the merit function with respect to these coefficients following any suitable numerical root-finding routine. However, under the assumption of symmetric error variances, it is possible to derive an analytic solution for the regression coefficients. This solution, when the data are appropriately normalised, leads to a slope estimate which is both scale-invariant and symmetric with respect to the data.

Application of Eq. () to physics data requires that all measured variables are dimensionally consistent so that χ2 is dimension-free. Moreover, the analysis in this section requires that the measurements are presented in an appropriate dimension-free system. This facilitates the direct comparison between different parameters, such as the measured variables or the associated error variances. By applying the coordinate transformation introduced in Eqs. () to (), we therefore intend to solve the simplified bivariate linear system of equations, 22ηi′=βsdshξi′≡βWξi′, where ηi′, ξi′ and βW are dimension-free. For the specific choice of normalisation by Eq. (), the intercept (αW) is always zero in the transformed coordinate system. The merit function in Eq. () can be further simplified by invoking a homoscedastic standard weighting model . This error model assumes that the error variances are independent of data point, thereby simplifying the merit function as 23χ2(βW,αW=0)≡∑i=1N(hi′-βWdi′)2sδ′(λ+βW2), where sδ′ and sε′ are constant error variances of the measured log⁡10(1/τ) and heights respectively in the normalised (or, dimension-free) coordinate system, and λ is the ratio 24λ=sε′sδ′. The χ2 minimisation of Eq. () with respect to βW leads to the analytic expression for the EIV slope parameter in terms of the variances (sd′, sh′) and covariances (sh′d′) of the measured variables, 25βW=sh′-λsd′+(sh′-λsd′)2+4λsh′d′22sh′d′ or 26βW≡1-λ+(1-λ)2+4λsh′d′22sh′d′. since sd′=sh′=1 from Eq. (). And the covariance (sh′d′) is computed using the standard definition, 27sh′d′=1N∑i=1N(hi′di′),forlargeN. Equation () can be solved if a prior knowledge of λ is available, which in turn requires a precise estimate of all sources of errors in the measured data. In the more practical case for unknown λ, we need to initiate a good starting estimate. Using the calibration procedure described by Eqs. () and (), the mean values of sε′ and sδ′ are found to be 0.62±0.04 and 0.37±0.06 respectively (without any contour selection: Table ). Since the EIV estimate of the slope requires only the ratio between sε′ and sδ′, a good choice of this starting value is λ=1. Furthermore, for λ=sε′=sδ′=1, there is a simple geometric interpretation of the merit function in Eq. (). This solution corresponds to minimising the Euclidean or orthogonal distance between the fitted line and the measured data. The residual function to be minimised with respect to the regression coefficients is 28χ2=∑i=1N(hi′-βWdi′)21+βW2≡∑i=1N(hi′-di′)22 since βW=1 when λ=1 from Eq. (). Following Eq. (), we therefore have our first estimate of β in the scattered plot of log⁡10(1/τ) and heights, 29β=shsd. Equation () is commonly referred to as a reduced major axis (RMA) solution in statistics literature . In practice, this is just the geometric mean (GM) of the two OLS estimates, βOLSd and βOLSh, as can be seen by combining Eqs. () and (): 30βGM=βOLSdβOLSh≡shsd. The GM solution in Eq. () has the unique feature that this is the only case of EIV estimate which is both scale-invariant and symmetric in the variables e.g.. While these properties do not necessarily imply that the GM solution is the correct solution (discussed below), this first estimate reflects the nature of the biasing effect in the estimated slope in relation to the data selection process. A specific example of GM fitting is presented in Fig. a. The standard error in the OLS and GM slope estimate reported in Fig. a is from . Due to the relatively low detections with Sodankylä radar (Fig. c), the 2σ error in the estimated temperature using the GM solution is found to be significantly higher (13 K on average at contour level 0). This 13 K of noise level in the temperature can be reduced by a factor of 3 or 5 if a 3 or 5 d running mean of temperature is estimated with this radar. However, to test the robustness of the proposed method, this paper has estimated the daily averaged temperatures.

The systematic offset between MR temperature and co-located lidar temperature as a result of using Eq. () is reflected in Fig. a (red curve) and b (red histogram). Each of these temperatures are then compared with lidar temperatures at 88, 89 and 90 km for the dates when lidar data are available. The intrinsic noise in the lidar temperature is about 5–10 K , which implies no temperature gradient is observed between 88 and 91 km in these data. Figure b (red histogram) reveals that the MR temperatures are overestimated by a mean value of +58 K for the case of the GM solution.

When compared to the temperature gradient model derived from optical, satellite and rocket climatology e.g., it can be easily argued that our MSIS-derived gradient model (Fig. a) is more negative than expected. If these values are shifted by a constant positive offset of +1 Kkm-1, the absolute value of the estimated temperatures will increase by 10 K . This will further increase the offset between lidar and MR temperature, thereby shifting the histogram (in red) in Fig. b further to the right.

On the other hand, lidar temperatures are usually obtained during the night-time, which can lead to a systematic offset due to day–night differences or tidal variations. As discussed by , the day–night temperature difference at these altitudes is of the order of 3–4 K. This is significantly less than the standard errors in these temperatures, which is on average 6 K at contour level 0. Hence we expect the day–night difference in Sodankylä MR temperatures to be insignificant during the winter period. Moreover, any attempt to estimate MR temperatures using only night-time data has the adverse effect of reducing the accuracy in the estimated temperatures due to data loss. While no specific studies of tidal variation have been made for this location, the data from other sites e.g. show that the temperature variation due to tidal activity is typically less than 10 K. We can therefore rule out the possibility of an offset in the MSIS gradient model or tidal effects as the primary cause for the +58 K offset seen in Fig. b (red histogram).

emphasised that the biasing in GM solution is conditional upon the value of the correlation coefficient (r) between the variables, while demonstrated that this solution will be an overestimate in the case of low r values. For the data set used in this work, we found that the correlation between log⁡10(1/τ) and height is typically 0.50±0.05, thereby indicating the presence of significant natural variation in the measurements. Furthermore, used error modelling to conclude that r must be more than 0.6 for the GM solution to be acceptable. In fact, we have observed that if we restrict our data selection process by excluding all data beyond the density contour 0.2 (Fig. b), this increases the r to be typically around 0.66±0.06, with the consequence of reduced biasing in the GM solution. Such a contour selection process essentially removes the erroneous data at higher and lower altitudes from the tail of the distribution, whereby the assumption of equal error variances (i.e. sε′≈sδ′) is achieved in the normalised coordinates (Table ). In other words, the validity of GM solution is conditional upon how close the parameter λ is to 1 for a given data selection process.