For linear polarization, Kdp is proportional to the integral of the raindrop size distribution and the real part of the difference of forward-scattering amplitudes at orthogonal polarizations. It is mathematically formulated as 1Kdp=0.18λπ∫0∞N(D)⋅ℜfhh0,D-fvv0,DdD(∘km-1), where λ is radar wavelength in millimeters, D is raindrop size in millimeters, N(D) is size spectrum in cubic meters per millimeter (m3 mm-1), and fhh,vv(0,D) is forward-scattering amplitudes at horizontal and vertical polarizations, respectively.

By considering the Rayleigh–Gans scattering from identical and horizontally oriented oblate spheroids, such as raindrops, the forward-scattering amplitudes are proportional to the inverse square of radar wavelength, i.e., fhh,vv(0,D)∝1/λ2, leading to the fact that Kdp is inversely proportional to radar wavelength, i.e., Kdp∝1/λ. Therefore, the values of Kdp at the X band are often larger than that at the S band by a factor of 3, indicating that X-band radar can provide better Kdp data than S-band radar when retrieving the rainfall rate. The conclusion is still valid even if the Mie effect is taken into account .

However, Kdp cannot be detected by polarimetric radar directly, whereas its integral Φdp is measurable. Hence, Kdp can be estimated as the range derivative of the profile of Φdp, i.e., Kdp=ΔΦdp2Δr, where r is the radar range in kilometers. An alternative approach to estimating Kdp is to apply a regression fit to the profile of Φdp, and the first-order polynomial is usually considered as the fitting function . Subsequently, if the Φdp measurements are equally spaced in range by Δr, Kdp is then estimated by 2Kdp=∑i=1nΦdpri6i-3(n+1)Δrn(n-1)(n+1)Δr2, where n is the number of gates. Equation () shows that the accuracy of Kdp estimates is determined by the number of gates (n) and the accuracy of Φdp. By assuming σ2(Φdp) is relatively stable for all gates along a ray and noting that Φdp(ri) is the only variable in Eq. (), σ2(Kdp) is formulated as 3σ2Kdp=3σ2ΦdpΔr2n(n-1)(n+1). In Eq. (), σ2Kdp is proportional to σ2Φdp, which is related to the spectrum width, cross-correlation coefficient, and the dwell time , and inversely proportional to n3. This method has been widely used in the existing radar system . The details of the regression-based estimation of Kdp are given in and Appendix .

Moreover, it is notable that the backscattering phase shift is not negligible at the X band; thus the total propagation phase shift (Ψdp) consists of Φdp and the backscattering differential phase, δco; i.e., Ψdp=Φdp+δco. The backscattering phase shift is often shown as a sudden jump over one or a few range gates in a monotonically increasing Ψdp profile of rain , with a value much larger than the standard deviation σΨdp. The presence of δco over a small number of consecutive gates can be eliminated by a simple filter .

The specific differential phase is a unique polarimetric variable in terms of statistical errors in the rain rate estimation, since it is the range derivative of the phase measurement Φdp. The errors in the calculation of the first derivative also need to be taken into account. In this study, we consider a Gaussian mixture as the data generation model, which plays an important role in the estimation of Kdp and σ2(Kdp).

The Gaussian mixture is a statistical model for data probability density estimation, assuming that the data points are generated by a mixture of a finite number of Gaussian distributions associated with their weights . Intuitively, it is used to model the multimodal data, with each Gaussian component corresponding to a subpopulation of the data. The mathematical formulation is given as 4f(z)=∑i=1mwiNz;μi,Σi, where m is the number of components in the Gaussian mixture, wi is a weight with ∑i=1mwi=1, and Nz;μi,Σi is the ith Gaussian distribution with mean μi and covariance Σi; i.e., Nz;μi,Σi=(2π)kΣi-1/2exp⁡-12(z-μi)TΣi-1(z-μi), where k is the data dimension.

It is prevalent to use an Expectation–Maximization (EM) algorithm to estimate the parameters, w, μ, and Σ, by constructing the lower bound of the log-likelihood based on Jensen's inequality . The EM algorithm is divided into two steps, namely, an expectation (E) step and a maximization (M) step. In the E step, a degree of membership toward to the jth cluster is calculated; i.e., 5Qji=py(i)=j|x(i);w,μ,Σ, where i is the ith data with a total number of n data points, and y is a latent variable that determines the corresponding cluster. Here, Q gives a tight lower bound for the log-likelihood, equivalent to maximizing the expectation. In the M step, the exact form of the lower bound based on Jensen's inequality is expressed as 6L(w,μ,Σ)=∑i∑jQjilog⁡exp⁡-12(xi-μ)TΣ-1(xi-μ)wj(2π)kΣQji. By maximizing the lower bound with respect to each parameter, wj, μj, and Σj are updated as 7wj=∑iQjin,8μj=∑iQjix(i)∑iQji,and9Σj=∑iQji(x(i)-μj)(x(i)-μj)T∑iQji,respectively. Notably, the M step increases the log-likelihood monotonically, if the covariance Σj is a positive-definite matrix. Finally, the E step and M step are iteratively operated until the log-likelihood converges to a value with the difference between two successive steps below a certain threshold. In addition, the EM algorithm requires a specification of the number of clusters, m, prior to the E and M steps, and an inappropriate choice of m may lead to meaningless values of the parameters. To tackle this problem, the Bayesian information criterion is often calculated to select the optimal m, while a Dirichlet process may also be used to model a prior probability to construct an infinite Gaussian mixture.

One of interpretations of the Gaussian mixture is to view each distribution as a cluster with a Gaussian probability density, while the individual data point is attributed to a specific cluster or a weight toward the cluster, regarded as unsupervised learning . The clustering procedures based on the Gaussian mixture model have been applied to the identification of storm structure , as well as the particle identification at S-band and X-band frequencies. Furthermore, the Gaussian mixture model can be extended to fit a set of unknown parameters in the prior probability of the Bayesian framework, forming a Bayesian–Gaussian mixture model . The prior is then multiplied with the known conditional probability of data given the parameters to be estimated, yielding the posterior probability with a new set of parameters. The expectation of the posterior is often used to retrieve the conditional mean of the new parameters based on least squares criteria.

For the regression problem, the characteristics of the Gaussian mixture imply that the direct modeling of a regression function is very difficult. Nevertheless, the joint probability of the measurements and the estimated parameters may be modeled as a Gaussian mixture, leading to a regression function derived from the joint density model. Due to the asymptotic consistency of a Gaussian mixture model, it is capable of estimating a general density function in Rn in any shape . Moreover, the speed of calculating unknown parameters within a Gaussian mixture linearly depends on the number of the training data points, and the computation of the outputs is independent of the size of the training data. Consequently, regression based on a Gaussian mixture can be achieved very rapidly, compared to Gaussian process regression that grows with the data size. In addition, the Gaussian mixture can also be used to solve the regression problem with multiple dimensions, and a subset of dimensions can be selected to handle the missing data .

The presence of clutter in the Ψdp measurements may severely affect the Kdp estimation, producing significantly large variations on the estimates. It is well known that the effect of clutter can be reduced by applying a spectrum filter to the time-series data e.g., . However, some residual clutter echoes are still shown on the radar measurements including Ψdp . Therefore, the clutter needs to be well handled in GMM, prior to the deviation of the regression model based on the joint PDF.

In LR, the clutter is often eliminated by some criteria based on Ψdp or ρhv. For instance, we use the thresholds of the local standard deviation of Ψdp less than 10∘ to classify valid points. Further, 10 consecutive range gates of valid points signify the beginning of a rain cell, and 5 consecutive gates of invalid points finish the associated rain cell. Overall, the thresholds give a fairly good performance on the MZZU radar; however, the clutter may be incorrectly identified in the regions of high reflectivity or for the echoes mixed by weather and clutter, which are often associated with large Ψdp variation.

In contrast, GMM adopts sophisticated procedures, as depicted in Fig. . It is clear that there are five stages in the data masking, beginning with the input of raw Ψdp and ending with masked data. At the first stage, the raw data are fitted to a Gaussian mixture initialized by the k-means clustering, while the covariance is set to be diagonal for simplicity. The clusters with no more than five points are promptly masked out, before they pass to the second stage. Stages two, three, and four of the process all involve the clusters. At the second stage, the clusters are validated according to two sets of thresholds with respect to mean reflectivity. For the MZZU radar, the ratio of the standard deviations, σ(Ψdp)/σ(r), less than 14.2∘ km-1 and σ(Ψdp) less than 4.1∘ (threshold 2) are used for ZH less than 41 dBZ. To reduce the misclassification in the hail regions, the thresholds are increased for higher ZH, resulting in σ(Ψdp)/σ(r)<47.9∘ km-1 and σ(Ψdp)<6.3∘ (threshold 1). Next, the entire Ψdp profile is divided into multiple rain cell segments by considering the gaps between two consecutive clusters. Similar to the first stage, the segments containing no more than five points are excluded from the output of masked data. Following this, the dominant one is determined for each segment by comparing the weight accumulations of weather and clutter clusters. For a clutter segment with mean height below 200 m, the clusters within the segment are reevaluated by thresholds of σ(Ψdp)/σ(r)<2.0∘ km-1 and σ(Ψdp)<0.8∘; on the other hand, the clusters in a weather segment are reexamined using σ(Ψdp)/σ(r)<34.7∘ km-1 and σ(Ψdp)<6.1∘. This step can efficiently identify the clutter-contaminated weather echoes, which are often associated with large variances. At the last stage, some isolated points along the azimuth are obscured in the final results.

Figure  illustrates two examples for data masking, including a convective case (Fig. a) and a stratiform case (Fig. b). The data points in the two cases show steadily increasing trends related to anisotropic media along the wave propagation path. However, between 1.3 and 15 km at an azimuth of 252∘ in the convective case (Fig. a), the data present significant fluctuations with the minimum value at about 0∘ but the maximum value at 180∘. Since the dynamic range of Ψdp is from 0 to 180∘ for the MZZU radar, the measurements near the ground are likely to be the clutter returns, verifying the results of data masking. After 15 km, the Ψdp points start from about 50∘ and go all the way up to 180∘. Notwithstanding this trend, the points sharply decrease to about 10∘ at about 40 km, indicating the occurrence of phase folding. The data masking can effectively detect the phase folding and provide valid masked data for deriving the joint PDF. On the other hand, the weather echoes are more frequently observed at 1∘ in azimuth in the stratiform case (Fig. b). By taking a closer look at the Ψdp data, we can discern that the points largely fluctuate between 40 and 80 km due to low signal-to-noise ratio. In LR, these points may be incorrectly discarded based on σ(Ψdp) thresholds, leading to some missing data in the stratiform regions. In contrast, the data masking accurately identifies weather echoes characterized by a number of vertically oriented density ellipses. The continuous and uniformly distributed regimes are consistent with the physical interpretation of stratiform precipitation. In addition, the data masking is also sensitive to sudden jumps at the beginning of the Ψdp data, which may be caused by δco.

In the previous section, it is shown that the Ψdp profile varies along the range r. It rises quickly for horizontally oriented anisotropic scatterers, and, conversely, it falls steadily for vertically oriented particles . The nonuniform beam filling is also an important source of errors for the Ψdp measurements .

To estimate the relationship between r and Ψdp, we consider r as an independent variable, denoted as x, and Ψdp as a dependent variable, denoted as y. If the minimization of the mean square error is required, the regression function is obtained by taking the average value of y at fixed x, equivalent to estimating the expected values of y conditioned on x; i.e., 10y‾(x)=E(y|x)=∫yp(y|x,β)dx, where β is a set of unknown variables – for example, β=(m,w,μ,Σ) for the mixture model. Since the Gaussian mixture can be used to model any shapes of probability density with a rapid speed, the (x, y) points are then assumed to follow a joint PDF of Gaussian mixture, as defined in Eq. (). Moreover, the properties of the multivariate Gaussian distribution in each cluster determine the Gaussianity of the marginal distribution of either variable and the conditional distribution of one variable given the other . Therefore, the conditional PDF of y given x is expressed as

11p(y|x,β)=∑i=1mwiy|xNy;μiy|x,Σiy|x,with12μiy|x=μiy+Σiyx(Σixx)-1(x-μix),13Σiy|x=Σiyy-Σiyx(Σixx)-1Σixy,14wiy|x=fi(x)f(x)=wiN(x;μix,Σixx)∑j=1mwjN(x;μjx,Σjxx), where wi, μi=(μix,μiy)T and Σi=ΣixxΣixyΣiyxΣiyy are obtained by the EM algorithm. In Eq. (), f(x) is the marginal PDF of x with the parameters identical to the mixture, and fi(x) is the weighted marginal PDF of each cluster; i.e., f(x)=∑i=1mfi(x). By substituting Eq. () into Eq. () and noting the linearity of the mathematical expectation, the expected value of y conditioned on x is then obtained as 15E(y|x)=∑i=1mfi(x)f(x)(aix+bi),with16ai=Σiyx(Σixx)-1,17bi=μiy-Σiyx(Σixx)-1μix, and the conditional variance is given as (see Appendix ) 18σ2(y|x)=∑i=1mwiy|xΣiy|x+μiy|x2-∑i=1mwiy|xμiy|x2.

Equations () and () play an important role in the joint PDF-based regression analysis, called the regression and skedastic functions . In Eq. (), it can be seen that the regression function in GMM consists of multiple linear kernels, which is similar to LR. However, the weighting function wiy|x is not determined by the local structure but the marginal PDF of global data x. The Gaussian mixture method is flexible to capture the data information, while it still retains a finite set of parameters. Moreover, Eq. () readily estimates the point-wise variances σ2(y|x) that characterize the random errors in the measurements. In contrast, Eq. () for the LR presents the relationship between the errors σ(Ψdp) and σ(Kdp) under the ideal conditions, which does not consider the random errors in Ψdp. It indicates that the GMM has a better error characterization based on the measurements when compared to the LR.

Figure  compares the Ψdp profiles given by Eqs. () and () with that obtained by LR. Figure a gives the same example as Fig. a, but the EM algorithm is configured differently. In the Ψdp density estimation, the mixture with full covariance yields density ellipses of random shapes. Furthermore, the algorithm repeats the fitting procedures three times to avoid the local maxima of the log-likelihood. Meanwhile, the choice of the cluster number relies on the Bayesian information criterion calculated for each m, starting at 10 clusters. It can be seen that the mixture composed by density ellipses characterizes the data points well, since the root-mean-square error is small relative to the expected values. Between 15 and 35 km, the narrow ellipses result in Ψdp with a rising trend consistent with LR. On the other hand, the mixture has very small variances, giving a high confidence for the fitted parameters. From 35 km, the ellipses become wider, and the associated variances increase due to the low signal-to-noise ratio at the edge of radar echoes. What is notable, however, is that the Ψdp profile dramatically increases to a large value, whereas LR remains a relatively steady trend. It indicates the importance of the Ψdp unfolding for the Ψdp density estimation.

Figure b presents another example of the density estimation. It is clear that the Ψdp profiles produced by GMM and LR both rise considerably along the range, and the trends for the two methods are very similar with a strong correlation of 0.998. The profile starts at about 50∘ and remains relatively stable before rising dramatically between 35 and 55 km. By 65 km, Ψdp has more than doubled, and then there is a steady increase for Ψdp reaching about 130∘ at the end of the profile, which is around 70∘ up on the ranges of 0 and 35 km, and 10∘ more than recorded at the ranges of 55 and 65 km. If we examine Ψdp measured at X-band frequency, we can see that some points fall out of the dashed lines corresponding to 1 standard error (i.e., 95 % interval). Most notably, the Ψdp profile shows a sudden slump between 18 and 20 km, while the ZDR corresponds to a local peak (not shown). It may indicate the occurrence of δco. In conclusion, the expected value and the variance of Ψdp can be obtained from the joint PDF, but the mixture needs to be tuned in terms of Ψdp unfolding and δco elimination in order to obtain the PDF of Φdp.

According to the continuity and consistency of the phase data, we can discern that some issues exist in the density estimation, such as ambiguous Ψdp and δco. Since Ψdp is a range accumulative measurement of the propagation phase, depending on the initial Ψdp(0), the measurements may exceed the dynamic range of 0–180∘ when the wave propagates through a rain medium. This situation is even more significant at X-band frequency than at S-band due to the inverse relation of the wavelength and the rate of phase shift. Nevertheless, it can be noted that Ψdp gives a nonnegative trend along the range for rain, and therefore the ambiguous Ψdp may be corrected accordingly .

In LR, Ψdp is first averaged over a small window for weather data, and a linear fit is then performed to obtain the increment for the range gate next to the window. In the following stage, a reference is predicted by summing up the average and the increment and compared to the observed value at the same gate. If the difference between the predicted and observed values is larger than 90∘, the observed Ψdp is then increased by 180∘. Finally, the correction process is iteratively operated until the last gate.

On the other hand, the Ψdp unfolding is more straightforward in GMM. Figure  shows the flowchart of the Ψdp unfolding and the δco elimination. After obtaining the PDF of Ψdp, the initial step of the Ψdp unfolding selects the density ellipses with at least six data points. Next, the second step calculates the difference of the means μi between the two consecutive density ellipses along the range. At this point, the PDF of Ψdp is ready to be corrected for ambiguous Ψdp. In the final step, the mean of the latter density ellipse is added up 180∘, if the former mean is larger than the latter one by 80∘.

As illustrated in Fig. a, the profile Ψdp reaches 180∘ at about 38 km and then becomes ambiguous between 38 and 42 km. In LR, the Ψdp values at these locations are interpolated according to the trend of the previous few gates, and the maximum value is 180∘. In contrast, the corrected density ellipses in GMM show an upward trend between 38 and 42 km, while the Ψdp profile reaches a maximum value of about 195∘, indicating the effectiveness of the Ψdp unfolding in the region of heavy rain. In addition, when we apply the algorithm to a larger dataset, the rate of false alarm reaches a very small value at 0.66 %.

In addition to the ambiguous Ψdp, the estimation of the joint PDF may also be affected by nonzero δco, which is defined as the phase difference between the horizontal and vertical polarizations upon the backscattering of the particles in a radar resolution volume. This effect occurs more frequently at X-band frequency than S-band due to Mie scattering . The δco is shown as a sudden phase change over a small number of gates in a monotonically increasing trend for rain. According to this manifestation, the magnitude and gate number of the Ψdp perturbation can be used to eliminate δco .

The linear regression model often adopts an iterative filter technique, which generates a new Φdp profile from either the raw data or the filtered one based on a threshold . If the filtering alters the data by 4∘, the new profile selects the filtered data; otherwise the raw data remain. The new profile is then used as input in the next iteration until the convergence condition is satisfied.

As shown in Fig. , the δco elimination is embedded into the process of the Ψdp unfolding. For two consecutive density ellipses, the latter density ellipse is removed if its mean is larger than the former one by 85∘. Prior to this step, the mean of the first density ellipse in the mixture should be below 90∘ to reduce the δco effect at the first few gates. Since δco occurs over a small number of range gates, a mixture pruning is also employed to remove the density ellipses with weights less than 0.0501, equivalent to 2 % of the data.

It is clear from Fig. b that δco has occurred at multiple locations in the data. The Ψdp profile starts at a high value and drops somewhat over the first two gates. Notably, there is a narrow gap between 18 and 20 km, while the corresponding ZDR presents a local peak. It signifies that nonzero δco occurs in this region. In GMM, these data are characterized by a density ellipse with a slightly decreasing trend, and the resulting expected values are consistent with the filtered data in LR. Between 70 and 90 km, a few isolated points beyond the density ellipses are associated with δco. Both of the methods can produce Φdp following the main trend of the data, which suggests that the process is effective for the δco elimination.

As discussed in Sect. 2.1, Kdp is the first derivative of Φdp with respect to the range r. According to the mean value and dominated convergence theorems, the derivative of the expected value of Φdp conditioned on r is equal to the expected value of the derivative of Φdp with respect to r, i.e., Kdp (see Appendix ). Following the notation in Sect. 4.2, we denote Kdp as y′. Therefore, the expected value of Kdp is obtained by taking the derivative of Eq. (), yielding 19Ey′x=1f2(x)∑i=1m∑j=1mfi(x)fj(x)x-μjxΣjxx-x-μixΣixx(aix+bi)+ai. The variance of y′ conditioned on x can be approximated by the first-order Taylor series expansion (see Appendix ); i.e., 20σ2y′|x=E′′(y|x)2σ2(y|x), where σ2(y|x) is given in Eq. (). By taking the derivative of Eq. (), E′′(y|x) is expressed as 21E′′(y|x)=2∑i=1maiwiy|x′+∑i=1m(aix+bi)wiy|x′′. From Eq. () in Appendix , it is clear that 22wiy|x′=gi(x)f2(x)=1f2(x)∑j=1mfj(x)fi(x)x-μjxΣjxx-x-μixΣixx, where gi(x) is the summation term. Subsequently, the second derivative of wiy|x is given as 23wiy|x′′=gi′(x)f(x)-2f′(x)gi(x)f3(x),where24f′(x)=-∑j=1mx-μjxΣjxxfj(x),25gi′(x)=∑j=1mfj(x)fi(x)x-μixΣixx2-x-μjxΣjxx2+1Σjxx-1Σixx. Equations () and () are the regression and skedastic functions for the Kdp estimation. In Eq. (), it is clear that the expected value of Kdp can be divided into two components, including Eqs. () and (). On the one hand, Eq. () is related to the changing rate ai weighted by the marginal distribution of each cluster in the mixture, equivalent to a linearly weighted combination of small portions of data. If a data point is dominated by a specific cluster, i.e., the weight of a cluster is significantly larger than the others, Kdp is determined by the coefficients of the cross-correlation and auto-correlation of r, and independent of the means and auto-correlation of Φdp, yielding a constant value within the dominated cluster. On the other hand, Eq. () shows that the weighting function also contributes to the Kdp estimates by considering the Gaussian derivative of the Φdp estimates in two or three adjacent clusters along the range. The sign of Kdp is then determined by the marginal means and variances of the clusters, weighted by the difference of their contributions to Φdp.

In Eq. (), it can be seen that σ2(Kdp) is proportional to σ2(Φdp), which is similar to Eq. (). When the LR is applied to the MZZU radar, we often assume that σ(Φdp) is equal to 2.61∘ with 32 pulses, 1 m s-1 for Doppler spectrum and 0.98 for ρhv under the ideal conditions. However, in the GMM, σ2(Φdp) varies along the range due to the random errors of the Φdp estimates, indicating the GMM can provide a better model for the error characteristics in the Φdp measurements. In addition, the statistical errors with respect to signal processing may be included in Eq. () as an additive term, independent of Φdp. Moreover, σ2(Kdp) in GMM is closely related to the first derivative of Kdp in Eq. (). As the changing rate of Kdp increases, the random errors associated with the Kdp estimates rise dramatically.

Figure b illustrates Kdp and its variance estimated from Φdp in Fig. a, which is the same case as given in Figs. a and a. It is apparent that the Kdp estimates present a large fluctuation, while the associated variances are significant. In GMM, Kdp starts from about 0.5 ∘ km-1 and then fluctuates between 17 and 20 km and between 24 and 42 km. In the profile, there are six local peaks with the maximum at about 8.5 ∘ km-1. Meanwhile, the Kdp variances vary as the Kdp estimates change. Between 15 and 17 km and between 20 and 24 km, the Kdp estimates remain constant, leading to small Kdp variances in these regions. When short excursions are present, such as that between 18 and 20 km, Kdp variances increase significantly due to the contribution of the first derivative of Kdp in Eq. (). Furthermore, the large Φdp variances between 35 and 42 km also result in an increase in the Kdp variances. In contrast, LR gives less fluctuation in Kdp estimates with two peaks at about 20 and 34 km. The comparison of Kdp obtained by the two methods may suggest that a smoothing procedure is required to reduce the significant variance in GMM.

As discussed previously, the Kdp variance is small for high Kdp but relatively large for low Kdp. Therefore, an adaptive estimation is adopted in LR. For radar reflectivity (ZH) less than 20 dBZ, the gate number n in Eq. () is set as 15, while n is 8 for 20≤ZH<35 dBZ and 2 for ZH≥35 dBZ. On the other hand, GMM also applies an adaptive technique based on the finite impulse filter (FIR) to the expected values of Kdp in order to reduce the associated variances. Figure  shows the time responses of the FIR with the cutoff frequency of 0.053 and the Gaussian window of 28, which yield the best performance for the MZZU radar. The impulse response (Fig. a) is peaked at the center and gradually decreases towards the two ends. Furthermore, the step response (Fig. b) gives the accumulation of the impulse response, indicating that the magnitudes around the center change faster than those at the two ends. If a longer window is required, the order of the FIR is increased accordingly. In this study, we gradually increase the order number to calculate the difference between the Kdp profiles obtained by the FIR filters with two adjacent order numbers. The optimal order of the FIR filter is then set when the relative square error of the two Kdp values is below 0.001. For profiles with sufficiently large data points, the order number is between 29 and 33 for the MZZU radar.

To obtain the reduced variance, we consider the filter as a number of weighting functions, denoted as hi(x), and subsequently the smoothed data become 26y=∑i=1nhi*xi, where y is a smoothed data point, xi is the original data within the smoothing window, and n is the window length. By taking the variance on both sides of Eq. (), we have 27σ2(y)=∑i=1nhi2σ2(xi). Therefore, the variance of the smoothed data is the weighted sum of the variances of the original data within the smoothing window. Since the FIR coefficients are much less than unity, σ2(y) is smaller than σ2(x) at the same gate. Furthermore, the Kdp estimates with the reduced variances can be used to reconstruct Φdp to obtain smaller Φdp variances. For a fixed gate spacing Δr, the reconstructed Φdp for the jth range gate is 28Φdpj=∑i=1jKdpiΔrand29σ2(Φdpj)=∑i=1jσ2(Kdpi)Δr2.

The red curves in Fig. a and b illustrate the reconstructed Φdp and the smoothed Kdp using FIR, respectively. The smoothed Kdp in Fig. b is more consistent with the LR results compared to the original Kdp produced by the GMM. In the first few kilometers, the smoothed Kdp gradually rises and then peaks at about 21 km. With no fluctuations, the smoothed Kdp falls gradually, followed by a growth before reaching a plateau at 33 km. After a slight decrease between 33 and 36 km, Kdp rises dramatically, which is very different from LR. Meanwhile, the variances are small at the beginning but get larger as Kdp is climbing. Between 20 and 33 km, the Kdp estimates do not change very much, leading to small variances in this region. But after 33 km, the variances begin to increase and retain large values until the end of the profile. Overall, the smoothed Kdp is stable, producing a profile considerably consistent with LR, and the variances are significantly reduced compared to the original data. In addition, the reconstructed Φdp (Fig. a) constantly increases with few local fluctuations, while the associated variances are smaller than the Φdp variances in GMM.

On 24 March 2016, a severe storm developed in central Missouri and moved eastward across Columbia, MO, causing strong winds and heavy precipitation at the surface. When the storm became mature, the S-band radars at Kansas City and St. Louis observed the storm structure at high levels, since each radar was about 150 km away from the storm. Notably, the Kansas City radar showed positive and negative Doppler velocities in a small area (not shown), indicating the occurrence of a downburst. On the other hand, the MZZU radar illustrated a bow echo of ZH close to the radar center (Fig. b). In addition to ZH, the GMM-based Kdp (Fig. d, e and f) was also obtained to investigate the storm structure near the surface.

Figure  illustrates that the convective storm evolves from a strong and large echo to a bow shape echo and then dissipates at far range. At 03:04 UTC (Fig. a), a cell with strong ZH moves into the radar area, while Kdp is moderate with a maximum of about 3 ∘ km-1 (Fig. d). As the cell is transforming to a bow shape, the radar echo becomes intensive and forms a rain band with embedded convective cores (Fig. b). It is clear to see that Kdp reaches over 10 ∘ km-1 in these core regions (Fig. e), indicating very heavy precipitation at the surface. With the fast movement of the storm, the downburst is weaker, and the storm starts to dissipate (Fig. c). At 04:41 UTC, it can be seen that Kdp is gradually reduced at the far range, while its maximum is much lower than that at the mature stage.

In this storm, the bow echo is shown as a number of convective cores embedded in a rain band, while the downbursts occurred at the leading edge near the echo center. The bow echo can be considered as a mesoscale convection with a horizontal dimension of more than 60 km. To gain further insight, Fig.  shows raw Φdp and Kdp for the bow echo. In Fig. a, raw Φdp presents large gradients along the leading edge, rising from about 50∘ to over 140∘. Due to the sharp increase, Φdp exceeds the maximum dynamic range, leading to ambiguity in the areas of X of -20 to -18 km and Y of 12 to 18 km as well as X of -40 to -23 km and Y of -8 to -5 km. In addition, the echoes behind the convective cores occasionally vanish as a result of signal attenuation. Nevertheless, LR (Fig. b) produces continuous Kdp by Φdp unfolding and linear interpolation according to the trends of the profiles, but some missing data still exist within the storm, due to the low signal-to-noise ratio. In contrast, GMM (Fig. c) corrects these data with the expected values derived from the joint PDF and simultaneously obtains the statistical errors in the production of Kdp. It is evident that the GMM method can efficiently handle the missing data via the mixture model, which is another advantage over the LR model. Furthermore, the statistical errors are not very large in these areas, since the missing data are filled by the distribution of the entire data profile. Additionally, the GMM Kdp estimates are generally a few degrees per kilometer (∘ km-1) higher than the LR ones, particularly for the regions of high ZH.

By taking a closer look at GMM Kdp, we can see that the bow echo is generally characterized by Kdp of above 2.5 ∘ km-1, while five pockets of high Kdp are identified. In the bow head, the first pocket presents very high Kdp associated with a rapid growth of Φdp. Behind this pocket, there is a region of negative Kdp, whereas LR generally yields positive values. It may be due to a reduction of the cross-correlation coefficient caused by the low signal-to-noise ratio, since the signals have been significantly attenuated after propagating through the pocket. In the middle of the second and third pockets in the bow center, LR and GMM both show lower Kdp compared to the two pockets, while Kdp is substantially consistent with the gradient of Φdp in the area. By considering the high ZH in Fig. , these moderate Kdp values may indicate less anisotropic scatterers, such as small hail in the process of wet growth. Similarly, a hail signature with maximum ZH of above 66 dBZ and small Kdp of 1–2 ∘ km-1 can also be identified in the middle of the fourth and fifth pockets in the bow tail. Along with the expected values of GMM Kdp, Fig. .d depicts the statistical errors σ(Kdp) in the calculation of the expected values. The five pockets of high Kdp are generally associated with small σ(Kdp) of a few tenths of a degree per kilometer. However, the estimates behind the top four pockets yield very large σ(Kdp) values with a maximum above 10 ∘ km-1, and the expected values of Kdp are sometimes below 0 ∘ km-1, such as areas of X of -25 to -20 km and Y of 11 to 20 km. In contrast, a region of high σ(Kdp) appears in front of the bottom pocket, superimposed on the high ZH area associated with hail. In conclusion, the GMM Kdp estimates of high confidence give good agreement with the gradients of Φdp in the leading edge of the bow echo, while large σ(Kdp) values are expected at the region of high variation of the Kdp estimates.

To give a further evaluation of the GMM Kdp, Fig.  compares the scatterplots of ZH, ZDR, and Kdp to the self-consistency (SC) relations. Referring to , , and , the X-band SC relations are given as 34ZDR=0ZH≤9.50.051ZH-0.4869.5<ZH≤552.319ZH≥55,35Kdp=1.37×10-3×100.068ZH×10-0.042ZDR,36Kdp/Zh=1.2×10-4-4.1×10-5ZDR(ZDR<1.6), where ZH=10log⁡Zh and Zh is in mm6 m-3. Figures a and b illustrate that the points concentrate at the region with ZH between 10 and 40 dBZ, where the Kdp shows a low and steady increase. Both the LR Kdp and GMM Kdp agree well with the SC relation in Eqs. () and (). It is notable that the Kdp rises dramatically from a few tenths to 8 ∘ km-1 for ZH larger than 40 dBZ. As depicted in Fig. a, the LR Kdp increases greatly when ZH reaches 50 dBZ, showing a difference from the SC relation. In contrast, the GMM Kdp in Fig. b gives some improvements over the LR Kdp in Fig. a when compared to the SC relation. Furthermore, the points of ZDR–Kdp/Zh in Fig. c and d may be grouped into two clusters with high populations. The cluster with lower Kdp/Zh agrees with the SC relation in Eq. (). On the other hand, the clusters centered at ZDR around 0 dB are likely caused by hails, since they are less anisotropic than raindrops with the same size. In addition, the LR Kdp and GMM Kdp produce a similar distribution of ZDR and Kdp/Zh, though the distribution for the GMM Kdp tends to be narrower.

Moreover, the computational time is crucial for the real-time application of the Kdp retrieval algorithms. For processing the data in Fig.  in Window 10 on a PC or Linux 7 on a supercomputer, the GMM takes about 7.058/4.068 s to process the Kdp with/without the data masking, whereas the LR reduces the time to about 2.037 s. It indicates that the LR has the advantages of simplicity and efficiency. Nevertheless, the GMM can obtain more information from the radar data, which is useful for the model studies.

In order to quantitatively evaluate the accuracy of GMM Kdp, hourly accumulated rain amounts are derived from the X-band rainfall rate algorithm and compared to the rain gauge data collected at Bradford, Sanborn, Auxvasse, and Williamsburg between 1 April 2016 and 2 June 2018. The scatterplots presented in Fig.  illustrate the comparison between GMM-based radar and gauge rain amounts, and the accompanying table (Table ) gives the RMSE, NB, and ρRG results obtained by GMM and LR.

Consistent with the data in Table , the rainfall at the four sites is predominately made up of light rain with hourly rain amounts no more than 2.5 mm h-1. Nevertheless, according to Fig. , moderate rain with amounts between 2.6 and 8 mm h-1 provides a considerable contribution to the total rain events, followed by a small portion of heavy rain with amounts more than 8 mm h-1. When we study the scatterplots and statistics for each of the four sites, it is apparent that Bradford (Fig. a and b) and Sanborn (Fig. c and d) are more concentrated on the red line than Auxvasse (Fig. e and f) and Williamsburg (Fig. g and h), since Bradford and Sanborn are closer to the radar. Accordingly, RMSEs for Bradford and Sanborn (Table ) are relatively small, about 13 %–35 % lower than Auxvasse and Williamsburg. Furthermore, it can be seen that Bradford and Sanborn show negative bias associated with negative NBs, indicating an underestimation of rain amounts by GMM Kdp. In contrast, a slight overestimation may be concluded for Auxvasse and Williamsburg by considering the point trends and the positive NBs. Additionally, Sanborn claims the highest ρRG out of the four sites, yielding the best consistency between the radar and gauge.

When compared to LR statistics as given in Table , it is clear that GMM improves the RMSEs, NBs and ρRG for Auxvasse and Williamsburg. Notably, the GMM-based NB for Auxvasse reaches a very small value of 0.04, one-fifth of LR-based NB. For Bradford, RMSE is reduced by GMM, but the absolute value of NB is slightly increased, while ρRG remains the same. On the other hand, for Sanborn, the GMM-based RMSE has been increased and the GMM-based NB and ρhv have been decreased comparing to the LR ones. It may be due to the local complex terrain near the radar. However, the difference of RMSE, NB, and ρhv between GMM and LR is a few hundredths of a millimeter, which is not significant relative to their absolute values. Overall, the rainfall estimates of GMM Kdp give a better performance than that of LR in terms of RMSE, NB, and ρRG at the far ranges.

To improve the accuracy of the radar rainfall estimation, we have optimized the R–Kdp relation in terms of the RMSE using the radar–gauge dataset. It leads to the relation as 37R(Kdp)=17.33Kdp0.92. Figure  shows the scatterplots of the radar–gauge data and the statistics of RMSE, NB, and ρRG obtained by Eq. () for all the four sites. It is clear that Eq. () has improved the negative trend in Eq. (). As illustrated in Fig. a and b, the points give a better concentration on the one-to-one reference line with noticeable changes for higher R(Kdp). In Fig. a, the LR R(Kdp) achieves fairly good RMSE, NB, and ρRG values at 2.30, 0.02, and 0.80, respectively. On the other hand, the GMM R(Kdp) presents a similar distribution to the LR R(Kdp), but the points in Fig. b are shifted toward the vertical axis. Moreover, the GMM R(Kdp) gives better RMSE and ρRG at 2.22 and 0.81, respectively, and slightly worse NB at -0.03.

It can be found that the rain rates based on the GMM Kdp have a moderate consistency with the rain gauge data. To further improve the results, some advanced rain rate algorithms can be considered, such as the rain–ice separation technique in the IFloodS campaign and the radar–gauge comparison method in the MC3E campaign . Nevertheless, the GMM has the advantage over the existing methods, since it can yield the variance of Kdp. Furthermore, the variance of R can also be obtained by the Kdp mean and the Kdp variance via the R–Kdp relation, leading to the variability in the error characteristics of R. Thus, the variances can be used to study the streamflow trends in the hydrological model.