Evaluation of micro rain radar-based precipitation classification algorithms to discriminate between stratiform and convective precipitation

In this paper, we present two micro rain radar-based approaches to discriminate between stratiform and convective precipitation. One is based on probability density functions (PDFs) in combination with a confidence function and the other one is an artificial neural network (ANN) classification. Both methods use the maximum radar reflectivity per profile, the maximum of the observed mean Doppler velocity per profile and the maximum of the temporal standard deviation (±15min) of the observed mean Doppler velocity per profile from a micro rain radar (MRR). Training and testing of the algorithms 5 were performed using a two year data set from the Jülich Observatory for Cloud Evolution (JOYCE). Both methods agree well giving similar results. However, the results of the ANN are more decisive since it is also able to distinguish into an inconclusive class, in turn making the stratiform and convective classes more reliable. Copyright statement. TEXT

cesses are highly sensitive to the empirical parameters and assumptions.
In order to improve the parameterization of evaporation from convective rain a big data set of convective rain cases is needed to generate robust statistics. Since it is a large effort to manually discriminate between stratiform and convective cases, automated algorithms were developed. 5 In previous approaches, stratiform and convective rain are separated based on the rain drop size distribution measured by a disdrometer (Caracciolo et al., 2006;Thompson et al., 2015;Ghada et al., 2019). Precipitation was also classified using radar images and radar wind profiler data (Rosenfeld et al., 1995;Williams et al., 1995;Tokay and Short, 1996;Tokay et al., 1999;Yang et al., 2013). Deng et al. (2014) classified convective precipitation based on thresholds of the radar reflectivity and the 10 gradient of accumulative radar reflectivity retrieved from a vertically pointing cloud radar. Geerts and Dawei (2004) used a decision tree to separate different precipitation types by means of cloud radar variables. Additionally, discrimination algorithms using an ANN were developed (Yang et al., 2019;Ghada et al., 2019) . The ANN approach of Yang et al. (2019) is based on ground-based Doppler Radar observations. Lazri and Ameur (2018) combined a support vector machine, ANN and random forest to improve the stratiform convective classification using spectral features of SEVIRI data. Jergensen et al. (2020) classify 15 thunderstorms into three categories: supercell, part of a quasi-linear convective system, or disorganized using radar data in a machine learning approach.
In summary, several approaches such as ANN, fuzzy logics, or decision trees based on different instruments such as disdrometer, cloud radar, precipitation radar, or radar wind profiler were developed in the past. In this paper, two methods are 20 developed which classify rain as stratiform or convective event based only on MRR observations to enable a wide spread and straightforward usage for ground-based remote-sensing sites.

Supersite JOYCE
In recent years, the Jülich Observatory for Cloud Evolution (JOYCE 1 ) was equipped with a combination of synergistic ground- 25 based instruments (Löhnert et al., 2015). JOYCE is situated at 50 • 54 31 N and 6 • 24 49 E with an altitude of 111 m MSL.
In 2017 JOYCE was transformed into a Core Facility (JOYCE -CF) funded by the DFG (Deutsche Forschungsgemeinschaft) with the aim of high quality radar and passive microwave observations of the atmosphere. The supersite operates a variety of ground-based active and passive remote sensing instruments for cloud and precipitation observations, for example: X, Ka, and W-Band radars, ceilometers, a Doppler wind lidar, an atmospheric emitted radiance interferometer (AERI), a Sun photometer, 30 disdrometers, several radiation measurement systems, as well as an MRR. The latter is the main instrument in this study and is explained in detail in the following sub-section. The data used in this study was gathered in 2013 and 2014. The data from 2013 covers the entire year and was used to train the algorithms (training data set). The data from 2014 covers almost the entire year apart from February. It is a completely independent data set and is used as test data set for the algorithms. In 2013 and 2014, 471 and 683 hours of rain were observed, respectively.

5
The micro rain radar (MRR) which is built by the Metek (Meteorologische Messtechnik GmbH) company, is a compact FM-CW (frequency modulated-continuous wave) Doppler radar operating at 24 GHz (Peters et al., 2002). The MRR at JOYCE (in 2013 and 2014) is an MRR-2 system operating with 32 range gates. The lowermost range gates (number 0, 1 and 2) up to 200 m are affected by near-field effects and the last range gate of 3100 m is too noisy. These range gates are usually omitted according to Maahn and Kollias (2012). Hence, 28 range gates from 300 to 3000 m remain for the analyses in this study. The 10 vertical and temporal resolution amounted to 100 m and 1 min, respectively. The MRR data was processed according to Peters et al. (2005). The instrument was zenith pointing and measured the radar Doppler spectrum from which the mean Doppler velocity (v D ) were derived. The radar reflectivity factor (Z) is derived via integrating over the drop size distribution according to Peters et al. (2005). 15 3 Stratiform convective discrimination

Convection indices
Several weather indices can be used to describe the stability of the atmosphere (Kunz, 2007). Three indices that are based on thermodynamic profiles are described in the following. All give a hint on the probability of convection based on COSMO (Consortium for Small-scale Modeling) EU model data. COSMO-EU has a horizontal resolution of 7 km and a vertical reso-20 lution between around 60 m and 370 m below 3 km. The temporal resolution amounts to 1 h. The weather index total totals is a combination of the vertical totals (V T ) and cross totals (CT ). The V T is the temperature (ϑ in • Celsius) difference between 850 hPa and 500 hPa while the CT is 850 hPa dewpoint (τ ) minus the 500 hPa temperature: (1) 25 The higher the T T , the more probable is convection.
(2) The lower the KO index the higher the potential of convection.
The soaring index 2 (S) is intended to be a tool in soaring and sporting aviation because it gives a hint on thermal lift and hence on instability. It is defined as: The higher the S index the higher the probability of convection.

Convection score
First, a convection score to classify three types of precipitation labelled as stratiform, convective and inconclusive, is defined by applying a threshold range to six different variables. Three variables are based on thermodynamic profiles (T T , KO, S) and three are based on the MRR observations. Specifically, the used MRR variables are: the maximum of reflectivity (Z max ) 10 per profile, maximum of the mean Doppler velocity (v D,max ) per profile and the maximum (per profile) of the temporal standard deviation (± 15 min) of the mean Doppler velocity (σv D,max ). The profile maxima are calculated between ground and 3 km.
It is expected that larger rain drops are usually caused by convective precipitation (Niu et al., 2010) which leads to higher Z and v D values, respectively. Furthermore, stratiform precipitation is expected to be less variable over time whereas convective precipitation results in a larger standard deviation of v D over time. It is assumed that ± 15 min is a reasonable time span for 15 classification of rain events. If the rain event is shorter than 30 min (± 15 min) the variability is determined over this shorter period. The maxima of the height dependent Z, v D and σv D are used to assign the vertical properties to profile properties. In case of cold stratiform rain there might be a clearly defined melting layer. The so-called radar 'bright band' is indicated by erroneously high reflectivity values Z in the layer of melting ice particles which force the detection to be convective. Therefore, two other variables (v D and σv D ) are chosen which are not affected by the melting layer and both will counteract the false classification and force the retrieval to classify stratiform.
These strict thresholds enable a very certain classification with a low amount of false classifications. The inconclusive zone 20 between stratiform and convective indicates a transition between both. The thresholds of 3 and 5.5 were chosen to confidently separate two classes which are mainly free of false classified rain events resulting in a confident data set for training the algorithms. This approach replaces a manual inspection by visual classification of each single profile. However, several rain events ::::::: (approx. ::::: 10 %) : were reviewed by eye to verify a correct classification. That means randomly selected cases were checked if the convection score worked as intended and the synoptic situation was reviewed.

25
At this step, each profile is either classified as stratiform, inconclusive, or convective using the convection score and this assignment is stated as true state to train the algorithms explained below. Since the motivation of this work is to classify the precipitation type and its confidence purely based on the MRR observations the following methods based on PDFs or ANN are developed. Since the PDF and ANN method are based on training, the MRR data has to be free of extreme or unphysical 30 values. Therefore the MRR data (input) is filtered. Only measurements with Z max between -10 and 50 dBZ, v D,max between 0 and 10 m s −1 and σv D,max between 0 and 2.5 m s −1 are taken into account.
Here, the question might arise why inconclusive profiles should be learned by algorithms. In fact, rain events can be ambiguous and cannot be classified into stratiform or convective, especially stratiform rain moving towards mountainous area  which causes convection. On the other hand, vertical air motion and turbulence influence v D,max and might shift stratiform profiles towards higher convection scores and convective profiles to lower scores. A class with inconclusive profiles accounts for the mentioned features and avoids misclassifications into the stratiform and convective classes, respectively.
The frequency distribution of rain rate at 300 m height is shown in Fig. 3. The precipitation cases are separated by the convec-5 tion score. The stratiform precipitation mostly causes low rain rates below 1 mm h −1 whereas high rain rates above 15 mm h −1 are very rare. In contrast, high rain rates above 15 mm h −1 are caused by convective precipitation. It has to be considered that the absolute number of occurence differ from Fig. 2 because precipitation disappears due to evaporation on the way through the atmosphere and is not reaching 300 m which is the lowest available MRR height.

Rain classification method based on PDF
This algorithm was developed based on the classification algorithm by Liu et al. (2004) which was originally developed for the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) aerosol cloud discrimination (Winker et al., 2009). It shows that the confidence of a discrimination algorithm can be improved by using three measurement variables instead of only one or two. Later on, Liu et al. (2009) improved the algorithm by using five instead of three variables. Here, this 5 separation approach is modified for MRR variables to classify precipitation into stratiform or convective.
The confidence function is defined as: able to unambiguously distinguish between stratiform and convective precipitation indicated due to existing overlap regions.
The ambiguity can be reduced by adding a second dimension to the PDF. Figure 4 (g), (h) and (i) illustrate the distribution of an ambiguous assignment. In principle, these ambiguous assigned profiles with f values between −0.9 and 0.9 could be stated as inconclusive. However, the PDF algorithm is not trained to classify inconclusive cases. A quantitative estimation of how 5 well the discrimination works is given at the end of this sub-section.
By using all three mentioned MRR-based variables a three-dimensional (3D) PDF can be created which is visualized in Fig. 4 (j). It is indicated that both stratiform and convective profiles are clearly separated with a very small region of overlap. The quality of the 3D PDF-based classification in contrast to 2D and 1D can be explained in terms of failure rates R f (Liu 10 et al., 2009): As explained above the performance of the classification is limited by the amount of overlap in the PDFs. The smaller the overlap, the more clear is the separation between stratiform and convective profiles. Figure  This is much lower than the failure rates of 1D and 2D PDFs for stratiform-convective discrimination, which range between 2 to 7 % for the training data set and 3 to 15 % for the test data set.
It was shown that the algorithm performance could be improved by adding more variables. However, the amount of independent variables only obtained by MRR is limited. Z calculation is based on the drop number concentration. Other MRR 5 variables such as rain rate or liquid water content are also based on drop number concentration and are hence not independent from Z and would not add any more information to the discrimination algorithm.

Results
After the successful development and evaluation of the classification algorithms, both the 3D-PDF-based and the ANN were applied to two case studies. The first one was a rainy night on 26 May 2013 (Fig. 7 a-g). Figure 7 (e) shows the time-height display of the radar reflectivity factor. The day began with rain from 00:00 UTC to 02:30 UTC. The rain fell homogeneously with only small variations in Z max (a), v D,max (b) and σv D,max (c). The calculated convection score (d) was very low which 5 means that these rain events were stated to be stratiform. For this springtime rain events, the PDF (f) and ANN (g) method produce very similar results and both agree with the true class given by the convection score.
The right panel of Fig. 7 shows the same quantities as on the left panel but for 23 July 2013. This case indicates convective rain falling between 15:00 UTC and 16:00 UTC. Z max (h), v D,max (i) and σv D,max (j) and the calculated convection score 10 are characterized by high values representing convective rain. Figure 7 (l) shows the radar reflectivity factor of the shower. The PDF-and ANN method are in a very good agreement and classify each profile as convective in conformity with the convection score (truth).
The performance of both algorithms over a whole year (test data year 2014) is illustrated in Fig. 8. It shows the relative 5 frequency of occurrence of precipitation profiles that are defined by the convection score (truth) to be stratiform (a), inconclusive (b), or convective (c). For the PDF method cases are stated as stratiform when the f value is lower than −0.9, inconclusive when f is between −0.9 and 0.9, and convective when f is larger than 0.9. For the stratiform cases the PDF and ANN methods classify most stratiform cases to be stratiform (84.7 % and 96.1 % for ANN and PDF, respectively, see Fig. 8 a). Only 15.3 % and 3.9 %, respectively, are erroneously classified as inconclusive. These are cases with higher convection scores with averaged 10 values around roughly 2.5 which is closer to the transition of convection scores larger than 3 that are stated as inconclusive. As expected, neither ANN nor PDF misclassified true stratiform cases as convective. The performance of the classification of true convective cases (c) is very similar. There are almost no completely misclassified cases and only a few percent of erroneously inconclusive cases. Here the averaged convection scores are roughly 6 which means on the lower edge of the convective classification and close to the transition of convection scores of less than 5.5 that are stated as inconclusive. 85.8 % and 98.1 %

15
(ANN and PDF) of the true convective cases are correctly classified as convective.
However, the most critical point is the classification of the true inconclusive cases (Fig. 8 b). Only 71 % and 35.8 % (ANN and PDF) are correctly classified. That means that nearly 30 % of ANN-based profiles and nearly 65 % of PDF-based profiles of all true inconclusive cases are classified as stratiform or convective. The ANN is performing better here. This is caused by 20 the strict convection score discrimination which was stated as truth. In fact, these inconclusive cases might be classified as stratiform or convective but the thresholds were chosen very strict to confidently separate two classes which are mainly free of misclassified rain events. The averaged convection score of the false stratiform inconclusive cases (ANN) amounts to 3.6 and of the false convective inconclusive amounts to 5 (Fig. 8 b). The erroneously stratiform and convective classified inconclusive cases of the PDF method (25.9 % and 38.3 %) has averaged convection score values of 3.6 (stratiform) and 4.9 (convective). 25 Apparently, these cases would be correctly classified in case of less strict convection score thresholds than currently used (3 and 5.5, see Fig. 2). It is expected to improve the ANN and PDF performances by gathering more data for algorithm training.
It has to be considered that the total amount of data is different for both methods. This is due to the fact that some combinations of the three input variables do not appear within the training data causing gaps in the 3D-PDF. Those combinations 30 cannot be classified but its amount is less than 0.2 % for the 2014 test data set.

Conclusions and outlook
In order to improve microphysical parametrizations within small-scale models one has to deal with large data sets. The presented rain type classification methods based on PDF and ANN algorithms are suited to process micro rain radar data from long time seriesand outperform traditional convection score methods. The effort of creating a robust training data set without unphysical data between both methods is similar and the application of both methods is straightforward. The main advantage 5 of the ANN in contrast to the PDF method is that the ANN method was trained to directly classify inconclusive profiles which leads to a lower amount of false classified profiles.
In a next step, evaporative cooling rates will be estimated for convective rain events to parametrize the cooling by means of temperature, relative humidity and rain droplet number concentration. It is also planned to apply the algorithms to different 10 ground-based remote-sensing sites that have long-term MRR observations to create stratiform-vs-convective rain event climatologies. At present, the new MRR of the University of Leipzig 5 is running 24/7. In the near future, the classification algorithms will be applied operationally and will be improved with continuously gathered data.
Code availability. The open source machine learning library for research and production TensorFlow (Abadi et al., 2015) used for this publication is available under https://www.tensorflow.org/, last accessed: 2020-12-01.