Improvement in algorithms for quality control of weather radar data 1 (RADVOL-QC system)

. Data from weather radars are commonly used in meteorology and hydrology, but they are burdened with serious 8 disturbances, especially due to the appearance of numerous non-meteorological echoes. For this reason, these data are 9 subject to advanced quality control algorithms. The paper presents a significant improvement of the RADVOL-QC system 10 made necessary by the appearance of an increasing number of various disturbances. New algorithms are mainly addressed to 11 the occurrence of clutter caused by wind turbines (DP.TURBINE algorithm) and other terrain obstacles (DP.NMET 12 algorithm), as well as various forms of echoes caused by the interaction of a radar beam with RLAN signals (set of SPIKE 13 algorithms). The individual algorithms are based on the employment of polarimetric data as well as on the geometric 14 analysis of echo patterns. In the paper the algorithms are described along with examples of their performance and an 15 assessment of their effectiveness, and finally examples of the performance of the whole system are discussed.

various factors that cause measurement errors, such as the occurrence of different types of non-meteorological echoes, 23 especially those caused by interfering radio local area network (RLAN) signals and wind turbines (e.g., Bringi et al., 2011; Huge effort to resolve quality-related issues in radar observations has been made by various international research The network consists of eight C-band Doppler radars (Fig. 1) from which three are polarimetric. All the radars were 64 manufactured by European company Leonardo S.p.A., formerly Gematronik (Germany). The main parameters of the radars and designed scan strategy are listed in Table 1. There is a plan in place to replace all radars in this network with dual-66 polarization ones and to expand it by two more radars in the years 2022-2023. The 3-D raw data, so-called volumes, generated by POLRAD radars are quality controlled by the RADVOL-QC system. The

71
RADVOL-QC quality control system includes data correction and determination of QI resulting from each recognized error Values from a neighbouring higher elevation after adjustment to the vicinity mean (for DP.TURBINE, DP.NMET, and TURBINE). Interpolation from not burdened neighbouring bins across the radar beam (for SPIKE) NMET Removal of non-meteorological echoes: biological, caused by anomalous propagation, etc.
Analysis of altitude and intensity of echoes SPECK Removal of measurement noise (speckles) Analysis of a given bin vicinity and analysis of a spatial distribution of echo intensity MHV Compensation of underestimation due to distance to the Earth's surface . Its values range from 0 for extremely bad to 1 for perfect quality data. The specific value measurements, including the EUMETNET OPERA programme, issued the following drastic statement: "As an extreme 117 solution, and on a theoretical basis, an entire C-band network could be replaced by a dense network of X-band radars (...)" into any application, such as input into hydrological rainfall-runoff modelling or assimilation into mesoscale numerical complicates their removal because it disrupts their spatio-temporal pattern applied in the relevant algorithm.

123
As a response to the increasing demand for renewable energy, the number of wind turbines is growing rapidly in 124 many countries around the world. Their impact on weather radar performance has been extensively studied in recent years, were finally selected to be employed in this algorithm: the differential reflectivity factor (dB), the cross-correlation 143 coefficient between horizontally and vertically polarized radar returns , and the differential phase (°). The is 144 employed in the algorithm directly, whereas for the others standard deviations sd( ) and sd( ) computed within grids 145 of 3 bins x 3 bins are employed. Histograms of the selected parameters obtained for Ramża radar from two daysone with a It is notable that for correlation coefficient , the non-meteorological echoes generate values clearly different to 153 those for meteorological echoes, and the range of overlap is quite narrow. The standard deviations sd( ) and sd( ) 154 also differ in values for the three classes, especially for stratiform precipitation. The above information collected for these 155 three parameters allows one to deduce the presence of non-meteorological echoes with a relatively high certainty.

166
Having selected the most appropriate polarimetric parameters, a fuzzy logic approach was applied to categorize radar where is the echo class (meteorological or non-meteorological), is the parameter number, 3 is the number of 179 parameters, ( ) is the membership function for -th parameter for echo class , and ( ) is the weight ofth parameter for echo class . These weights equal 1.0, 0.5, and 1.0 for parameters , sd( ), and sd( ),

181
respectively. Comparison of the weighted sums decides which echo class a considered radar bin belongs to.

182
The designed algorithm proved to be effective, especially for the detection of wind turbines. The effectiveness of the 183 algorithm was checked on data in the form of monthly precipitation accumulation determined for the lowest antenna  200 Table 3 shows the influence of the DP.TURBINE algorithm on the values of monthly accumulation of radar echoes in 201 places where wind turbines and other obstacles (masts, chimneys) disturb the radar observations from Ramża radar. There are many such objects here and their monthly precipitation accumulations resulting from disturbance from these turbines 203 often significantly exceed a thousand mm. The of the developed algorithm for echoes from wind turbines in 204 range of the Ramża radar, presented in Table 3, is on average 0.82. However, if the intensities of echoes from wind turbines 205 are lower, this efficiency is lower.

206
In Fig. 5 an example of the DP.TURBINE running is also presented for the Ramża radar. Within this radar range a lot 207 ofseveral obstacles can be visible in the monthly accumulations, but only a small number of them are due to wind turbines.

208
The extremely dense strong echoes placed within a distance approximately 30 km from the radar site are a result of the 209 industrial and urbanized area (the Upper Silesia conurbation) located close to the Ramża radar. The existence of high 210 buildings produces such strong echoes due to side lobes from the radar beam. Other disturbances are caused by RLAN 211 interference.

212
The efficiency of the DP.TURBINE algorithm is significantly clearly visible in the right picture. It should be noted

265
"echo" denotes radar reflectivity which is different from the numerical value assigned to "no echo" in a given radar data 266 processing system. In the Rainbow system for Leonardo radars, this value is -32 dBZ).

267
It turned out that the DP.TURBINE algorithm is better at extracting small-area echoes associated with wind turbines, 268 while the DP.NMET algorithm detects more other non-meteorological echoes, especially those located near the radar (but

283
The following algorithm has been developed for the semi-automatic generation of static echo maps from wind 284 turbines and other groundclutter. At the first stage, fields of precipitation are generated for non-rainy time-steps (when a 285 number of rainy bins is below a defined threshold) for the lowest elevations of each radar. Every few months they are accumulated, and on this basis maps of permanent echoes from wind farms as well as residential and industrial buildings are corrected by comparing them with the initial fields and with previous masks in order to exclude from the masks echoes such 289 as those from the mountains to avoid significant reduction of precipitation in these areas. 293 Figure 9 shows the monthly precipitation sums for selected radars of the POLRAD network: from raw data and after 294 processing with the RADVOL-QC system without and with the TURBINE algorithm; the latter stands for the complete 295 system. The RADVOL-QC system proved to be very effective even without the use of the TURBINE algorithm because 296 some other algorithms, especially those based on data from dual polarization radars, detect echoes from wind turbines as 297 well. This improvement is especially noticeable for radars covering mountainous areas, such as Pastewnik and Ramża, as 298 well as highly urbanized areas such as Legionowo and Ramża. More effective removal of wind farms employing the 299 TURBINE algorithm for Legionowo can be explained by the fact that this radar is not dual-polarimetric.

315
In C-band weather radars, signals generated by external RLAN systems, and also from the sunso-called spike 316 echoesare interpreted by radar as precipitation echoes. The shape of these echoes is very specific: in Cartesian radar 317 images they have the form of elongated narrow spike-shaped echoes, located along the radar beam, sometimes with high 318 reflectivity. A significant problem with the RLAN-derived echoes is that although they may be partially removed or 319 decreased due to the low threshold of signal quality index (SQI) of radar data, this in fact complicates their removal because 320 gaps or changes in the intensity of these echoes introduced by this filter deform their spatial pattern, which makes the 321 algorithm effectiveness lower.
The SPIKE algorithm actually consists of a set of sub-algorithms that are sensitive to the various properties of this 323 type of radar echo. Since this algorithm has been an important element of the RADVOL-QC system from the very 324 beginning, it has been described in detail in the earlier work of Ośródka et al. (2014). However, it has undergone major 325 changes since then, so in this paper the areas where the most significant changes took place will be described in greater 326 detail.

327
In the algorithm for eliminating this type of echo from radar pictures, the spatial structure of the reflectivity field is 328 analysed separately for each elevation. The algorithm is divided into several sub-algorithms used to remove different types 329 of spike echoes, which are named in simple terms:

353
This newly introduced sub-algorithm is applied to spike echoes of generally elongated shape, but discontinuous along and 354 across the radar beam. The sub-algorithms described above cannot manage with such spikes due to discontinuities in their 355 pattern.

356
The algorithm consists of two steps: at first potential azimuths with "discontinuous" spikes are detected, then they are 357 confirmed by appropriate changeability of the radar beam. For the lowest elevation the first step is omitted because this scan 358 can be very rainy, so detection of potential spikes may not be efficient and the azimuths with potential spikes are taken from 359 the neighbouring higher elevation.
In the below procedure for each azimuth for a given elevation (apart from the lowest one) the echo bins are

377
Due to the specific shape of this type of echo, an additional check is introduced in order to smooth the field of 378 relevant potential spikes: azimuth is considered a potential spike if azimuths − 1 and + 1 are potential ones.

379
In the final step, bins in azimuths with such potential spike echoes are confirmed for each elevation employing

386
The concept of the above formula is that "discontinuous" spike echoes have a specific variance of radar reflectivity in 387 echoes across the radar beam. The variance calculated in dBZ, hence logarithmic (1 dBZ = 10 • log 10 (1 mm 6 m −3 )), is high 388 for echoes of this type (Fig. 10, on the left). However, high variance in dBZ is also characteristic of intense meteorological 389 echoes, especially those originating from convective rainfall. On the other hand, in the case of non-logarithmic values (mm 6 m -3 ), the variance of meteorological echoes is relatively high, because their values are generally higher and at the same time 391 more internally differentiated than that of spike echoes. Thus, the algorithm for the detection of "discontinuous" spike 392 echoes assumes that they are both characterized by a high variance calculated for reflectivity in dBZ and a relatively low 393 variance calculated for reflectivity in mm 6 m -3 .

394
The example in Fig. 10, shows that the criterion associated with high dBZ values clearly indicates radial echoes 395 located south and slightly west as "discontinuous" spikes, but also at the edges of clearly meteorological echoes. On the 396 other hand, the criterion based on the values of mm 6 m -3 prevents meteorological echoes from classification as potential 397 spike echoes, which could occur in the case of a few small echoes marked by a green ellipse. The red ellipse shows example of spike echoes. The echoes at the edges of meteorological echoes, which were classified as "potentially" nonmeteorological by both conditions, were finally not confirmed as spikes (see paragraph "Verification of potential spike-type

428
where ℎ is the threshold value. This procedure is performed for values of from 1 to 2 ℎ .

430
Verification of potential spike-type echoes

431
This SPIKE sub-algorithm plays a verifying role for all the sub-algorithms described above: it is used to check all echoes 432 that have been flagged as "potential" spike echoes, that is, suspected to be non-meteorological echoes.

433
For this aim, in a given azimuth and elevation the number of bins where "potential" spike echoes have been 434 detected is counted separately for each spike echo type:. iIf a pre-set threshold value has been exceeded, then the bins with 435 "potential" spikes of all types at a giventhe azimuth and elevation are recognized as true considered (confirmed) spikes.

436
An example of detecting spike echoes, including the "wide", "narrow", and "inverse", is shown in Fig. 11. ( , , ) = { 0.2 the "wide" spike is detected in a bin ( , , ) 0.7 the "wide" spike is detected in a beam 0.5 the "narrow" spike is detected in a bin ( , , ) 0.8 the "narrow" spike is detected in a beam 0.8 the "shorter longitudinal" spike is detected in a bin ( , , ) 0.9 the "shorter longitudinal" spike is detected in a beam 0.5 the "discontinuous" spike is detected in a bin ( , , ) 0.8 the "inverse" spike is detected in a beam 1.0 no spike in a beam

482
The effect of disturbances in weather radar data on the uncertainty of the results of the various meteorological and 483 hydrological models ingesting the data, e.g., in assimilation to mesoscale numerical weather prediction models or as an input 484 to hydrological rainfall-runoff models (Sokol et al., 2021), depends on the specific application. It is difficult to carry out one 485 general verification of the effectiveness of algorithms used to quality control this data.

486
The simplest form of verification is a visual investigation of the effect of corrections on particular time-steps. Fig. 12 487 shows an example of the combined performance of all the algorithms of the RADVOL-QC system. This example is 488 presented in the Cartesian system with the radar site in the centre, as the data from a single radar are usually distributed to 489 applications and end-users in this form. Surface rainfall intensity (SRI) radar product at the height of 1 km above ground

505
The verification can be performed from the perspective of systems issuing warnings about heavy precipitation. The 506 graphs in Fig. 13 show the number of exceedances of the threshold value of 1 mm / 10 min on the lowest elevation before 507 and after RADVOL-QC corrections over one month; areas with more than 200 alarms are marked in red. It is evident the 508 corrections prevent the generation of false warnings to a large extend.

514
All places marked in red can be associated with permanent non-precipitation echoes. These echoes are mainly from 515 mountain areas and large urban centres. For some radars, these are also echoes caused by signals from RLAN antennas 516 (Pastewnik and Ramża radars). The relatively extensive echo visible for the Legionowo radar to the east comes from a large 517 wind farm complex: this is the largest echo from wind turbines visible on the radars of the POLRAD network (see Fig. 8).

519
Over recent years a significant increase in the number of external disturbances in radar measurements has been observed, 520 especially those related to RLAN signals interfering with C-band radar signals, as well as to echoes from turbines and wind 521 farms, because their moving parts affect the radar beam in a specific way. On the other hand, a huge advancement in the 522 technology of radar signal processors used in modern weather radars has been observedthese are much better at filtering For these reasons, national meteorological services and various research centres are constantly developing more and 525 more effective algorithms for the detection and removal of non-metrological echoes from radar observations. Software involving the detection and correction of each type of disturbance separately must be constantly developed to take into 530 account their new manifestations.

531
A large part of this study is devoted to such new challenges. This paper does not describe the entire RADVOL-QC 532 system used to quality control data from the Polish weather radar network POLRAD, as it was already published in detail