Quality-based generation of weather radar Cartesian products

Weather radar data volumes are commonly processed to obtain various 2-D Cartesian products based on the transfer from polar to Cartesian representations through a certain interpolation method. In this research an algorithm of the spatial interpolation of polar reflectivity data employing quality index data is applied to find the Cartesian reflectivity as plan position indicator products. On this basis, quality-based versions of standard algorithms for the generation of the following products have been developed: ETOP (echo top), MAX (maximum of reflectivity), and VIL (vertically integrated liquid water). Moreover, as an example of a higher-level product, a CONVECTION (detection of convection) has been defined as a specific combination of the above-listed standard products. A corresponding quality field is determined for each generated product, taking into account the quality of the pixels from which a given product was determined and how large a fraction of the investigated heights was scanned. Examples of such quality-based products are presented in the paper.


Introduction
Weather radar measurements of reflectivity are burdened with numerous errors that are caused by both technical and meteorological factors (e.g.review by Villarini and Krajewski, 2010).These errors are recognised thanks to intensive empirical work performed at the national level (by national meteorological services) and at the international level, e.g. in the frame of weather radar-related COST (European Cooperation in Science and Technology) actions (Michelson et al., 2005) or the BALTRAD (An Advanced Weather Radar Network for the Baltic Sea Region) project (Michelson et al., 2012).The next step after error identification is the development of algorithms that can help to correct the data (Einfalt and Michaelides, 2008).Simultaneously to the correction of data, their quality can be estimated quantitatively, e.g. by means of quality index (QI) (Einfalt et al., 2010;Norman et al., 2010).Related research work, which has become more advanced with continuous progress in the field of correction algorithms, has continued with a view to operational work (Germann and Joss, 2004;Ośródka et al., 2010;Elo, 2012;Szturc et al., 2012a).
Raw weather radar data are generated as so-called volumes, i.e. 3-D polar data.Practically, such volumes consist of sets of measurement gates organised in polar scans related to the rotation of an antenna at selected elevation angles.Based on the transfer from polar to Cartesian representations through a certain interpolation method the volumes are processed to obtain various 2-D Cartesian products dedicated to specific user requirements (Heistermann et al., 2013).
The transformation is not a trivial task because the distances between neighbouring polar gates vary considerably with their location in relation to the distance to the radar site and scan strategy.The amount of information lost during this transformation was analysed: the effect of the conversion of polar to Cartesian coordinates is significant, especially for smaller catchments such as urban or mountainous ones (Gonzalez-Ramirez and Cluckie, 2006).Research works have been undertaken to improve the transformation, especially in terms of radar precipitation estimation.For instance, Henja and Michelson (1999) stated that employing some distance-weighting technique is more appropriate than using the value from the nearest gate.At present such techniques are operationally employed in different hydrological systems (e.g.Harrison et al., 2009;Elo, 2012)..5, 1.4, 2.4, 3.4, 5.3, 7.7, 10.6, 14.1, 18.5, 23.8The main assumption of the presented work is to ensure maximum reliability of the final products, so the transformation and subsequent specific product generation should be quality-based; i.e. particular algorithms should be designed taking into account the quality of particular measurement gates.Therefore the quality index fields assigned to reflectivity volumes should play an essential role in the task of 2-D product generation.
The paper is organised as follows.Since information about the weather radar data quality (expressed as QI) is incorporated into radar product definitions in the research, the method of quality characterisation is briefly described in Sect.2.2.The technique of quality control is based on algorithms developed for the RADVOL-QC package (Sect.2.3).Having 3-D volumes of reflectivity and relevant quality information, the set of 2-D Cartesian PPI (plan position indicator) products may be generated (Sect.3.1) together with corresponding quality fields (Sect.3.2).The quality-based PPIs constitute a starting point for the generation of more sophisticated products such as echo top (ETOP), maximum of reflectivity (MAX), and vertically integrated liquid water (VIL) (Sect.4.1).In addition, a non-standard product named CON-VECTION, which is dedicated to the identification of convective area based on all the previously described products, is defined.In Sect.4.2, the technique of QI field determination for the above products is described.Validation of the quality-based products is described and discussed in Sect.5, and finally their examples are demonstrated in Sect.6.

Data
The framework of quality-based products generation has been tested on data from the Polish weather radar network POLRAD operated by the Institute of Meteorology and Water Management -National Research Institute (IMGW-PIB), which is a national meteorological service in Poland.The network consists of eight C-band radars with the scan strat-egy defined in Table 1.The strategy includes 10 scans at elevations from 0.5 to 23.8 • with a beam width of 1 • .Sampling is performed every kilometre along the beam of 360 • in azimuths.

Quality index approach
The quantitative estimation of error magnitude is necessary not only in order to gain general knowledge about data uncertainty but also to apply quality information in further data processing, e.g. in the generation of standard or user-related specific products.One of the most common approaches in the characterisation of the quality of weather radar data is to employ QI, which is defined as a unitless quantity that provides information on the data reliability in a digital scale.Most often, the QI ranges from 0 (for the poorest quality) to 1 (for the best data) according to the EUMETNET OPERA (Operational Programme for the Exchange of Weather Radar Information) definition (Michelson et al., 2014), but other scales can also be applied (see review provided by Einfalt et al., 2010).Each category of errors burdening radar data is characterised by specific properties, spatial and temporal structure, and the possibility of diagnosis and correction; this consequently requires dedicated quality control techniques.Thus, the processing of the radar data is performed in a certain number of steps and after each one the data quality improves and a particular QI field is generated.Having determined a set of quality indices, a total QI field describing overall data quality can be computed, most often by using a multiplicative scheme (e.g.Fornasiero et al., 2005;Germann et al., 2009;Ośródka et al., 2014).

RADVOL-QC algorithms
In this research the quality control of radar reflectivity volumes was performed by means of dedicated software RADVOL-QC, which was developed to correct the data and generate QI fields (Ośródka et al., 2014).The software was integrated with the BALTRAD system for radar data exchange (Michelson et al., 2012), where it can work on data in HDF5 file format according to the EUMETNET OPERA digital information model ODIM (OPERA Digital Information Model) (Michelson et al., 2014).Additionally, in the IMGW-PIB the RADVOL-QC version developed for Gematronik Rainbow radar software works operationally since 2014 as a volume postprocessing of data in native Rainbow format.
The RADVOL-QC is a system designed for quality control of 3-D volumes in polar coordinates, which includes the data corrections and QI determination due to each recognised error source, and multiplication of the particular QIs into total QI.At present the system consists of the following algorithms (Ośródka et al., 2014;Szturc et al., 2012b): -quality characterisation due to effects related to the distance to the radar site (BROAD), -removal of conventional non-meteorological echoes (NMET), -removal of geometrically shaped non-meteorological echoes caused by external signal interference (SPIKE), -removal of measurement noise (SPECK), -correction due to partial and total beam blockage (BLOCK), -correction due to attenuation in rain (ATT).
The algorithms enable both the correction of data (excepting BROAD) and the estimation of the quality of corrected data expressed as QI.It should be emphasized that if a specific gate is found burdened with an error, its quality index is reduced even though the reflectivity value is improved, because each correction algorithm introduces some uncertainty in the data.

Quality-based transformation of 3-D polar data into 2-D Cartesian data
The raw data volume is organised in a set of scans consisting of measurement gates expressed in polar coordinates: scan elevation angle (ε), azimuth (α), and the distance from the radar site to the gate along the radar beam (l).For further processing every scan needs to be transformed into Cartesian coordinates (x, y).This is achieved by looping through all the Cartesian pixels of the 2-D output field and finding the corresponding neighbouring polar gates by means of trigonometric functions (Elo, 2012;Selex, 2010).
Here, an algorithm based on spatial interpolation of polar reflectivity data with respect to QI data is applied to find the Cartesian reflectivity Z data as PPI product and generate a corresponding QI PPI field.Following this, standard products such as MAX or VIL can be generated based on the set of PPIs and related quality information.

Generation of quality-based PPI product
PPI is one of the standard Cartesian products that represents reflectivity data generated from a single radar scan for constant elevation angle ε.The algorithm transforms values for measurement gates of polar coordinates (ε, α, l) into values interpolated for Cartesian pixels defined by coordinates (x, y).The values are projected onto a 2-D plane although they can originate from different altitudes.Usually, the transformation is performed while considering two or four of the closest gates, not considering the quality of particular gates.
In the proposed technique, the method of the quality-based interpolation depends on the density of the gates within the given Cartesian pixel.If the number of the gates is larger than the preset threshold, which occurs close to the radar, then they all are taken into interpolation (the so called nearfield sub-algorithm); otherwise, i.e. for pixels farther away from the radar, at most four gates (independently of their distances to the Cartesian pixel centre) are considered (the far-field sub-algorithm).
In order to distinguish between the near-field and far-field pixels a threshold value for distance from the radar site (D) is determined by the following function of the measurement parameters: where dα is the step in azimuth ( • ), dl is the step in distance from the radar site (km), and dx is the spatial resolution of Cartesian pixel (km).For instance, for typical data resolution dα = 1 • , dl = 1 km, and dx = 1 km, the threshold D equals 57.5 km.The formula was empirically determined based on analysis of the number of gates projected onto particular pixels assuming different data spatial resolutions.
The near-field sub-algorithm.In cases when the distance from the radar site to the given Cartesian pixel does not exceed the threshold distance D, then the number of gates within the pixel is determined.If this number is higher than two, the near-field method based on quality-weighted interpolation is used: where n is the number of gates within the investigated area.
Otherwise, if the number of gates is not higher than two, the far-field sub-algorithm is applied.
The far-field sub-algorithm.In the far-field area the closest gates are determined in a different way.The coordinates of the Cartesian pixel centre are transformed into polar coordinates and the four surrounding gates are taken into account.
Generally, the reflectivity for the pixel is interpolated from the four corner values (Fig. 1), unless some of the corners (one or two) are very close to the considered pixel centrethen only the closest gates are taken into calculation.
Reflectivity in a given pixel with centre in (x, y) is estimated as weighing an average value Z(x, y) from selected gates Z i , taking account of both distance to the gates and data quality information (QI i ): where n is the number of the closest gates taken into account (1, 2, or 4) and W Di is the weight related to the distance of i gate to the pixel centre (x, y), determined by means of one of the standard methods: nearest neighbour, uniform weights, inverse distance to the first or second power, bilinear method, or Cressman method.Differences in the overall view of 2-D products generated employing these methods are not very noticeable.However, they can be significant for the estimation of precipitation for small river catchments in cases of flash floods.

Characterisation of PPI product quality
Simultaneously to the determination of reflectivity for each Cartesian pixel (x, y), the relevant quality index QI PPI is calculated depending on the sub-algorithm applied to the data interpolation: -for the near-field sub-algorithm -for the far-field sub-algorithm

Generation of quality-based 2-D radar products based on PPI set
The below-described algorithms (ETOP, MAX, and VIL) employed for 2-D Cartesian product generation are standard ones (apart from CONVECTION product which is developed by IMGW-PIB for its needs).The proposed approach is to apply quality-based PPIs as input instead of standard ones, which allows us to obtain quality-based products.Moreover, the output products are complemented with related quality fields.

Echo top product
The ETOP product represents a Cartesian image of heights of echo (cloud) tops defining the cloud boundary at a preset level of radar reflectivity Z 0 (in dBZ).The ETOP (in kilometres) is detected in a preset range of heights (between h min and h max ) and generally is determined by interpolation of reflectivity Z in pixel (x, y) between the two highest PPIs for which the reflectivity passes Z 0 value (Fig. 2a).

Maximum of reflectivity product
The MAX product represents a Cartesian image of the highest measured value of radar reflectivity Z (in dBZ) in each vertical column.Generally, the product generation involves searching PPIs within a preset range of heights (between h min and h max ) for the maximal Z value in the column (Fig. 2b).

Vertically integrated liquid water product
The VIL product represents a Cartesian image of the water content residing in a user-defined layer in the atmosphere (in dBA).The VIL is defined by the following formula: VIL (dBA) = 10 log 10 where the liquid water content M (in cm 3 m −3 ) is related to radar reflectivity Z according to the so-called Z − M relationship (Selex, 2010).The integration range depends on values of both the required heights (between h min and h max ) and the measurement scope (between h lowest and h highest , which are determined for the lowest and highest PPIs respectively); the integration is performed from h min the lower height of the upper limits (Fig. 2cd).

Detection of convection (CONVECTION) algorithm
The algorithm for the separation of convective precipitation from stratiform background was developed as the first stage of the SCENE (Storm Cell Evolution and Nowcasting) model of precipitation nowcasting which forecasts convective and stratiform precipitation in different ways (Jurczyk et al., 2015).Radar reflectivity data provide one of the most significant pieces of information in the algorithm, and in an elementary version only radar information is employed (without data from other sources).
The dedicated radar product named CONVECTION is a second-order product as it is generated not from a set of PPIs but from earlier produced ETOP, MAX, and VIL products.Moreover, the horizontal structure of the radar reflectivity field turned out to also be a useful factor for distinguishing between convective and stratiform precipitation; therefore, the fields of parameters computed from the analysis of the spatial structure of the MAX and VIL (Jurczyk et al., 2012) are factors in CONVECTION field determination: exceedance of the Z background ( Z = Z/Z mean ) and exceedance of the VIL background ( VIL = VIL/VIL mean ).The two parameters are calculated as a ratio of the value in a considered pixel to the average of the rain pixels within the surrounding background of an 11 km radius (Z mean and VIL mean values).
The algorithm was designed by employing a fuzzy logic approach.For both precipitation classes (convective C or stratiform S) membership functions f class (i.e.f C or f S ) are defined for the five parameters described above.Then for each pixel the functions' values are aggregated as weighted sums for the classes: where "class" is the precipitation class (C or S), x is the particular convection parameter, f x class is the membership function value for x parameter, and W x class is the weight of x parameter.Comparison of the weighted sums for the classes decides which category, C or S, a considered precipitation pixel belongs to.

Characterisation of product quality
Generally, the quality of the 2-D product X (e.g.ETOP, MAX, and VIL), expressed by QI X , depends on the two factors: -the quality of reflectivity data from which a given product was determined, QI Xsource , and -how large a fraction of investigated heights (between h min and h max ) was scanned, QI Xscope .
The value of the first component QI Xsource is based on the quality of the PPI products (QI PPI from Eq. 4 and 5) defining the given product.Namely, -QI ETOPsource for ETOP is obtained from the QI PPI value in the pixel for which the ETOP was observed and in cases of interpolation from two measurements, the minimum quality is chosen; -QI MAXsource for MAX equals the QI PPI of the pixel for which MAX value was observed; -QI VILsource for VIL is an average quality of all PPIs defining the specific VIL.
For all the products, when the value of a given product equals "nodata" then the QI Xsource = "nodata" (and also the final QI X = "nodata"); when it equals "undetect" then the QI Xsource = 1.The second component, QI Xscope , is determined based on the heights of the highest and lowest scans for considered Cartesian pixel (h highest and h lowest respectively) in relation to h min and h max .Its value depends on what part of the height range between h min and h max defining the given product was scanned over the given pixel (Fig. 3 and Table 2).
The final quality index QI X is taken as product of the two components: The above procedure of quality determination is applied to first-order products like ETOP, MAX, and VIL.The quality index for a CONVECTION product is defined by the values of the two considered membership functions (see Eq. 7):

Validation
The effectiveness of quality-based generation of 2-D radar products can be indirectly evaluated by analysis of statistical properties of investigated product accumulated for longer time periods -at least 1 month for the whole radar range (up to 250 km).It is assumed that spatial data reliability can be assessed by symmetry and smoothness (Joe, 2011) expressed in the following way.
-The symmetry coefficient is quantified from differences between values x in pixels symmetrical with respect to the centre of the image: where i is the radar pixel number, i ∈ (0,. . ., n − 1); n is the number of pixels in the whole radar image; trunc() means truncation to integer; -The smoothness coefficient is evaluated employing a quantity called ENL i (equivalent number of looks) cal-culated locally around i pixel as the ratio of squared mean and variance within a certain vicinity from the formula: where µ i is the mean value in 5-pixel vicinity (grid of 11 × 11 = 121 pixels) of i pixel and var i is the variance in the same vicinity.The smoothness is defined as a mean of ENL i for the whole radar image: The evaluation has been performed for Brzuchania radar on data collected during May and June 2014.This radar was selected due to its specific features: location near mountainous area and strong presence of spike type echoes from interfering signals (mainly from Wi-Fi).For this reason the quality index field is strongly changeable in space, so employing the proposed quality-based definition of 2-D products can ensure meaningful benefits.The analysis was conducted on MAX products generated in a standard way, i.e. from PPIs obtained without quality information, and from quality-based PPIs (as described in Sects.3 and 4).
The results of the analysis on 2-month accumulations are presented in Table 3.The slightly higher values of the both coefficients indicate enhanced reliability of the quality-based MAX product in comparison to the standard one.
Evaluation of benefits from the proposed quality-based algorithm is difficult because its significant impact is observed only on relatively small excerpts of radar images.This impact is limited to these places where quality index considerably changes within a small distance.However, the quality-based products can serve as more reliable input data to rainfallrunoff models in the cases of flash floods and to other applications when very high spatial resolution is crucial, especially if software is able to take advantage of associated quality field.

Example
For this case study, data derived by Brzuchania radar on 27 May 2014 were employed.On that day convective rainfall with a large number of individual cells and multicell systems was observed.In Fig. 4 a quality index for the lowest scan (0.5 • ) depicted in polar coordinates is presented together with the raw and corrected reflectivity data.
In this case numerous errors were detected, especially the non-meteorological echoes near the radar site and the external signal interferences (narrow echoes along radar beams), and blockage on terrain in the southwest section are the most evident in Fig. 4a.A quality index is reduced for gates where these errors were detected; moreover, influence of radar beam broadening (the QI decreases with distance to the radar) and attenuation in rain (especially in azimuths about 315 • ) are noticeable (Fig. 4c).In such a situation the quality index field has a very diversified spatial pattern and is dynamic in time.
The radar data volume, consisting of reflectivity and QI data, is the basis for the generation of the 2-D products described above.In Fig. 5, the products generated from the example volume are demonstrated: the lowest PPI, ETOP, MAX, VIL, and CONVECTION, along with the related quality fields.
The first four products are similar to those obtained when not taking quality into account, because the changes are evident only locally where QI values strongly vary from gate to gate.The relevant quality index fields are mostly influ- ship functions of both classes and may be connected with the probability of the presence of convection in a given pixel.

Conclusions
Reliable quality information is crucial for user-expected radar-based products because it can be helpful in the gen-eration of more advanced information for various applications.The QI was found to be an appropriate quality metric.
A starting point for this research was processing by means of RADVOL-QC software, which corrects 3-D weather radar data and provides the total QI for each gate as a result of considering selected quality factors.The proposal to generate some 2-D products from 3-D raw radar data in a more advanced way when compared with standard procedures (e.g.Selex, 2010) is presented here.The main idea of the proposal is that interpolation of 3-D data into a set of PPIs and then into other 2-D products is performed in an optimal way, employing quality information related to each measurement gate.Here the following qualitybased versions of standard products are defined: Echo Top, MAX, and VIL.Moreover, the CONVECTION product for the identification of convective precipitation based on the abovementioned products is described as an example of more advanced quality-based radar products.It is expected that the final products will be of higher reliability not only thanks to previous correction steps but also because of the skilled introduction of quality information into the algorithms of product generation.Verification on the 2-month data set confirmed a slight improvement.However, due to employed methodology the validation was performed on the whole radar range whereas considerable enhancements are limited to smaller areas because of specific pattern of radar quality field.
The quality information assigned to the generated product also seems very important.It is obvious that individual schemes of quality characterisation for each specific product should be developed.However, a consistent framework needs to be agreed and implemented.The paper presents a proposal of such a framework.
The essential role of quality information in radar data processing is commonly appreciated.Areas where it may play an important role include the estimation of radar-based and multi-source (combined) surface rainfall rate, the generation of more reliable hazard indices for various services like civil protection or air traffic control, and especially the generation of probabilistic rainfall fields in the form of data ensemble or percentiles, as well as various other areas.

Figure 1 .
Figure 1.Scheme of interpolation of gate values into Cartesian pixels.

Figure 2 .
Figure 2. Schemes of generation of products from reflectivity Z values at particular PPIs: (a) echo top, ETOP; (b) maximum of reflectivity, MAX; (c and d) vertically integrated liquid water, VIL (cases for the highest measurement gate above and below the highest measurement level).

hFigure 3 .
Figure 3. Quality QI Xscope determination for 2-D product in terms of fraction of scanned heights.

Table 1 .
Scan parameters currently used in the POLRAD weather radar network of IMGW-PIB.

Table 2 .
Scheme of algorithm for QI Xscope determination.Case QI scope determination h highest ≤ h min QI scope = "nodata" (and QI = "nodata") h min ≤ h highest ≤ h max and h lowest ≤ h min QI scope = h highest −h min h max −h min h highest ≥ h max and h lowest ≤ h min QI scope =1 h highest ≥ h max and h min ≤ h lowest ≤ h max QI scope = h max −h lowest

Table 3 .
Criteria of reliability of standard and quality-based MAX products (Brzuchania radar, May-June 2014).