Title: Evaluation of convective cloud microphysics in numerical weather prediction model with dual-wavelength polarimetric radar observations: methods and examples

Author(s): Gregor Köcher, Tobias Zinner, Christoph Knote, Eleni Tetoni, Florian Ewald, and Martin Hagen

Summary

This study examines a series of regional storm-resolving weather forecasting with different microphysics schemes against long-term observations of dual-wavelength polarimetric radars over Munchen area. Although there are many papers inter-compare microphysics schemes available in WRF, the noble aspect of this study is to utilize polarimetric radar simulator and cell-tracking algorithm for more consistent sampling of polarimetric radar observables and dual-wavelength ratio. However, there are some major questions/suggestions related to 1) missing citations, 2) separation of convective cells, and 3) actual rain drop-size distributions, and 4) uncertainties of the forward model. By improving these issues, this manuscript could be quite powerful. Thus, my suggestion is “major revisions” in order to publish in ACP.

Major Comments/Suggestions

1) Missing citations

This study cites several microphysics evaluation papers, but it completely misses previous studies directly using forward polarimetric radar models. Here are suggested references. Please take a look and relate yours to their findings and approaches. 

Jung, Y., M. Xue, and G. Zhang, 2010: Simulations of Polarimetric Radar Signatures of a Supercell Storm Using a Two-Moment Bulk Microphysics Scheme. J. Appl. Meteor. Climatol., 49, 146–163, https://doi.org/10.1175/2009JAMC2178.1

Snyder, J.C., H.B. Bluestein, D.T. Dawson II, and Y. Jung, 2017a: Simulations of Polarimetric, X-Band Radar Signatures in Supercells. Part I: Description of Experiment and Simulated ρhv Rings. J. Appl. Meteor. Climatol., 56, 1977–1999, https://doi.org/10.1175/JAMC-D-16-0138.1

Ryzhkov, A., M. Pinsky, A. Pokrovsky, and A. Khain, 2011: Polarimetric Radar Observation Operator for a Cloud Model with Spectral Microphysics. J. Appl. Meteor. Climatol., 50, 873–894. doi: http://dx.doi.org/10.1175/2010JAMC2363.1

Putnam, B.J., M. Xue, Y. Jung, G. Zhang, and F. Kong, 2017: Simulation of Polarimetric Radar Variables from 2013 CAPS Spring Experiment Storm-Scale Ensemble Forecasts and Evaluation of Microphysics Schemes. Mon. Wea. Rev., 145, 49–73, https://doi.org/10.1175/MWR-D-15-0415.1

Also a following citation is recommended for microphysics finger print from polarimetric radar signals. This Ph.D. dissertation describes various microphysics finger prints related to cloud microphysics processes. 

Kumjian, M.R., 2012: The impact of precipitation physical processes on the polarimetric radar variables. PhD. Dissertation, University of Oklahoma, 327 pp.

2) separation of convective cells

One of the advantages of this paper is the large sampling volume, but it simultaneously induces ambiguity for analysis. During a long-term period, there must be various sizes of convections from shallow, congestus, and deep convective/stratiform cells. When you bundle all into a single CFAD, it tends to smear out important aspects of microphysics. Please check the following papers on how it separates cloud type and better evaluates different aspects of microphysics from long-term simulations/observations.

Matsui, T., X. Zeng, W.-K. Tao, H. Masunaga, W. Olson, and S. Lang (2009), Evaluation of long-term cloud-resolving model simulations using satellite radiance observations and multifrequency satellite simulators. Journal of Atmospheric and Oceanic Technology, 26, 1261-1274.

You can use echo-top height from each cell-tracked target for separation. But if this type of separation is too difficult to implement (or too much effort), please just discuss and try it in the future. 

3) rain drop-size distributions

Probably the most robust finding in this study is the variability of rain-DSD related radar signals (ZDR and DWR) among different microphysics schemes as seen in Figures 5, 6, and 7. Above-melting-zone evaluations tend to have more uncertainties in the forward model (described in next). To augment your finding in radar signals and discussion, it’s much better to directly examine simulated rain drop size distribution profiles (like CFAD format) from different microphysics schemes. This should not be a difficult task. ( it’s much better if you have disdrometer observations!)

4) uncertainties of forward model

Details and uncertainties in assumptions of the forward model (CR-SIM) are not discussed. In order to represent simulated microphysics in polarimetric observables, one must assume particle shape and orientation simultaneously in the forward model, because these are not “explicitly” simulated in most of the microphysics schemes in WRF. Following paper discusses and tests different assumptions. Please describe what kind of shape/orientation assumptions are made for each microphysics in Section 2.4 (or Appendix), and related discussions in Section 3.3. 

Matsui, T.,  Dolan, B.,  Rutledge, S. A.,  Tao, W.‐K.,  Iguchi, T.,  Barnum, J., &  Lang, S. E. (2019).  POLARRIS: A POLArimetric Radar Retrieval and Instrument Simulator. Journal of Geophysical Research: Atmospheres, 124. https://doi.org/10.1029/2018JD028317

Minor Comments/Suggestions

Line 25: “the huge number” -> “a large number”

Line 26: “on scales of μm to mm and” -> “on scales of mm or smaller” for consistency. In fact, microphysics processes occur less than the scale of micron, such as ice crystallization processes. 

Lin 82: “with a sound statistical basis” ?? I don’t understand.

Line 89: “separate the microphysical impacts from possible feedbacks.” I agree. But more bottom line, I would argue whether your set of numerical weather model resolved dynamics or not with 2km horizontal grid spacinig. 

Line 108: “frequency” -> “frequencies”

Line 149 and repeat the same: “a horizontal resolution of 10 km,” must be replaced by “horizontal grid spacing of 10km”. Numerical atmospheric model does not have 10km resolution with 10km of horizontal grid spacing. Effective dynamic resolutions are x5 ~ x10 of horizontal grid spacing in numerical dynamic core. Apply this correction elsewhere in the manuscript. 

Pielke, R. A. (1991). A Recommended Specific Definition of “Resolution”, Bulletin of the American Meteorological Society, 72(12), 1914-1914. Retrieved Oct 30, 2021, from https://journals.ametsoc.org/view/journals/bams/72/12/1520-0477-72_12_1914.xml

Line 159: Please briefly describe other physics options, such as land surface, PBL, and radiation schemes. 

Line 203: Did you store and use all 33bin of hydrometeor classes to calculate radar observables in CR-SIM?

Line 286: “but none of them as pronounced as in the observations.” Well, this is typical situations that relatively coarse-resolution model won’t be able to resolve tiny cells. So you are running with 2km horizontal grid, meaning that you can resolve convective features in 10km or 20km well, but never be able to resolve 2km-size of convection, which tend to have shallower echo-top heights. So, don’t blame to microphysics, but model dynamic core and grid spacing you chose. 

Line 313: “Contoured frequency by altitude distributions” -> “Contoured frequency of altitude diagram”

Line 322: “image 5” -> “Figure 5”?

Title: Evaluation of convective cloud microphysics in numerical weather prediction model with dual-wavelength polarimetric radar observations: methods and examples.

Authors:          Köcher, Gregor

                        Zinner, Tobias

                        Knote, Christoph

                        Tetoni, Eleni

                        Ewald, Florian

                        Hagen, Martin

doi: 10.5194/amt-2021-299

Summary: This study compares polarimetric dual-wavelength observations of convective cells observed by three radars to simulations conducted using 5 different microphysics schemes on a large statistical basis. The study is well-motivated and described with clear, informative figures and has the potential to be very well-received as it is a very topical study. The study also benefits from its large sample size (versus individual case studies done in the past). However, in addition to a few changes for clarity and requested further exploration, I have concerns about some of the analysis and conclusions drawn, contributed to by both vagueness of the details of the radar operator and the understood assumptions about the microphysical schemes employed. In addition, some of the analysis seems to rely on simplifying assumptions/conjecture that could potentially be resolved by including additional info from the simulations (e.g., PSDs) besides the bulk polarimetric quantities. Because of the fundamental nature of these concerns, I recommend major revisions before publication in AMT.

Main comments:

1. I really think much more information is needed about the forward polarimetric radar operator applied. Even though a citation is given, the realism of the assumptions made about the treatment of 1) particle shapes, 2) particle orientations, and 3) dielectric constants (especially during multi-phase environments, such as melting), etc. could really strongly influence the resultant simulated polarimetric radar variables and thus deserves to be fleshed out here. None of this uncertainty (or the inevitable reduction in variability inherent in applying fixed relations within an operator like this) is currently acknowledged or taken into account in the subsequent analysis. Finally, the lack of details about things like aspect ratio relations provided complicates the understanding of other parts of the discussion, such as raised in the following comment.
2. There seems to be some confusion about the nature of the microphysics schemes employed that influences some of the manuscript’s analysis and main conclusions. The primary issue is with regard to the Thompson microphysics scheme, although similar language/conclusions permeate the paper. The authors state on line 186 that snow is “not considered to be spherical” in Thompson in contrast with other schemes, which treat particles as spherical. An examination of the Thompson et al. (2008) manuscript indeed sees similar language employed to describe the scheme, which has a mass-size exponent that differs from the “spherical” value of 3. This value, of course, comes from the volume of spherical particles (D³) multiplied by a density that varies inversely with diameter (D^-1), as stated in the abstract of Thompson et al. (2008), leading to an ultimate dependence of mass on D². However, despite the language used concerning this exponent, which upon reflection I now consider a misnomer, this does not actually ensure that the treatment of the particles is non-spherical in the physical shape sense. In fact, I am not aware of any operational microphysics scheme that actually explicitly predicts the shape of the snow with the exception of the FSBM (and possibly the P3?), that uses fixed aspect ratio-size relations to evolve particle shape as mass is gained/lost (see A1 in Shpund et al. 2019). However, other schemes may implicitly incorporate some shape information through things like the capacitance term in the deposition/sublimation rate equations, etc (for example, this is done in Thompson; see Deposition/sublimation section in the Appendix in Thompson 2008), but it isn’t clear that this implicit information is actually being used by the radar operator. Hence, similar language about other schemes (e.g., Line 221 about “non-spherical” snow in the P3) is also potentially misleading.

This confusion in framing/treatment leads to incorrect assumptions further on, such as line 360 where it is stated that the Thompson scheme actually treats snow as “oblate” particles in a way that would actually affect scattering amplitudes at different polarizations. That is, to my understanding, only something that would be specified within the forward polarimetric radar operator, which is why it is important to include details of how shapes, etc. are being handled as per Main Comment 1. One could envision two alternative scenarios in conflict with these ideas: a model scheme that considered snow to be “spherical” (in the Thompson parlance) for microphysical purposes that has a constant density and an m-D relation with an exponent of 3 but that in the radar operator is assigned an aspect ratio < 1 that results a ZDR > 0 dB. Alternatively, one could have an m-D relation with an exponent of 2 that was “nonspherical” (in the Thompson parlance) but that in the radar operator treated all snow as spheres regardless of the density varying across the size spectrum, resulting in a ZDR of 0 dB regardless.

As a result, the assertion on line 359 that the particles are being treated as “spherical” in the FSBM scheme is the reason for its poor ZDR observational agreement is (to my understanding) necessarily incorrect, as 1) shape *is* predicted in the FSBM via size-shape equations, 2) the inverse-dependence of density on particle diameter for snow is also taken into account in the FSBM, so it is “non-spherical” even in the D²/Thompson parlance, but also 3) this also leads to the incorrect conclusion that that is related to why the simulated ZDR is 0 with no mention of the radar operator. If snow and ice particles were in fact treated as spheres in the radar operator, where the ZDR calculations are actually being performed, every single ZDR value aloft would be 0 dB, but we do see spread apparent in the CFADs even in the FSBM and Morrison schemes, which can’t be explained by this theory of spherical treatment in the microphysics scheme. All of this also obfuscates the role that density and the assumed PSD form in each scheme are likely playing in the spread of ZDR values aloft, which are hardly discussed at all in the manuscript’s results section. I believe much of the analysis needs to be re-examined in light of these understandings.

Specific comments:

1. Line 105: Were cases chosen in any systematic way (e.g., precipitation intensity, coverage, etc) or just randomly throughout the 2019 and 2020 seasons?
2. Line 115: Is there a reason KDP was neglected in the analysis? It is available from CR-SIM and would provide important additional information for contextualizing the differences between observations and simulations. Otherwise please include an explanation of why the study was limited to Z/ZDR/DWR.
3. Section 2.1: Can information about any efforts for radar calibration be included, particularly for the non-operational research radars? Poor calibration could, in theory, affect both Z and especially ZDR.
4. Line 132: It isn’t clear how far away from the radar these cells typically were when being scanned, but was any effort made to correct the ZDR values to account for high-elevation scans in the RHIs?
5. Line 138: Was the cell movement just a simple extrapolation of storm centroids, or done by visual inspection?
6. Line 155: Please include UTC conversions and list in terms of LST instead of ‘am/pm’.
7. Line 156: What exactly is meant by ‘nudging’ from the GFS? Was the entire (large-domain) model background replaced with the new GFS analysis, or was it incorporated/assimilated somehow? Or did this only apply to the boundary conditions?
8. Line 168: It may help to specify which D is being referred to: maximum diameter, equivolume diameter, etc.
9. Line 179: Just for clarity, I would add “fixed” before non-zero mu just to make clear it is not a free parameter.
10. Line 238: How was this 32 dBZ threshold chosen and why? Is this the default TINT value?
11. Line 254: When there are multiple Cartesian input grid points for a given target spherical grid, how are they “all included”? Means? Median? Weights? Etc.
12. Line 284: It isn’t clear to me exactly why 32 dBZ is used as a threshold here – it seems much larger than most studies. I understand 32 dBZ is used for TINT (as per comment 10), but once a cell is identified can’t a lower Z threshold be chosen for cloud/echo top height? Similarly, I am not sure I like the use of ‘echo top’ here given such a high reflectivity threshold, as this normally applies to thresholds like -10 dBZ or 0 dBZ while 32 dBZ is solidly in the middle of many convective cells. By using “echo top height”, it implies something about the depth of the simulated storms, while in actuality the trends seen seem to just reflect high biases in the simulated Z throughout the depth of the cells. Consider alternate language.
13. Line 305: This entire paragraph seemed a bit random and out of place to me. The results here are never compared to the findings of Caine et al. (2013), and the subsequent defense of the study has already been thoroughly provided earlier in the paper.
14. Lines 319-324: It still isn’t clear to me if a bias may be being introduced here due to the RHI scanning strategy. Were the +/- 2 deg RHI scans typically still within the precipitation core or on the edges/ flanks of the cells? With all simulated columns being included in the CFADs it almost seems inevitable that more weak precipitation regions would be captured that way?
15. Line 332: I know this probably varied among cases but including an approximate ML height here may be useful.
16. Line 335: While the distributions are certainly broader and extend to higher Z values above the ML, the medians for most schemes still appear quite close to the median in the observations to me.
17. Line 338: Were PSDs actually examined? It says “(not shown)”, but this may be helpful to include. While it is definitely plausible that the graupel produced is too large, could it also be that there’s just too much riming in general (so the particle density is the problem, not its size)?
18. Line 355: While I have no doubt in general that the flexibility of the FSBM is aiding its ability to reproduce realistic ZDR values, have other factors been considered, such as the different treatment of drop breakup among schemes, etc?
19. Lines 357-362: In addition to the issues raised in the main comments regarding lines 359-360, how much of the differences (for example, the narrowness of the ZDR distributions) between schemes is due to differences in the calculated differential attenuation versus differences in ‘intrinsic’ variables that affect ZDR, such as shape and density? In general the differential impact on density needs to be explored. The profound differences in ZDR at high altitudes between the FSBM/Morrison scheme and the Thompson/P3 schemes also deserves to be explored.
20. Line 366: While size is definitely the main factor, I would not diminish the role that density plays in determining the resonance parameter and thus whether non-Rayleigh scattering is occurring. (Although, of course, in these simplified schemes density is at best a simple function of size, so it isn’t a free parameter…)
21. Line 380: This is incorrect. While the P3 is indeed more flexible, Thompson et al. (2008) says in its abstract, “[this scheme employs] a bulk density that varies inversely with diameter as found in observations and in contrast to nearly all other BMPs.”
22. Line 390: I appreciate the discussion about the potential erroneous growth by collection that is influencing the DWR below the melting layer. However, it is also interesting how different the DWR already is immediately below the ML. It seems to me that the particles leaving the melting layer may be very different in size between the observations and simulations. In the observations, perhaps stochastic breakup is occurring toward the surface that is reducing the DWR of large droplets in the obs while the simulated drops are too small and grow by collection instead? I think this is worthy of further exploration since it is one of the most pronounced differences between observations and simulations.
23. Line 392: I am confused by the sudden discussion about vertically pointing radars, which were not used in this study?
24. I don’t think the acronyms need to be redefined in the summary (e.g., NWP, FSBM, PSD, etc).
25. Line 445: Again, I am not sure this is a correct conclusion to draw as it depends more on the details of the radar forward operator.

Typos, etc.

1. Line 115: “differential phase” should be “specific differential phase”.
2. Line 117: For clarity, “single-polarimetric” should be “single-polarization” (some readers automatically infer multiple polarizations from the term ‘polarimetric’).
3. Line 147 and elsewhere: "times" should be "x" or "by"
4. Line 218: “Mass size” should be “Mass-size”
5. Line 260: db should be dB
6. Line 256: “cumulated” should be “accumulated”
7. Line 280: "extend" should be "extent"
8. Line 324: “image 5” should be “Figure 5”