Review of AMT-2022-62:

ICE-CAMERA: a flatbed scanner to study

inland Antarctic polar precipitation.

authored by M. Del Guasta

April 4, 2022

Global discussion

This manuscript presents the design concept and some data applications

of a smart measurement device: ICE-CAMERA. The instrument is able to

operate in an extremely harsh environment, to collect autonomously precious

data of hydrometeor type on the Antarctic plateau and to routinely perform

a calibration/focus correction procedure. The operation of an unmanned

device collecting this type of data in the environment of the Antarctic plateau

is remarkable and the data themselves are unique and precious. The paper

fits well the scope of AMT. I have a few major remarks and a certain number

of other comments that should be taken into consideration before publication.

My current recommendation is of major revision.

Major remarks

1. As mentioned, the topic fits well the scope of AMT. For this particular

journal a higher level of detail on the instrument itself (see also

minor remarks below) is needed. I believe that the author has both

data or simulation tools available to better characterize the sampling

uncertainties associated to ICE-CAMERA. I would like for example

to see statistics on the number of particles collected according to the

wind speed (I see data are available in Fig. 8) or according to humidity/

dew point which influences sublimation. The author provides interesting hints about sublimation time for individual particles in section

1.6.2. The author may apply similar techniques over a population

of particles, for example assuming a PSD (particle size distribution)

and repeat the simulations multiple times for multiple PSDs and other

parameters (temperature for example). A first approximation could be

to use a constant PSD/snowfall intensity over the measurement period

or to elaborate more realistic scenarios.

This is only an idea, and the evaluation could certainly be done in

a different way but the objective is the following: by extending the

simulation to a population of particles, something more could be said

in terms of (theoretical) catching efficiency of the instrument, a key

aspect for the readership of this journal.

2. It was not clear if the main focus of the paper is the instrument or

the CNN-based classification. I recommend to clarify it. In either case

the level of detail should be increased (see my other comments). My

assumption was that the main target is the instrument so I did not

comment in much detail about the CNN.

3. The literature overview and the current state of knowledge about ice

crystal habits is not enough documented and detailed in the introduction

and more in general in the text. See minor remarks below. I

would like to see the instrument and its capabilities better put into the

context of current state of research and the known research gaps.

4. Data access. The instrument is collecting data since 2014. So there are

now 7 or 8 years of unique data collected at the Concordia station and

not yet available. The data are the real added value of each innovative

instrument. It is crucial in my opinion that as much data as as possible

is made available, behind a DOI to allow for citation and recognition, at

the time this paper will be published. I see it as a necessary complement

to allow the readers to understand the quality and the scientific impact

of the data produced by the instrument. Platforms as Zenodo, just

to cite a common choice, are easy and user friendly and they allow to

obtain a DOI and a citation for the dataset.



Minor remarks

1. In the introduction or somewhere in the text I would recommend to take

the time to explain all the difficulties and possible failures associated

to the long term operation of an instrument in such an environment.

This is briefly mentioned through the text but I am convinced that

the average reader would not get the idea of the actual and maybe less

known challenges: breaking of rubber parts, (low) humidity issues in

heated environments, etc.

2. L14-19, L22-24: some references should be added to backup these scientific

statements.

3. L29: 90 um → 90 μm. Same through the text

4. L34: change forms to shapes

5. L79: I believe the scanning principle should be briefly introduced before

this point. Here we read after an entire scan, but we actually do not

know yet what a scan exactly is, for this instrument.

6. Fig. 3: I found this schematics not really informative. I would recommend

either to use pictures of the actual parts of the instrument (a

qualitative but illustrative way) or to provide a higher detail electrical

/ communication schematics, even as Appendix material.

7. L139: what is the expected lifetime of the LEDs?

8. L162: add a reference or a statement about the expected maximum

height of blowing snow in Concordia.

9. L177 (and related section):

• Could you show data evidence of partial sublimation bias in the

collected images? A caveat that you mention also later on in the

manuscript.

• The role of relative humidity should be discussed here. If relative

humidity is higher, sublimation rate will be lower.



10. Fig 8: I see some clear patterns / clusters / populations in the data. I

recommend to try to explain why such patterns appear. I suspect they

are related to individual events/seasons, but they may be related to

more interesting physical aspects or technical issues.

Also, I would try to produce similar scatterplots but including relative

humidity, which should be routinely available in Concordia.

11. L294 (and related section): a flow-chart could help to present the processing

steps rather than a bullet list. The text of this section should

be edited to be descriptive rather than a simple enumeration of steps.

12. L331: how often does this situation occur?

13. L352: Why this approach end up to be unreliable for your instrument?

14. L363: does this imply that the images collected have to be resized?

(down or upscaled?) Is the resolution of the instrument constant?

15. Fig 14: can spheroidal particles actually be supercooled liquid water?

It would be good to have some size reference in this image.

16. L402: about trigonal plates. This statement here seems to be contradicted

by the results of Fig. 17, and in fact this is highlighted in L471.

I would either remove the statement here or remove this class from all

the analysis of the paper.

17. L407: a good idea actually.

18. L560-561: I maybe missed it, but are there statistics on the total

amount of data collected since the instrument is operational?

19. L565: I would recommend to clarify explicitly the content of the governmental

policies mentioned here. Nowadays most of the institutions promote

FAIR policies (https://www.go-fair.org/fair-principles/), so this

Code and Data availability statement may be sounding strange for some

readers. Secondly, only the CNN and a test dataset is mentioned here.

The paper, as the title suggests, is not about the CNN but about the

instrument. I would expect to see a data statement about access to

the data collected so far by the instruments and maybe some link to

relevant technical documentation of the instrument itself.



20. Plot quality: the quality of some plots seems very low (for example

Fig. 5)

The paper presents a new instrument to image snow and ice precipitation together with automated image processing. As such it is a worthy contribution to AMT as only few ice particle imaging instruments exist. The instrument is specifically designed for the Antarctic environment.

While interesting details are provided, they are not sufficient to describe a new instrument. Some aspects of the instrument’s performance and the automatic image analysis are not discussed properly yet. Thus, I recommend publication after a major revision. In the following I highlight more important shortcomings first and then list other comments, suggestions, and corrections.

Major comments

=================

1)

Abstract is not adequate.

2)

Particles’ collection?

L77-78: “The principle is simple: at low temperatures and low wind speeds (conditions encountered at DC), precipitation falling over

a flat glass accumulates with time and remains frozen until it sublimates, …”

Language should be improved, but also the collection and issues with collection efficiency should be discussed here or later in the paper. Particles deposit on the DS, but what makes them stay there (“remains frozen”)? What are the effects of wind both during collection (deposition) and after collection (can particles be removed by wind)? Could any biases be introduced based on wind effects or other effects depending on size or shape?

3)

1.2 Camera/1.3 Focusing and 2.1.2 Limitations:

Say something about resolution (resolving power). Pixel size corresponds to 7 um, but resolution does not seem to allow detecting particles smaller than 70um (70um, not 60um as you state in L333, is equivalent to 40000um2).

How much does the focus change between calibrations?

How long does focus calibration take? (Could it be done before each scan instead of 6h intervals?)

Consider including a discussion of accuracy of provided particle sizes.

4)

Thermal control:

L168-170: “The NI-Labview software controls the internal temperature of ICE-CAMERA above -40°C, and the DS temperature always under -5°C (“Heater 1” in Fig.3).”

Is this always possible?

Specify if this is general heating of ICE CAMERA or specific for sublimation (then refer to Sect 1.6.1).

Unclear where Heater 1 and 2 are.

Heater 1 and 2 swapped in Fig 3.

L179-181: “An indoor test (Fig.7), showed a heating of rate of 2.5°C min-1, and a cooling rate of 1 °C min-1. The cooling rate is almost 50% of the heating rate just due to the sandwich heating-glass structure, with the heating layer at the middle.”

“…just due to the sandwich...”: is this passive cooling, what is "heating layer at the middle"?

Is the indoor test really relevant here? You then report the heating/cooling rates outside. So, Fig7 could maybe be replaced with a cycle outside.

In Sect 1.5 you mentioned about heated air between glass sheets (DS and second sheet) and keeping DS frost free; is that the actual heating of the DS?

Detail: how much air pumped inside and through the space between the windows?

Distance between glass sheets?

When air between glass sheets is not pumped, is it sealed or can air circulate?

L192: “Heating is anyway interrupted if the DS temperature exceeds -5°C…”

How often does this happen?

5)

Deposition/scan/sublimation cycle:

The cycle of deposition, scan, and sublimation is not properly explained in detail. The reader gets the details somewhere between the lines.

L190-192: “…sublimation of the majority of particles (D<1000 um) is complete within 20 minutes, with just a few big (D>>1 000 um) grains still present after 30 minutes.

After these tests, the heating time was set at 10 min.”

“D” is not defined.

“within 20 minutes”: 20min after the 10 min heating period (after 10min heating was turned off)?

“few big grains” Can you be more specific? After 30 min grains were larger than 1mm? Or ice particles that original had D larger than about XY mm?

Then you continue to discuss the cycle and for how long the DS is “sensitive to falling ice particles” and that this time is variable depending on conditions and that there is an uncertainty associated with this.

What is in L195 the “evaporative removal of particles during the accumulation period”? Is this wind related; why evaporative now and not related to sublimation; what is accumulation period”.

All these details should be explained more clearly and then the issues discussed clearly too. In particular, make it clear that the deposition phase is not clearly defined (only its end due to the scan). I think I have not seen how long a scan takes. Would scanning more often be a way to study sublimation issues related to smaller particles (100 um)? Would could then see how they sublimate in consecutive scans.

6)

Fig 8:

What measurements does the figure show? For one 1h cycle, how many temperature measurements are shown?

Fig 8 left) at what wind speed?

+25degC at still air, +20degC at 8m/s: are those temperatures averages, median, or ? (I can see a large range, maybe 20degC to 35degC.)

Wide spread in blue; two regions (sublim and deposition phases)?

Left: better also show dT instead of DS temperature.

Could Fig8 be separated in one fig. only the deposition periods of 20 min prior to scan, one for after/during sublimation heating? This could be clearer instead of heaving heated and not heated temperatures mixed in one figure.

7)

Sect 1.6.2 Sublimation of ice particles

You discuss sublimation, both unwanted and wanted sublimation here. When saying “negative effect” you refer to the unwanted sublimation. Be careful to keep clarity in this section.

L214: “as DS ice is always super-saturated relative to the surrounding air”

Wrong as it is stated. It is also unclear what "DS ice” refers to.

At the heated DS the equil. vap. pressure will be higher; thus the surrounding air will have a lower RH at the heated DS and ice on the DS will experience sub-saturated conditions.

L215-216: “the vapour pressure of ice on the DS relative to the surrounding air (saturated relative to ice…”

Wrongly or unclearly stated: "the vapour pressure of ice on the DS..." should be something like:

"the vapour pressure saturated relative to ice at the DS temperature".

Fig 9: Why show this ratio (e0ice(T_DS)/e0ice(T_amb))?

More intersting would be the inverse (e0ice(T_amb)/e0ice(T_DS)), i.e. RHice at DS if surrounding air is at RHice=100%. For dT=5K this would range between 45% at -80degC to 65% at -10degC. It shows the sub-saturation. Fig 9 is not further discussed or used later on. If you want to keep it, I suggest to plot inverse.

You use over saturation, oversaturation, over-saturation, and super-saturation. Only use one term for clarity, I would suggest super-saturation.

L228: “growth rate of the facial area” Growth perpendicular to facets?

L229-230: Unclear what critical super-saturation and critical temperature are.

L231 “0.05% to 5% under-saturation”. How is “under-saturation” defined here?

L234 “The steady-state shape of the sublimating crystal depends”

Unless explained or defined, it seems contradicting: a sublimating particle is not in steady state.

L246-247 “The simulations assume the completion of preliminary sublimation of points and edges of the particle”

Unclear language (preliminary, points, edges).

L252 “still evaporate” Unclear what the “still” refers to => “sublimate”.

L253 “could survive along the heating period” => “will survive longer than the heating period”?

L259: Meaning of "After the sublimation period, DS is exposed to falling crystals."?

DS is always exposed. Particles depositing during the sublimation period may partially sublimate and then be included in "collected" particles. (see comments for Deposition/scan/sublimation cycle above)

Fig11: Do you have any experimental data on sublimation times for actual particles (similar to your spheroids)? That would help to set your statement in L270 in relation, or that you only see few particles with partial sublimation.

8)

Sect. 2.1.2

1) Specify better: 2000 per scan?

2) Explain what you mean with “By default”. Relate to resolution (see 3) above).

3) Specify the segmentation error (particles are overlapping?). How can a particle be counted twice (double counting)? What is 12% in your worst case (overlapping particles account for 12% of particles in one scan?)?

4) Two issues are brought up together but are rather two limitations that should be listed/discussed separately. It is unclear how and on what basis these spheroidal particles are disregarded.

9)

Sect 2.2.2 Training dataset

I would like to see a better description of the various datasets.

What is the “image dataset” (L437)? How many images; from what time period?

How have these images been selected out of all available images? Randomly or by some other selection criteria?

This image dataset is then apparently split in 10% validation dataset and 90% training dataset (25705 images).

What about a testing dataset?

Say how testing (see Sect 3) was performed, in Sect 2.2.2 or Sect 3 for example. How many and which particles were used to test the CNN after training is complete, i.e. to produce the results shown in Sect 3?

Provide more details about the augmentation: To what size is the original training dataset of 25705 images increased after augmentation? Each class augmented the same (number of images increased by same factor)?

10)

Biases during training

The number in each class is different. This can generate biases in the CNN, please comment. In particular, I think that part of the low performance for trig plates is due to the fact that this is the smallest class in the training dataset.

11)

Data

A few example images in the full resolution should be included in the paper. This will help when discussing resolution and sizing accuracy (see Major point 3) above).

Reconsider if you interpreted government policies with respect to sharing data correctly. It would be useful to share the whole image dataset. Should that not be possible, then please share at least a sample dataset.

Minor comments

=================

Consistency with terminology and spelling in various places.

E.g.

GoogLeNet

dataset

I would expect that Introduction is Sect 1.

Sect 2 is on instrument; 2.1 Overview (instead of 1.1)....

L69: “In this work, the term ‘precipitation' will include both diamond dust and”

Should perhaps refer to “precipitating diamond dust”.

“cooling speed” => “cooling rate”

“eventual” => “occasional” (2 x)

“deposition window” => “DS”

“By the way” is not good English for an article.

Fig 12: “Reg.growed” Use correct label and be consistent with text.

Fig 13: Remove title “ICE-CAMERA: Summary of detected grains…”

Be consistent: what is synthesis image?

summary-image

synthesis image

mosaic, summary image

L325 “measures” is not a noun.

L325 and L83: specify what weather data are.

L384 “hexagonal prism”

I would call this class "long column" or something else with "column"

Fig 14 “compact prism” should be “compact column” (as in L395).

L401: Why refer to “insoluble”?

L451 “3.1 Precision of the classifier” should be “3.1 Accuracy of the classifier”

L460 “mistaken” = “mistaken with each other”

L 462 “Compact columns are misclassified almost 20%”

I read 15.4% in the bottom row, not 20%.

L473-474 “The three-dimensional structure of the ice particles is lost in the ICE-CAMERA images, so that some thick ice forms such as C4a, P1b, G3b, CP1a, etc. (Kikuchi et al, 2013), if any, are likely to be misclassified”

Be a bit more specific (or give at least one example).

L498-499

“The classifier was used…” You are showing results of the classifier but then also of the "image measurements" for the resulting classes.

Is the Jan-Feb 2017 period not part of the image dataset used for training/validation and testing?

L506 “maximum whisker length is here equal to the interquartile” seems wrong.

Total box height is equal to the interquartile range. What do the whisker lengths indicate?

L507 “some relevant differences” seems to refer to one relevant difference.

Fig 20: can you include a grid line for 100um?

L532 "diameter of the circle equivalent to the bounding box"

definition is ambiguous:

"diameter of the circle with the same area as the bounding box

Similarly, in L533: ambiguous definition.

Fig 23: Suggestion: put (as whiskers only) on scatter plot Feret-box vs projected-surface equiv. diameters. In that way direct visual comparison is possible.

L543-544 “Commercial or customized instruments do not have this flexibility, more typical of old-style handcrafted products”

Purpose of this comparison, what instruments are you comparing to?

L548 “convolutive” => “convolutional”

L552 “precisely” Do you mean precise here, or accurate?

Anyway, neither of the two was discussed.

See Major point 3) above.

Concluding remarks on potential improvements (in particular with respect to the sublimation problem). Use of ICE-CAMERA in other environments (not Antarctica)?