Review: “TROPOMI/S5P Total Column Water Vapor Validation against AERONET ground-based measurements” by Garane et al.

General comments

In this study, Garane et al. validate a new TROPOMI TCWV product by Chan et al. (2022) using 2.5 years of ground-based measurements from AERONET as reference. They also investigate the influence of different input variables (e.g., viewing geometry, surface albedo, clouds) and the retrieval results themselves (i.e., H₂O SCD and AMF) on the retrieval performance.

Although the overall aim of the paper is interesting, I have some concerns about the validation analysis. I also feel that the authors miss an opportunity by not taking advantage of the high density of AERONET stations in certain regions and instead averaging the results zonally, which unfortunately also results in a lot of information being lost.

That being said, I recommend publication if the following points and concerns are addressed.

Major issues

1. It is not completely clear how the collocation of ground-based and satellite measurement was conducted. If I look at the total number of data and roughly calculate, I results in about 2 measurements per day per station (633000 / (365*351*2.5) ~ 2). And here all filters are already taken into account (CF<0.5, AMF, RMS, SZA). In my opinion, this number seems unexpectedly high, especially considering that not all stations were able to provide measurements for the most recent months. The authors must explain the collocation procedure more clearly: were several satellite pixels (within 10km to the reference station) compared with one measurement from AERONET? Or is simply a large fraction of the collocated measurements at stations at high latitudes? If the former is the case, then some reference measurements would be used more often than others, making the comparison inconsistent. I would therefore suggest either to take only the closest satellite pixel within 10km or to calculate the mean value of all satellite pixels within 10km and compare it with AERONET.

Furthermore, negative TROPOMI TCWV values also appear in the comparisons (for instance in Figure 3b and 6b). I would ask the authors to clarify where these negative values come from.

2. The use of zonal means does not make much sense, as TCWV has a high variability along a longitude. This also negates the great advantage of AERONET, namely the network’s high station density. I suggest the authors to restructure Section 4 as follows:

• One should carry out the analysis of TROPOMI vs. AERONET (i.e. the regression analysis) for each station individually and then present the fit results on a world map.
• For regions with a high station density (e.g. Europe and North America), separate plots could be shown separately.
• In the corresponding regions, one could interpolate the regression results of all stations and then analyse how well the performance of the retrieval depends on geophysical parameters, i.e. performance in humid/arid areas, influence of albedo, ground elevation, etc. This would also lead to an overall better understanding of the retrieval.

3. It is not really clear why AERONET is used at all when the authors themselves say that AERONET most likely underestimates the actual TCWV content (see Section 2.2) and the quality of the measurements also strongly depends on the calibration of the instrument on site (see line 166f). What is the great benefit of AERONET compared to other measurement networks? GPS measurements from SuomiNet or IGS, for example, have a much higher accuracy than AERONET and can also measure under all-sky conditions.

Other

Overall, the quality of the figures needs to be significantly improved: Instead of point clouds, 2D histograms should be used (e.g. Figure 5 or 6b). The numbers in Figures 10-13 are hardly readable. It might be better to show the amount of data points using a colorbar with coloured dots. 

Moreover, the language should be improved. Here and there the wording is not really appropriate. For example, "quantity" should be replaced by "variable" and "percentage difference" by "relative difference" throughout the paper.

Specific comments

L11: “blue wavelength band” to “visible blue spectral range”

L12: MetOp

L18: -3%: Table 2 shows much higher values (-4 to -10% in NH, +2 to +6% in SH).

L21: “low cloudiness” --> low cloud heights

L23: “-4 +- 4.3 % with the ground-truth”: In Section 5 it is written that it is -9 to -13%. Accordingly, one should write here that it is -4% in relation to AERONET, but probably -9-13% to the "truth". The slope of the linear fit in Figure 6b (Section 4) gives a value of 0.89. So the retrieval actually underestimates by about -10%?

L28: Water vapour does not have to form clouds to be transported around the globe.

L40: It should be mentioned that a major source of stratospheric H₂O is methane rather than tropospheric H₂O.

L43: Here some exemplary instruments (and corresponding papers) should be mentioned. GPS radio occultation is missing in the list.

L50: It is a bit strange to mention Schneider et al. (2020) and not refer to the other TCWV retrievals from TROPOMI by Borger et al. (2020) and Küchler et al. (2021). In particular considering that Borger et al. (2020) also retrieve TCWV in the visible blue.

L70: TROPOMI was launched in October 2017.

L80: DOAS: reference missing (e.g. Platt and Stutz, 2008)

L84: Which improvements have been implemented in the spectral analysis and the AMF calculations?

L92: Not really necessary to mention the data format.

L114: Replace “utilized” by “used”

L128: Is the reference source now Martins et al. (2019) or Smirnov et al. (2004) and Alexandrov et al. (2009)?

L131: “coverage of all continents”: looking at Figure 2 only North America and Europe are covered well. Please rephrase.

L140: How does the “in-house quality control” look like? Please clarify.

Section 3: Here it would be more interesting to show an example from another climate zone rather than showing two similar stations. Replace one of the examples with another one (maybe from the northern mid-latitudes, where most of the AERONET stations are located).

Figure 3b: Is the value shown for the correlation the Pearson’s correlation coefficient R or or the coefficient of determination R²? And is the linear fit based on ordinary least squares (OLS) or total least squares (TLS)? Since the uncertainties of TROPOMI and AERONET are of comparable magnitude, a TLS might be more appropriate.

L165: 0.788 < 0.79

L166: Shown is the monthly mean bias, right?

L168: The period of 2.5 years is much too short to speak of a high temporal stability, especially if the time series also has some gaps.

L185: With the high number of measured values, the standard error does not provide any additional, relevant information and should therefore be removed from all comparisons.

L196: A slope of 0.89 basically means underestimation of more than 10%? Has a TLS been used? Negative TCWV values in TROPOMI comparison. Please clarify, where they come from.

L207: “temporal stability”: see comment above

L212: “mainly representing the latitude belt 0° to 60° S”: redundant information, as only there is only one station in Antarctica.

Table 2: What has been the reason for the different latitude binning?

L265: “The performance mainly on the aspect of the surface albedo parameter credibility appears to be sound.”: Considering that the albedo is an elementary input parameter of a satellite retrieval, the performance should not be called “sound”. Rather acceptable?

L285: Instead of looking at the VZA dependence, it might be better looking at the row dependence so that one can see if a West-East dependence exists in the TROPOMI swath.

L298ff: Please clarify how these dependencies compare to the findings in Borger et al. (2020)?

L315ff: So why do you investigate surface pressures lower than 900hPa, when you only have a limited number of measurements?

L318ff: Since there are hardly any measurements for albedos > 20%, this should also only limit to 20%, but make the binning finer.

L360: “10-19%”: this is only valid for TCWV in the tropics. Likely much higher in the mid-latitudes.

L375ff: Since there are almost no measurements in the polar regions at high latitudes, the statistics are not meaningful.

L382: “cloudiness” --> cloud top height

L386: “temporally stable”: see comment above

L386f: “product of high quality and precision, temporally stable and not affected by any other parameters, except from clouds”: This is a contradiction in terms, considering that a few lines earlier it is mentioned that the TCWV retrieval likely underestimates by 10-13%, the time series is far too short for analyzing temporal stability and clouds are by far the most important input parameter in satellite retrievals.

L403: Competing interest: To the best of my knowledge, DL is an editor of AMT, which, according to journal’s publication guidelines, should also be mentioned here.

L448: Apparently, the reference from Kleipool et al. was mixed with another from Köehler et al.

General Comments

This manuscript presents a validation study of the new TROPOMI Total Column Water Vapor (TCWV) retrieved from the 435-455 nm wavelength region.  Validation involves comparing 2.5 years of data with AERONET Level 2 precipitable water vapour measurements.  Comparisons are performed globally and in several zonal bands to determine the mean bias.  The impact of viewing geometry (solar zenith angle, viewing zenith angle), cloud properties (cloud top pressure, cloud albedo, cloud fraction), and retrieval parameters (surface pressure, surface albedo, air mass factor, fit RMS, water vapour) on the comparisons are also examined.

The analysis is straightforward and provides a useful contribution to the evaluation of TROPOMI water vapor.  However, the paper would be strengthened by making greater use of the validation dataset, examining the differences across the 351 stations rather averaging across zonal bands.  Overall, I recommend publication after the comments below are addressed.

Specific Comments

Page 2, second last paragraph:  The previous paragraph mentions the TROPOMI SWIR TCWV data product and its validation against TCCON.  Why wasn’t this new TROPOMI TCWV product compared against the SWIR product?  Why wasn’t TCCON included, or other available water vapour datasets such as GPS/RO?  Explain why AERONET was chosen as the comparison dataset for this validation study. 

Page 6, line 147: Define the equation used to calculate percentage difference, e.g,. 100 x (TROPOMI – AERONET) / AERONET, so that this is clear.  Also, this quantity should be called the relative difference.  Is “minimize the noise” the best description?  The choice of coincidence criteria is a trade-off between maximizing N for better statistics and minimizing space and time differences between the comparison datasets.  No real justification is given for the choice of 10 km and 30 minutes; have these values been used in other water vapour validation studies or were trade-off curves constructed to find the optimum criteria?  Do the comparisons involve single or multiple pairs, i.e., is each TROPOMI measurement compared with the closest AERONET measurement (or vice versa), or are multiple comparisons allowed the space and time criteria are met for a TROPOMI measurement and multiple AERONET measurements (or vice versa)?

Page 7, line 156: The 633,000 coincident measurements constitute a rich validation dataset that could be investigated in more detail.  This large number would seem to be the justification for using AERONET for the validation, but the analysis doesn’t take full advantage of the resulting information.  Since a per-station analysis has already been done, there are 351 global comparisons – it would be interesting to examine these and to look more carefully for spatial differences and dependencies.  For example, consider adding a panel to Figure 8 that shows the seasonal and latitudinal variability of the mean bias, and a similar figure showing latitudinal and longitudinal variability, using the results from all 351 stations.  Another panel that could be added to Figure 8 is the seasonal and latitudinal variability of N, given the discussion on lines 228-233.

Page 7, line 160 and Figures 3 and 4:  State why monthly mean percentage differences are calculated and plotted (perhaps to provide more even annual coverage?).  If the mean bias and standard deviation are calculated using all individual points for a station, do the results differ from those obtained using the monthly means?

Page 7, line 167: “The mean relative bias per station (panels a) depends strongly on the ground-based instrument’s calibration,”  Since all AERONET data is Level 2.0, what calibration issues are there?  Nothing has been said about this elsewhere in the paper so this should be explained.  Is there a parameter defining the calibration status for each station so that the dependence of the mean relative bias on the calibration can be determined?

Technical Corrections

Page 1, line 9: here and elsewhere throughout the paper, delete “very”.  It is used too frequently and is not needed.

Page 1, line 11: (435-455 nm)

Page 1, line 15: although AERONET has 1300 stations, data from only 351 are used in the study – this should be noted here

Page 1, line 18: “of the order of only -3% for the mid-latitudes and the tropics (+-60deg)” does not seem consistent with the mean bias numbers in Table 2 which are -4.0, -5.9, -9.6, -5.9, 2.4, and 5.8 for the six bands between 60N and 60S, nor with the NH, SH, and global biases in Table 1.  Provide more specific results here.

Page 1, line 19: delete “influence”

Page 1, line 21: define CTP, clarify what “low cloudiness” means – low cloud top pressure?

Page 1, lines 25-29 and elsewhere in the manuscript (lines 33, 70, 72, 208, etc.): change “earth” to “Earth” (the planet) throughout

Page 2, line 31: delete “very”

Page 2, line 34: remove/replace one of the “therefore”s

Page 2, line 37: delete line break

Page 2, line 38: delete “very”

Page 2, line 40: high-latitude

Page 2, line 41: delete “key” (already say “important”)

Page 2, line 41: “for the evolution of the greenhouse effect and the projection …”

Page 2, line 41: climate change

Page 2, line 45: (435-455 nm)

Page 2, line 45: delete “further”

Page 2, line 47: delete “sensors”

Page 2, lines 50, 54: clear-sky

Page 3, line 66: delete “influence”

Page 3, line 70: TROPOMI was launched on 13 October 2017

Page 3, line 79: (435-455 nm)

Page 3, line 81: “in short, a two-step approach …”

Page 3, line 84: air mass factor

Page 3, lines 95-96: the seasons listed here (winter, spring, summer, autumn) only apply to the Northern hemisphere – either add NH before each season or remove the seasons from this sentence.

Page 4, line 99: “decreasing below 5-10 kg/m2 closer to the poles.”

Page 5, line 116: product

Page 5, lines 130-131: for the 351 stations used in this study, AERONET coverage of all continents is not actually “very dense” spatially as seen in Figure 2 – revise this description.

Page 5, line 131: delete “very”

Page 6, line 140: “resulted in the reduction …”  Describe the in-house quality control that reduced the number of stations with usable data from 1300 to 351.

Page 6, line 142: “as can be …”

Page 6, Figure 3 and page 7, Figure 4: The quality of the fonts is poor on these panels and hard to read – the fonts should be improved.

Page 7, line 160:  Why show monthly mean percentage differences

Page 7, line 163: delete “very nice” (here and elsewhere, these subjective descriptions can be removed)

Page 7, line 164: state whether the correlation coefficient is R or R^2

Page 7, lines 165 and 167: delete “very”

Page 8, Figure 5: The quality of this figure is poor; at a minimum, the y-axis should be extended beyond +-90 so that the highest latitude points are visible, the fonts should be improved, and “AERONET” removed from the top.

Page 8, line 180: 60S is mentioned here (and again on line 212) but line 177 says that the SH data only extend to 55S – should 60 be changed to 55?

Page 8, line 183: averaged

Page 9, line 191: on a global scale

Page 9, line 192: change “about” to “approximately”

Page 9, line 195: correlation coefficient R?  delete “very”

Page 10, Figure 7: the legend (TROPOMI) on the lower left of the panels and “AERONET” in the lower right should be deleted.  State in the figure caption what the errors bars are.

Page 11, Figure 8: delete “TROPOMI” and “AERONET” from the panel

Page 12, lines 235, 236, 245, 252: delete “very”, etc.

Page 12, line 253: 80deg S to 90deg S

Page 12, line 261: location

Page 12, line 264: regarded as [very] good.

Page 12, line 265: rewrite this sentence for clarity – it is not clear what is meant

Page 13, line 273: delete “influence”, change “quantities” to parameters or variables

Page 13, line 274-275: “detailed results” is a strange term – how is air mass factor a detailed result?  Are these outputs from the retrieval algorithm? 

Page 13, line 276 and Figures 10-13: the numbers at the top of each panel are unreadable – revise these plots to show this information in another way.

Page 13, line 276: change “of each figure” to “Figures 10-13”

Page 13, line 280: specify whether the SZA and VZA are for TROPOMI or AERONET.  Define what is meant by the viewing zenith angle.

Page 13, line 281: delete line break

Page 13, line 285: it’s not clear what is meant by “the dependence of the percentage differences on SZA is ~13%”.  The differences in Figure 10(a) vary from approximately -10% to +10% so where is 13% coming from?  Has a line been fitted to the data? If so, should it be added to the plot?  Discussion of these results should be clarified.

Page 13, line 285: “of the mean increases for larger/smaller SZA”  “higher SZA” is ambiguous – specify whether larger or smaller

Figures 10-14: Improve the presentation of numbers as noted above. Delete the legend (TROPOMI) on the lower left of the panels and “AERONET” in the lower right.  State in the figure caption what the errors bars are.  Revise the figure captions so that they are consistent between figures and fully describe what is shown in the panels.

Page 15, lines 297 and 304-307: discuss the dependence on cloud fraction above 0.3

Page 16, lines 326-327: change the title of this section – “Detailed results” is not informative.  Are these outputs from the TROPOMI retrieval algorithm?

Page 16, line 332: from Figure 13(a), the largest negative value for AMFs of 2-4 looks like approx.. 25%, not 18%

Page 16, line 336: “with a low …”

Page 16, line 338: delete “result”

Page 16, line 342: areas

Page 16, line 343: this sentence is unclear – what does “they” refer to and what “should be treated with caution?  Revise for clarity.

Page 17, line 346: “namely: TROPOMI (a)…”

Page 17, line 348: is “statistics” needed in the title of this section?

Page 17, line 350: consistency with

Page 17, line 353: corresponding to clear-sky

Page 17, line 355: summarized as follows

Page 18, line 358: delete “excellent”, change “their” to “the”

Page 18, line 359: mean bias is -4.7%

Page 18, line 363: should “accuracy” be “consistency”?

Page 18, line 381: does “low cloudiness” mean low cloud top pressure?

Page 18, line 386: Is 2.5 years of comparisons sufficient to claim temporal stability?  What is the basis for claiming high precision?  Accuracy appears to be -9% to -13% based on line 365.  Revise this sentence.

Page 18, line 388: delete “very”

Page 18, line 389: list other “blue-band satellites”