Accounting for meteorological biases in simulated plumes using smarter metrics

Vanderbecken, Pierre J.; Dumont Le Brazidec, Joffrey; Farchi, Alban; Bocquet, Marc; Roustan, Yelva; Potier, Élise; Broquet, Grégoire

doi:https://doi.org/10.5194/amt-16-1745-2023

Articles | Volume 16, issue 6

https://doi.org/10.5194/amt-16-1745-2023

© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/amt-16-1745-2023

© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 16, issue 6

Research article

|

31 Mar 2023

Research article |

| 31 Mar 2023

Accounting for meteorological biases in simulated plumes using smarter metrics

Pierre J. Vanderbecken, Joffrey Dumont Le Brazidec, Alban Farchi, Marc Bocquet, Yelva Roustan, Élise Potier, and Grégoire Broquet

Download

Final revised paper (published on 31 Mar 2023)
Preprint (discussion started on 04 Nov 2022)

Interactive discussion

Status: closed

RC1:
'Comment on amt-2022-226', Anonymous Referee #2, 09 Nov 2022

The paper proposes a new measure for forecast performance that accounts for displacement methods. Overall, the methodology seems sound, but per comment 1 below, it is unclear if this approach is really new, or what the new contribution is. Moreover, it is unclear if the added complexity of the approach over the displacement methods (e.g., as discussed in doi: 10.1175/MWR-D-19-0256.1) adds enough value to warrant its use. Therefore, I recomment acceptance after the authors consider the comments below.

1. The method proposed is very similar to several of the field deformation approaches described by the cited Gilleland et al. paper and several since that time: e.g., see doi: 10.1016/S0022-1694(00)00343-7, doi: 10.1175/2010WAF2222365.1, doi: 10.5065/D62805JJ, doi: 10.1002/2012GL053964, and doi: 10.1175/2010WAF2222351.1 to name just a few. In particular, doi: 10.3402/tellusb.v68.31682 uses the Wasserstein distance. A thorough literature review and comparison of the differences and added utility of the present approach is necessary to put this work into the greater context of these deformation methods. As it is, it is not clear what the new contribution is over these other works.

2. How does this approach address the issues outlined in doi: 10.5065/4px3-5a05 ?

3. The authors make reference to the measure's being fairer, but it is unclear what they mean by fair in the general concept of a fair verification measure.

Citation: https://doi.org/10.5194/amt-2022-226-RC1
- AC1: 'Reply on RC1', Pierre Vanderbecken, 07 Jan 2023
  
  First, we would like to thank you for your review and the relevance of your remarks and suggestions. You will find your answers in the attached file.
  Best regards,
  The Authors
  
  Citation: https://doi.org/10.5194/amt-2022-226-AC1
RC2:
'Comment on amt-2022-226', Anonymous Referee #1, 21 Nov 2022
The authors discuss how to compare satellite observations to simulated concentrations by limiting the weight of modelling errors due to the meteorology used to analyze the observations. The manuscript presents a lot of equations to describe the math behind the method. I’m not sure I fully understand all the details, particularly section 3. But the work generally looks sound to me. I recommend the following revision.

General comments:

Our view is that meteorology drives the position error between the plume observed and the plume simulated by the CTM. This is the motivation of the work. However, I don’t see how solid it is. There are several contributors to the errors. I don’t see the reason why meteorology is the driver.

I recommend the author to add a flow chart to demonstrate the method.

Specific comments:

Line 7. It shall be analyze instead of analyse.

The concept of pixel-wise norm has been proposed without giving any introduction. Same as double penalty issue, upstream correction, non-local matric optimal transport theory. It will be difficult for readers without strong background for this very specific field to follow. I understand that it is difficult to give the definitions for all those items in a short abstract. I would like to encourage the authors to reconsider the necessity of keeping all those items and the possibility of rephasing the paragraph in a more reader-friendly way.

Line 40-45. Meteorology is not the only contributor to modelled bias. Such information seems missing from the text.

Line 51. What is “position error”?

Line 47. “the relative weight of the meteorological uncertainties within the comparison between observation and simulation cannot be easily removed through pixel-wise comparison”. I don’t quite understand the meaning of this sentence. It sounds like the aim of the comparison performed at pixel level is removing meteorological uncertainties. Please try to rephrase it. Same for “This issue is shared in other fields”. I’m not sure the sentence is clear to readers.

Line 56. What is droplet or analogous decomposition? Please try to define before use.

Line 58. What does “fileds” represent here?

Line 64. What is a moving field?
Citation: https://doi.org/10.5194/amt-2022-226-RC2
- AC2: 'Reply on RC2', Pierre Vanderbecken, 07 Jan 2023
  
  Thank you for your review and for your comments and suggestions on the manuscript. You will find your answers in the attached file.
  Best regards,
  The Authors
  
  Citation: https://doi.org/10.5194/amt-2022-226-AC2

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload

AR by Pierre Vanderbecken on behalf of the Authors (07 Jan 2023) Author's response Author's tracked changes Manuscript

ED: Publish subject to minor revisions (review by editor) (04 Feb 2023) by Lok Lamsal

Dear Authors,

Thank you for submitting your revision and responses to reviewer's comments. I have now read, with great interest, the original manuscript, reviewer's comments, and revised version of the manuscript. I find that the work is interesting and it should be of value to the AMT readership.

Based on my own reading, I feel that the manuscript needs some work mostly clarifying some statements and describing figures in the figure caption. AMT might edit the manuscript, but it is important to ensure that the statements are clear and unambiguous. Please check the statements that I have listed below and improve them as necessary for clarity. I look forward to receiving your revisions soon.

Best Regards,

Lok

========
1) Title: Shouldn't it be "meteorological biases" instead of "meteorology biases"?

2) Line 11: Please revise the statement: "To circumvent this issue, we propose to either..... both".

3) Line 21: Please revise the statement: "It is found that discrepancies between two plume images due to wind direction errors in the meteorological conditions are less penalised by our new metrics with the upstream correction than without, thus avoiding the double penalty issue."

4) I am aware that some people use "spectra-imagery", but this can be misleading. I believe the authors meant "satellite imagery".

5) Please define the acronym "CO2M".

6) line 40: please revise the statement: "These fast methods require only the images to provide an estimation of the emissions, but, they do so, by assuming either simplified chemistry, transport or temporal variations of the emissions."

7) line 54: please revise the statement: "This issue is shared in other fields ....... comparisons"

8) line 56: please revise the statement: "Assuming that the temporal .....inversion"

9) line 81: please revise the statement: "We will either consider isometry ....... compared."

10) line 104: please revise the statement: "In the present article, we focus on two-dimensional images – typically of the total column of CO2 concentration, or of ground level concentration field –, full (no mask due to filtered data or clouds), with a discretisation of N pixels".

11) line 144: please revise the statement: "The idea is that, instead of considering the cost of the translation, the metric adds the cost to set to zero all pixels from the first Gaussian puff to the cost to enhance the pixels at the translated location."

12) line 149: please revise the statement - it look incomplete sentence: "More practically, the difference between the sum of the compared image pixels."

13) line 176: please define the acronym: "L-BFGS"

14) line 228: please revise the statement: "This shows that the Wasserstein distance w is a softer metric ............. positions."

15) line 254: I cannot see any equation with the division sign as mentioned here: "where ÷ is the entry-wise division in RN."

16) Page 13, Algorithm 1: I suggest to move this algorithm in the appendix section as I see it as a distraction.

17) line 339: please define the acronym "POT".

18) Figure 5: please expand the figure caption. It does not mention what the box, whisker, and red lines are representing. What are x- and y-axis and what are the units?

19) line 383: please revise the statement: "Consequently, the later in the day the plumes are emitted the shorter they are tracked."

20) line 386: please revise the statement and please pay attention to "between 50 km and km" in this sentence: "The horizontal grid resolution of the simulation domain (longitude: 6.82°W to 19.18°N; latitude: 42.0°N to 56.39°N, Fig. 6, Santaren et al., 2021)
varies between 50km and km".

21) Figure 6: please expand the figure caption to describe all we can see in the figure. What do you mean by "Domain Santaren and al." mean?

22) line 394: please make sure that the reference is cited correctly. "Denier" sounds like the first name of the author.

23) line 399: please revise the statement for clarity: "We ensure that the same daily profile is applied to the source emission, then for a given hour of the day, the difference between two simulated plumes is the meteorological state".

24) Figure 7: please expand the figure caption describing the box, whisker, and the colored line. Also please check what y- and x-axes are and what the units are.

25) line 416: please revise the statement: "which is why we do not considered thereafter wnum in our comparisons".

26) line 418: You may have meant "as" in place of "than".

27) line 537: please revise the statement "For the case of the pixel-wise distance with an upstream correction, this can have an impact on the optimum". Optimum what?

28) line 545: "behaviour smoother". Did you mean "smoothing behavior"? Please revise the statement as necessary.

Hide

AR by Pierre Vanderbecken on behalf of the Authors (13 Feb 2023) Author's response Author's tracked changes Manuscript

ED: Publish as is (19 Feb 2023) by Lok Lamsal

Dear Authors,

Thank you for submitting your revision and responses to reviewer's comments. I have now read, with great interest, the original manuscript, reviewer's comments, and revised version of the manuscript. I find that the work is interesting and it should be of value to the AMT readership.

Based on my own reading, I feel that the manuscript needs some work mostly clarifying some statements and describing figures in the figure caption. AMT might edit the manuscript, but it is important to ensure that the statements are clear and unambiguous. Please check the statements that I have listed below and improve them as necessary for clarity. I look forward to receiving your revisions soon.

Best Regards,

Lok

========
1) Title: Shouldn't it be "meteorological biases" instead of "meteorology biases"?

2) Line 11: Please revise the statement: "To circumvent this issue, we propose to either..... both".

3) Line 21: Please revise the statement: "It is found that discrepancies between two plume images due to wind direction errors in the meteorological conditions are less penalised by our new metrics with the upstream correction than without, thus avoiding the double penalty issue."

4) I am aware that some people use "spectra-imagery", but this can be misleading. I believe the authors meant "satellite imagery".

5) Please define the acronym "CO2M".

6) line 40: please revise the statement: "These fast methods require only the images to provide an estimation of the emissions, but, they do so, by assuming either simplified chemistry, transport or temporal variations of the emissions."

7) line 54: please revise the statement: "This issue is shared in other fields ....... comparisons"

8) line 56: please revise the statement: "Assuming that the temporal .....inversion"

9) line 81: please revise the statement: "We will either consider isometry ....... compared."

10) line 104: please revise the statement: "In the present article, we focus on two-dimensional images – typically of the total column of CO2 concentration, or of ground level concentration field –, full (no mask due to filtered data or clouds), with a discretisation of N pixels".

11) line 144: please revise the statement: "The idea is that, instead of considering the cost of the translation, the metric adds the cost to set to zero all pixels from the first Gaussian puff to the cost to enhance the pixels at the translated location."

12) line 149: please revise the statement - it look incomplete sentence: "More practically, the difference between the sum of the compared image pixels."

13) line 176: please define the acronym: "L-BFGS"

14) line 228: please revise the statement: "This shows that the Wasserstein distance w is a softer metric ............. positions."

15) line 254: I cannot see any equation with the division sign as mentioned here: "where ÷ is the entry-wise division in RN."

16) Page 13, Algorithm 1: I suggest to move this algorithm in the appendix section as I see it as a distraction.

17) line 339: please define the acronym "POT".

18) Figure 5: please expand the figure caption. It does not mention what the box, whisker, and red lines are representing. What are x- and y-axis and what are the units?

19) line 383: please revise the statement: "Consequently, the later in the day the plumes are emitted the shorter they are tracked."

20) line 386: please revise the statement and please pay attention to "between 50 km and km" in this sentence: "The horizontal grid resolution of the simulation domain (longitude: 6.82°W to 19.18°N; latitude: 42.0°N to 56.39°N, Fig. 6, Santaren et al., 2021)
varies between 50km and km".

21) Figure 6: please expand the figure caption to describe all we can see in the figure. What do you mean by "Domain Santaren and al." mean?

22) line 394: please make sure that the reference is cited correctly. "Denier" sounds like the first name of the author.

23) line 399: please revise the statement for clarity: "We ensure that the same daily profile is applied to the source emission, then for a given hour of the day, the difference between two simulated plumes is the meteorological state".

24) Figure 7: please expand the figure caption describing the box, whisker, and the colored line. Also please check what y- and x-axes are and what the units are.

25) line 416: please revise the statement: "which is why we do not considered thereafter wnum in our comparisons".

26) line 418: You may have meant "as" in place of "than".

27) line 537: please revise the statement "For the case of the pixel-wise distance with an upstream correction, this can have an impact on the optimum". Optimum what?

28) line 545: "behaviour smoother". Did you mean "smoothing behavior"? Please revise the statement as necessary.

Hide

AR by Pierre Vanderbecken on behalf of the Authors (20 Feb 2023) Author's response Manuscript

Short summary

Instruments dedicated to monitoring atmospheric gaseous compounds from space will provide images of urban-scale plumes. We discuss here the use of new metrics to compare observed plumes with model predictions that will be less sensitive to meteorology uncertainties. We have evaluated our metrics on diverse plumes and shown that by eliminating some aspects of the discrepancies, they are indeed less sensitive to meteorological variations.