the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 22 Nov 2023
Singular Vector Decomposition (SVD) of satellite datasets: relation between cloud properties and climate indices
Abstract. We describe a technique using singular vector decomposition (SVD), that can identify the spatial patterns that best describe the temporal variability of a global satellite dataset. These patterns, and their temporal evolution, are then correlated with established climate indices. We apply this technique to datasets of cloud properties over three decades, derived from five visible/IR imagers ((A)ATSR, SLSTR-A/-B and MODIS and jointly from the IR and microwave sounders on MetOp (IASI, MHS,AMSU-A), but it can be more generically used to extract the pattern of variability of any regular gridded dataset such as different parameters from satellite products and models. The leading singular vector for these three independent global data sets, on both cloud fraction and cloud-top height, from these polar orbiting satellites covering different time periods, is found to be strongly correlated with the ENSO index. The SVD approach could potentially offer a new tool for using global satellite observations in assessing global climate model (GCM) performance.
- Preprint
(3705 KB) - Metadata XML
- BibTeX
- EndNote
Status: closed
-
RC1: 'Comment on amt-2023-232', Anonymous Referee #1, 14 Dec 2023
This scientific paper presents the application of a statistical (or algebraic) technique that identifies that portion of data in a time series that is useful in interpreting the dynamics of the atmospheric system through correlations with climate indices. Specifically, the technique presented is one of the possible belonging to the group of the Eigen techniques, the Singular Value Decomposition (SVD), and the time series are those of cloud properties derived from satellite measurements.
After reading the manuscript, I find myself in the uncomfortable position of rejecting the work and having to suggest that the authors rethink the logic of the study from the ground up. This work is not only incomplete, it is also wrong in its logical premises, the type of data used, the technique presented and fail novelty. I set out my points below.- Is this scientific journal the most appropriate for such work? I don't think so. This manuscript presents no new algorithm, no validation of the data, no error analysis of the data. In truth, there is only the application of a decomposition technique of a historical series and correlations with climate indices. The authors themselves indicate in the abstract as an application to assess the performance of global climate models. AMT is not the appropriate venue for this type of message. Perhaps ACP (I have my doubts), but even better would be ClimDyn or JCLIM.
- With reference to the technique presented (SVD), the choice is unsatisfactory.
A characteristic hurdle in climate research is the size of the phase space. In a real system such as the climate it is basically infinite and in GCM is quasi-infinite. Clearly, the observations are per se limited and there is the need to single out those significant first-order components that best describe the underlying dynamics of the system. In the pursue of which, the direction in the phase space assumes a relevant role (which by the way justifies the difference between statistics and the isotropic statistical mechanics). Long story short: to achieve this goal we have already a myriad of Eigen Techniques, better suited to this purpose. Alone the EOFs, the rotated EOFs, or climate networks (e.g. Ludescher et al 2014, Donges et al. 2011, 2015). There are more out there that are more sophisticated and appropriate approaches. Why is this the case? Because in all the techniques I have mentioned it is possible to embed constraints to analyze what really counts: the variability.  The argument that SVD is preferable to other techniques because it is simpler is true at the expense of depth and accuracy of analysis. With all due respect to colleagues, this strikes me as more of a task for a freshman in a master of science course than for established researchers aiming at novel results. Moreover, nowadays, thanks to the open science paradigm, there are many public repositories where implementations of the respective techniques can be conveniently downloaded. Thus, the objection of having to code everything from scratch no longer exists.Â
Ludescher, J., Gozolchiani, A., Bogachev, M. I., Bunde, A., Havlin, S., & Schellnhuber, H. J. (2014). Very early warning of next El Niño. Proceedings of the National Academy of Sciences, 111(6), 2064-2066.
Donges, J. F., Schultz, H. C., Marwan, N., Zou, Y., & Kurths, J. (2011). Investigating the topology of interacting networks: Theory and application to coupled climate subnetworks. The European Physical Journal B, 84, 635-651.
Donges, J. F., Petrova, I., Loew, A., Marwan, N., & Kurths, J. (2015). How complex climate networks complement eigen techniques for the statistical analysis of climatological data. Climate dynamics, 45, 2407-2424. - The choice of data. The authors cannot use L3 monthly averages for their analysis, rather they must use L2 in conjunction with the standard deviation (or variance) and respective error metrics and ask how the aggregation of the L2 time series impacts the goodness of fit. This is because each sensor has a different spatio-temporal sampling and the patterns emerging from the analysis are affected by these differences. In other words, the authors must first try to answer the question of what correlation length is required for different sensors to represent the same cloud field. No data set can ever represent reality. So every data set is exposed to the same shortcomings. Before looking for a quasi-orthogonal basis to link to a climate index, one must make the observational data sets as homogeneous as possible among each other. And introduce error metrics.Â
- The manuscript is not consistently elaborated because all climate indices are stated in a table (superfluous at this point), but only ENSO is mentioned in the manuscript. From the title chosen by the authors, the manuscript suggests a (laudable and ambitious) generalisation, but this is nowhere to be found. Â
- Figures 2 and 3. I invite the authors to fill in the time gap between 2012 and 2017. Without coverage of these dates, it is not even remotely conceivable for me to scrutinise the results as the trend and eventual statistical significance (which is missing by the way). There are no prerequisites because the data presented are lacking.
- As a final point, I suggest that the authors, for the next draft of the paper, reserve some of their time and effort for interpreting the results they find. The correlation of CTH with ENSO, for example, is straightforward and immediately understandable on the basis of basic arguments of fluid- and thermo-dynamics. The matter becomes more interesting if one creates Hovmoller graphs of correlations between certain cloud properties and climate indices. The evolution of these teleconnections may reveal as yet unknown aspects. For example, the timing of the start of the monsoon season or exchanges of energy and momentum between low latitudes and the poles. Be that as it may, it is not an easy task, precisely because the authors want to tackle it empirically, on the basis of data and not models. But precisely for this reason, from my point of view, the highest possible precision must be requested in the formulation of the problem.
Citation: https://doi.org/10.5194/amt-2023-232-RC1 -
AC1: 'Reply on RC1', Elisa Carboni, 24 May 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-232/amt-2023-232-AC1-supplement.pdf
-
RC2: 'Comment on amt-2023-232', Anonymous Referee #2, 16 Apr 2024
General Comments:
This paper demonstrates an application of the Singular Value Decomposition (SVD) statistical technique to correlate cloud properties observed in satellite data against climate indices.
SVD is commonly-used in the atmospheric sciences as a method to understand the spatio-temporal variability of geophysical data. The authors effectively demonstrate how one can use SVD to compare the modes of variability in a long observational satellite record against climate indices.
Because the authors both a. emphasize the applicability of the technique to any gridded satellite dataset (not just atmospheric fields), and b. position this methodology as primarily novel for climate model validation, this paper may be better suited for a journal focused more generally on climate rather than atmospheric measurements.
That said, I think it would be appropriate for publication in Atmospheric Measurement Techniques with revisions addressing the specific comments below.
Specific Comments:
- The study is framed as a technique that can be effectively generalized to any gridded dataset, but only provides examples of cloud observations. An additional example would be helpful to support the generalization.
- The authors report strong correlations in the results but omit statistical significance.
- Spend time exploring what the authors themselves state as one of the novel aspects of using this technique: understanding the underlying causes of the variability.
- Did the authors try any other decomposition methods? How does using SVD compare to other methods such as using EOFs or other approaches? A clear statement for why this technique was chosen and what its limitations are should be included.
Technical Corrections:
- Paragraph 45: typo: de-seansonalised
- Paragraph 145: typo: de-sesonalized
- Paragraph 185: type: series off time-lags
- Figs 3, 4, 5: consider centering the maps on the Pacific Ocean instead of the Atlantic to better show the ENSO correlation
Citation: https://doi.org/10.5194/amt-2023-232-RC2 -
AC2: 'Reply on RC2', Elisa Carboni, 24 May 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-232/amt-2023-232-AC2-supplement.pdf
Status: closed
-
RC1: 'Comment on amt-2023-232', Anonymous Referee #1, 14 Dec 2023
This scientific paper presents the application of a statistical (or algebraic) technique that identifies that portion of data in a time series that is useful in interpreting the dynamics of the atmospheric system through correlations with climate indices. Specifically, the technique presented is one of the possible belonging to the group of the Eigen techniques, the Singular Value Decomposition (SVD), and the time series are those of cloud properties derived from satellite measurements.
After reading the manuscript, I find myself in the uncomfortable position of rejecting the work and having to suggest that the authors rethink the logic of the study from the ground up. This work is not only incomplete, it is also wrong in its logical premises, the type of data used, the technique presented and fail novelty. I set out my points below.- Is this scientific journal the most appropriate for such work? I don't think so. This manuscript presents no new algorithm, no validation of the data, no error analysis of the data. In truth, there is only the application of a decomposition technique of a historical series and correlations with climate indices. The authors themselves indicate in the abstract as an application to assess the performance of global climate models. AMT is not the appropriate venue for this type of message. Perhaps ACP (I have my doubts), but even better would be ClimDyn or JCLIM.
- With reference to the technique presented (SVD), the choice is unsatisfactory.
A characteristic hurdle in climate research is the size of the phase space. In a real system such as the climate it is basically infinite and in GCM is quasi-infinite. Clearly, the observations are per se limited and there is the need to single out those significant first-order components that best describe the underlying dynamics of the system. In the pursue of which, the direction in the phase space assumes a relevant role (which by the way justifies the difference between statistics and the isotropic statistical mechanics). Long story short: to achieve this goal we have already a myriad of Eigen Techniques, better suited to this purpose. Alone the EOFs, the rotated EOFs, or climate networks (e.g. Ludescher et al 2014, Donges et al. 2011, 2015). There are more out there that are more sophisticated and appropriate approaches. Why is this the case? Because in all the techniques I have mentioned it is possible to embed constraints to analyze what really counts: the variability.  The argument that SVD is preferable to other techniques because it is simpler is true at the expense of depth and accuracy of analysis. With all due respect to colleagues, this strikes me as more of a task for a freshman in a master of science course than for established researchers aiming at novel results. Moreover, nowadays, thanks to the open science paradigm, there are many public repositories where implementations of the respective techniques can be conveniently downloaded. Thus, the objection of having to code everything from scratch no longer exists.Â
Ludescher, J., Gozolchiani, A., Bogachev, M. I., Bunde, A., Havlin, S., & Schellnhuber, H. J. (2014). Very early warning of next El Niño. Proceedings of the National Academy of Sciences, 111(6), 2064-2066.
Donges, J. F., Schultz, H. C., Marwan, N., Zou, Y., & Kurths, J. (2011). Investigating the topology of interacting networks: Theory and application to coupled climate subnetworks. The European Physical Journal B, 84, 635-651.
Donges, J. F., Petrova, I., Loew, A., Marwan, N., & Kurths, J. (2015). How complex climate networks complement eigen techniques for the statistical analysis of climatological data. Climate dynamics, 45, 2407-2424. - The choice of data. The authors cannot use L3 monthly averages for their analysis, rather they must use L2 in conjunction with the standard deviation (or variance) and respective error metrics and ask how the aggregation of the L2 time series impacts the goodness of fit. This is because each sensor has a different spatio-temporal sampling and the patterns emerging from the analysis are affected by these differences. In other words, the authors must first try to answer the question of what correlation length is required for different sensors to represent the same cloud field. No data set can ever represent reality. So every data set is exposed to the same shortcomings. Before looking for a quasi-orthogonal basis to link to a climate index, one must make the observational data sets as homogeneous as possible among each other. And introduce error metrics.Â
- The manuscript is not consistently elaborated because all climate indices are stated in a table (superfluous at this point), but only ENSO is mentioned in the manuscript. From the title chosen by the authors, the manuscript suggests a (laudable and ambitious) generalisation, but this is nowhere to be found. Â
- Figures 2 and 3. I invite the authors to fill in the time gap between 2012 and 2017. Without coverage of these dates, it is not even remotely conceivable for me to scrutinise the results as the trend and eventual statistical significance (which is missing by the way). There are no prerequisites because the data presented are lacking.
- As a final point, I suggest that the authors, for the next draft of the paper, reserve some of their time and effort for interpreting the results they find. The correlation of CTH with ENSO, for example, is straightforward and immediately understandable on the basis of basic arguments of fluid- and thermo-dynamics. The matter becomes more interesting if one creates Hovmoller graphs of correlations between certain cloud properties and climate indices. The evolution of these teleconnections may reveal as yet unknown aspects. For example, the timing of the start of the monsoon season or exchanges of energy and momentum between low latitudes and the poles. Be that as it may, it is not an easy task, precisely because the authors want to tackle it empirically, on the basis of data and not models. But precisely for this reason, from my point of view, the highest possible precision must be requested in the formulation of the problem.
Citation: https://doi.org/10.5194/amt-2023-232-RC1 -
AC1: 'Reply on RC1', Elisa Carboni, 24 May 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-232/amt-2023-232-AC1-supplement.pdf
-
RC2: 'Comment on amt-2023-232', Anonymous Referee #2, 16 Apr 2024
General Comments:
This paper demonstrates an application of the Singular Value Decomposition (SVD) statistical technique to correlate cloud properties observed in satellite data against climate indices.
SVD is commonly-used in the atmospheric sciences as a method to understand the spatio-temporal variability of geophysical data. The authors effectively demonstrate how one can use SVD to compare the modes of variability in a long observational satellite record against climate indices.
Because the authors both a. emphasize the applicability of the technique to any gridded satellite dataset (not just atmospheric fields), and b. position this methodology as primarily novel for climate model validation, this paper may be better suited for a journal focused more generally on climate rather than atmospheric measurements.
That said, I think it would be appropriate for publication in Atmospheric Measurement Techniques with revisions addressing the specific comments below.
Specific Comments:
- The study is framed as a technique that can be effectively generalized to any gridded dataset, but only provides examples of cloud observations. An additional example would be helpful to support the generalization.
- The authors report strong correlations in the results but omit statistical significance.
- Spend time exploring what the authors themselves state as one of the novel aspects of using this technique: understanding the underlying causes of the variability.
- Did the authors try any other decomposition methods? How does using SVD compare to other methods such as using EOFs or other approaches? A clear statement for why this technique was chosen and what its limitations are should be included.
Technical Corrections:
- Paragraph 45: typo: de-seansonalised
- Paragraph 145: typo: de-sesonalized
- Paragraph 185: type: series off time-lags
- Figs 3, 4, 5: consider centering the maps on the Pacific Ocean instead of the Atlantic to better show the ENSO correlation
Citation: https://doi.org/10.5194/amt-2023-232-RC2 -
AC2: 'Reply on RC2', Elisa Carboni, 24 May 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-232/amt-2023-232-AC2-supplement.pdf
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 389 | 117 | 38 | 544 | 36 | 34 |
- HTML: 389
- PDF: 117
- XML: 38
- Total: 544
- BibTeX: 36
- EndNote: 34
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1