the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Optimal estimation of cloud properties from thermal infrared observations with a combination of deep learning and radiative transfer simulation
Abstract. While traditional thermal infrared retrieval algorithms based on radiative transfer models (RTM) could not effectively retrieve the cloud optical thickness of thick clouds, machine learning based algorithms were found to be able to provide reasonable estimations for both daytime and nighttime. Nevertheless, stand-alone machine learning algorithms are occasionally criticized for the lack of explicit physical processes. In this study, RTM simulations and a machine learning algorithm are synergistically utilized using the optimal estimation (OE) method to retrieve cloud properties from thermal infrared radiometry measured by Moderate Resolution Imaging Spectroradiometer (MODIS). In the new algorithm, retrievals from a machine learning algorithm are used to provide priori states for the iterative process of OE method, and an RTM is used to create radiance lookup tables that are used in the iteration processes. Compared with stand-alone OE, the cloud properties retrieved by the new algorithm show an overall better performance by using the spatial statistic information obtained by machine learning algorithm. Compared with stand-alone machine-learning based algorithm, the radiances simulated based on retrievals from the new method align more closely with observations, and physical radiative processes are handled explicitly in the new algorithm. Therefore, the new method combines the advantages of RTM-based cloud retrieval methods and machine-learning models. These findings highlight the potential for machine-learning-based algorithms to enhance the efficacy of conventional remote sensing techniques.
- Preprint
(2218 KB) - Metadata XML
- BibTeX
- EndNote
Status: closed
-
RC1: 'Comment on amt-2024-87', Anonymous Referee #1, 08 Jul 2024
The comment was uploaded in the form of a supplement.
-
AC1: 'Reply on RC1', he huang, 14 Aug 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2024-87/amt-2024-87-AC1-supplement.pdf
-
AC1: 'Reply on RC1', he huang, 14 Aug 2024
-
RC2: 'Comment on amt-2024-87', Anonymous Referee #2, 17 Jul 2024
This study proposed a cloud properties retrieval algorithm combining the optimal estimation (OE) method and convolutional neural network (CNN) method based on the infrared (IR) bands, in which the CNN-IR provides the a priori information of COT, CER, and CTH. Results indicate that the OE-CNN-IR method generally performs better than the stand-alone OE method (i.e., OE-IR) with fixed apriori values. In addition, OE-CNN-IR can retrieve all-day cloud properties that traditional two-channel methods using VIS and SWIR bands fail. The main concerns need to be addressed before accepting the manuscript.
- In terms of methodology, the authors need to be more specific about what improvements OE-CNN-IR has and the motivation for the combination of OE and CNN, in comparison to the TIR-CNN method, as the OE-CNN-IR iterative process is highly dependent on the priori information that TIR-CNN provides. Particularly, the cloud properties derived by TIR-CNN seem to have higher consistencies with those of MYD06 than OE-CNN-IR in Fig. 6.
- P3L83, please clarify the main purpose of this study instead of ‘A great number of cloud property users favor remote sensing products that offer explicit physical interpretations’ , which is too arbitrary. In my opinion, one of the biggest advantages of OE compared with CNN is that it could provide retrieval uncertainty, while CNN fails. In this case, any information on the retrieval uncertainty might be more valuable.
- In addition, the authors should have provided more explanation and physical meaning on the OE-CNN-IR. To emphasize the advantage of OE, I encourage the authors to extend the study of Fig. 4 using the synthetic data, by conducting information content analysis to check the best combination of available wavelengths, investigating the effects of values of Sy, Sa and Xa on the retrieval, error component analysis, etc.
- Table 1, the detailed wavelength information should be provided, in addition, why is the solar zenith angle excluded in the algorithm? What is the meaning of ‘cloud phase infrared’, ‘cloud phase optical properties’?
- In Figure 1, the flowchart is relatively simple, and some details of the inversion are still unclear, e.g., what is the priori information, is there any cloud phase detection, etc.?
- The sensitivity analysis in Figure 2 shows that when the COT is larger than 10, the changes of BT caused by the COT are no longer obvious, are the retrieval results reliable in the larger COT conditions?
- Fig.5 is a little bit confusing, since the authors want to emphasize the advantage of OE-CNN-IR, while the simulated BT based on the OE-CNN are more consistent with the observation than those of OE-CNN-IR. Then the readers might understand that OE-IR can get a more accurate retrieval through a better fitting of the observed spectral. In this case, I suggest authors provide more information on the retrieval, such as the degree of freedom.
- P11L227, iterations over 200 times seem to be meaningless since the cost function has converged after several times iterations according to Fig.4. In addition, there is no information on the ‘real’ COT, CER, and CTH in Fig. 4.
- Section 2.1.2, the title of “Active Lidar Detection cloud products” is misleading as the DARDAR product is based on CALIOP and CPR observations.
Citation: https://doi.org/10.5194/amt-2024-87-RC2 -
AC2: 'Reply on RC2', he huang, 14 Aug 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2024-87/amt-2024-87-AC2-supplement.pdf
-
EC1: 'Comment on amt-2024-87', Jian Xu, 23 Jul 2024
The third reviewer's comments were sent directly to me due to a missed discussion deadline. Please find the comments attached. Kindly address each point and make the necessary revisions to your manuscript.
Best,
Jian Xu
-
AC3: 'Reply on EC1', he huang, 14 Aug 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2024-87/amt-2024-87-AC3-supplement.pdf
-
AC3: 'Reply on EC1', he huang, 14 Aug 2024
Status: closed
-
RC1: 'Comment on amt-2024-87', Anonymous Referee #1, 08 Jul 2024
The comment was uploaded in the form of a supplement.
-
AC1: 'Reply on RC1', he huang, 14 Aug 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2024-87/amt-2024-87-AC1-supplement.pdf
-
AC1: 'Reply on RC1', he huang, 14 Aug 2024
-
RC2: 'Comment on amt-2024-87', Anonymous Referee #2, 17 Jul 2024
This study proposed a cloud properties retrieval algorithm combining the optimal estimation (OE) method and convolutional neural network (CNN) method based on the infrared (IR) bands, in which the CNN-IR provides the a priori information of COT, CER, and CTH. Results indicate that the OE-CNN-IR method generally performs better than the stand-alone OE method (i.e., OE-IR) with fixed apriori values. In addition, OE-CNN-IR can retrieve all-day cloud properties that traditional two-channel methods using VIS and SWIR bands fail. The main concerns need to be addressed before accepting the manuscript.
- In terms of methodology, the authors need to be more specific about what improvements OE-CNN-IR has and the motivation for the combination of OE and CNN, in comparison to the TIR-CNN method, as the OE-CNN-IR iterative process is highly dependent on the priori information that TIR-CNN provides. Particularly, the cloud properties derived by TIR-CNN seem to have higher consistencies with those of MYD06 than OE-CNN-IR in Fig. 6.
- P3L83, please clarify the main purpose of this study instead of ‘A great number of cloud property users favor remote sensing products that offer explicit physical interpretations’ , which is too arbitrary. In my opinion, one of the biggest advantages of OE compared with CNN is that it could provide retrieval uncertainty, while CNN fails. In this case, any information on the retrieval uncertainty might be more valuable.
- In addition, the authors should have provided more explanation and physical meaning on the OE-CNN-IR. To emphasize the advantage of OE, I encourage the authors to extend the study of Fig. 4 using the synthetic data, by conducting information content analysis to check the best combination of available wavelengths, investigating the effects of values of Sy, Sa and Xa on the retrieval, error component analysis, etc.
- Table 1, the detailed wavelength information should be provided, in addition, why is the solar zenith angle excluded in the algorithm? What is the meaning of ‘cloud phase infrared’, ‘cloud phase optical properties’?
- In Figure 1, the flowchart is relatively simple, and some details of the inversion are still unclear, e.g., what is the priori information, is there any cloud phase detection, etc.?
- The sensitivity analysis in Figure 2 shows that when the COT is larger than 10, the changes of BT caused by the COT are no longer obvious, are the retrieval results reliable in the larger COT conditions?
- Fig.5 is a little bit confusing, since the authors want to emphasize the advantage of OE-CNN-IR, while the simulated BT based on the OE-CNN are more consistent with the observation than those of OE-CNN-IR. Then the readers might understand that OE-IR can get a more accurate retrieval through a better fitting of the observed spectral. In this case, I suggest authors provide more information on the retrieval, such as the degree of freedom.
- P11L227, iterations over 200 times seem to be meaningless since the cost function has converged after several times iterations according to Fig.4. In addition, there is no information on the ‘real’ COT, CER, and CTH in Fig. 4.
- Section 2.1.2, the title of “Active Lidar Detection cloud products” is misleading as the DARDAR product is based on CALIOP and CPR observations.
Citation: https://doi.org/10.5194/amt-2024-87-RC2 -
AC2: 'Reply on RC2', he huang, 14 Aug 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2024-87/amt-2024-87-AC2-supplement.pdf
-
EC1: 'Comment on amt-2024-87', Jian Xu, 23 Jul 2024
The third reviewer's comments were sent directly to me due to a missed discussion deadline. Please find the comments attached. Kindly address each point and make the necessary revisions to your manuscript.
Best,
Jian Xu
-
AC3: 'Reply on EC1', he huang, 14 Aug 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2024-87/amt-2024-87-AC3-supplement.pdf
-
AC3: 'Reply on EC1', he huang, 14 Aug 2024
Viewed
HTML | XML | Total | BibTeX | EndNote | |
---|---|---|---|---|---|
351 | 129 | 33 | 513 | 14 | 10 |
- HTML: 351
- PDF: 129
- XML: 33
- Total: 513
- BibTeX: 14
- EndNote: 10
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1