General comments:

In this manuscript, the authors present a machine learning based cloud cover and fraction retrieval framework from AGRI observations onboard FY-4 satellites using LSTM network. As a reviewer who did the initial assessment before it came to a full review, I find the manuscript largely improved by the authors with the majority of my concerns addressed. However, there still exist some minor issues in the contents that could possibly confuse the readers. Therefore, I would suggest a minor revision before it gets published in this journal.

Specific comments:

1. Line 34~35, ‘the accuracy,… greatly improved’ should be ‘the accuracy gets greatly improved’.
2. Line 82, ‘is the based on’ should be ‘is based on’.
3. Line 83, ‘Such as’ should be ‘For example,’ or ‘For instance’.
4. Line 123, ‘It has total 14 channels’ should be ‘It has 14 channels in total’.
5. Line 129, when referring to spatial resolution of satellite imagers, it should be mentioned as ‘4km at nadir’ as the pixel size varies by geolocation.
6. Line 154, ‘for download’ should be ‘to download’.
7. Line 163, ‘distance difference’ should be ’spatial distance’.
8. Line 241, ‘hidden layer size’ can be confusing for readers whether that is the number of hidden layers, or the number of neurons on each hidden layer. Personally I would believe the number 3 refers to the number of hidden layers. But the saying on Line 238 which reads as ‘having a hidden layer that is too large’ also seems to indicate the latter meaning. Please explain in a clear way.
9. Line 251~252, are both models A and B using the same settings of batch size, optimizer and loss function? As far as I know, classification models (model A) and regression models (model B) usually use different loss functions.
10. Line 278, ‘convercast’ should be ‘overcast’.
11. Line 309, ‘inverted’ should be ‘retrieved’.
12. Line 310, ‘in this text’ should be ‘in this paper’.
13. Line 311, there should be an ‘and’ before ‘operational products’.
14. Line 315, again ‘inverted’, recommend the authors revise all ‘invert’ when referring to retrievals.
15. Line 349~352, the description of components (lines, dots, etc.) on Figure 2(b) should also appear in the caption of Figure 2 so that readers don’t need to look for the sentences when looking at the figure itself.
16. Table 4, what truth value are the statistics calculated against? According to Line 190~191 in this manuscript, 2B-CLDCLASS-LIDAR data is only available before July 2019.

Review comments for “Using machine learning algorithm 1 to retrieve cloud fraction based on FY-4A AGRI observations” by Xia, J. and Guan, L.

General comments:

This manuscript presents a novel application of a Long Short-Term Memory (LSTM) machine learning algorithm to retrieve cloud fraction using data from the FY-4A AGRI satellite. The study addresses an important aspect of meteorological research, providing a valuable contribution to the field by improving cloud detection and fraction retrieval accuracy. The manuscript is well-structured and comprehensive, detailing the methodology, data sources, and validation processes effectively. However, there are areas where the manuscript could be further strengthened. The introduction could benefit from a deeper explanation of why LSTM models are particularly well-suited for this application compared to other machine learning techniques. Additionally, while the results are promising, a more thorough discussion of potential biases and limitations, especially regarding the higher false alarm rates observed with the LSTM model, would provide a balanced view. Including confidence intervals or significance tests for the reported metrics would also add to the rigor of the statistical analysis.

The manuscript has the potential to be a valuable resource for researchers and practitioners in meteorology and environmental monitoring, but significant improvements should be made before.

Specific Comments

1. The manuscript primarily discusses the method, but the abstract and the content focuses only on the results. There is insufficient explanation about the method (machine learning and correction), its characteristics, and why it leads to improvements.

1. The manuscript lacks a discussion about the impact of different machine learning structures on the retrieval of cloud fraction. Please provide more results regarding this.

1. The data resolution of CloudSat and CALIPSO is not consistent with AGRI. It is crucial to discuss this dataset uncertainty and its impact on the retrieval.

1. The captions of the figures and tables are too simplified and do not provide essential information.

1. Line 63: What is the spatial resolution of the instrument?

1. Lines 65-81: Summarize these old and classic references, emphasizing only the important ones.

1. Lines 98-113: The logic is not clear. Why did the authors conduct this work, and what is the advantage of the LSTM compared to other frameworks?

1. Table 1: A reference should be provided.

1. Lines 149-155: Since cloud fraction is important for building the training dataset, the algorithm for the joint product should be introduced. Additionally, how the uncertainty in the dataset affects the retrieval should also be discussed.

1. Lines 253-258: When discussing LSTM, the use of time series information is implied. Is this the case in your study?

Minor corrections:

1. Lines 11, 43,62, 275 and etc., add a comma before the “and” or “or” in a list

1. Line 15, “Correction” should be “Corrections”

1. Line 49, “determined” is too strong, it should be “influenced”

1. Line 66, better to replace “encompassing” with “including”

1. Line 81, it should be made clear that why “season” and “climate” will influence the thresholds?

1. Line 203, “and demonstrates” not correct, or just remove the period

1. Line 206, remove the extra comma

1. Line 276, remove the quotation mark, and add “the” before the POD and FAR

1. Line 283, “retrieving cloud fraction”-> “retrieving cloud fractions”

1. Line 301, “Operational” -> “operational”

1. Lines 308-309, Line 317, and etc. , why did you use “cloud amount” and “inverted” instead of “cloud fractions” and “retrieved”?

1. Lines 467-471, use the abbr. “FAR” instead of “false alarm rate”

1. Lines 473-478, it is not appropriate to compare the “algorithm” to the “operation product”, so replace the “than” in Line 475 and 478 with “compared to”

General comments:

A simple LSTM performs detection and cloud fraction regression, using FY-4A AGRI data as input, with labels based on 2B-CLDCLASS-LIDAR. The task is important for meteorological and climate applications, and the authors do an excellent job of providing motivation for their work. However, the proposed machine learning method is unusual for a few reasons, and the unusual aspects are not justified by much explanation or comparison with other, more standard machine learning practices.

If the authors complete some major changes, this work has the potential to be valuable.

Line-by-line comments:

• 191-196: Why do you only use data from May and June 2019? FY-4A was launched earlier, so shouldn’t there be more data available? It would be good to take a random sample from a larger window of time with more seasonal variability. If there is some other reason only to use this time window, it needs to be stated here.
• 239-241: “Typically, the optimal size of the hidden layer is determined by trying different sizes and evaluating their performance on a validation set.” Correct. Did you do this? If so, the results should be in the paper.
• 241: Your hidden layer sizes are extremely small. Also, you need to state how many hidden layers you have. My subsequent analysis will assume you only have 1 hidden layer. Even so, this needs to be explicitly stated.
• 244-246: This is simply a restatement of the definition of batch size, and not an explanation. I’d remove both sentences. Instead, you can simply say something like “A larger batch size typically reduces the training time per epoch for complex reasons” and then cite some paper about batch size, e.g. [1].
• 254: Are the inputs to the network normalized? If not, they should be.
• 267: “a batch size of 60 is chosen due to the limited sample number in dataset B.” If you have limited data, that doesn’t mean you need to decrease the batch size. Given the extremely small size of your LSTM, I don’t think you will have a problem with overfitting, so you can safely use as large of a batch size as you have the VRAM for.
• 268: This is fine for the cloud fraction task, but you shouldn’t / can’t use MSE for the cloud detection task, as it is a classification and not a regression.
• Figure 2:
  • (a) circular lines should be thicker, text should be larger
  • (b) missing a legend for the colors
  • (c) put the degree sign ° on the tick labels in the color bar
  • (d) missing a legend for the colors, and you should use different marker types (e.g. triangles and circles) to help colorblind readers

Major changes requested:

• You use a temporal / recurrent network architecture (LSTM) but have no description of how the temporal aspect of your data is used by the network. You need to describe this, in detail. What are the timesteps? Is there a warmup time for your LSTM (a minimum time before it attains a stable state and a reasonable accuracy)? On the other hand, if there is no time series information in use here, you should not be using an LSTM!
• There needs to be some comparison with other models. A (single-layer?) network with only 3 hidden units is a very unusual choice. This is a tiny neural network. You could write down the full definition of this network on a piece of paper.
  • Why even use a neural network at that point, instead of something like a random forest? The latter would be more interpretable, easier to train / justify, and potentially more powerful than your very small model.
  • If you’re going to keep such a small network, I’d like to see a comparison with linear regression.
  • How does it compare with a larger LSTM?
  • Is the temporal component helpful (i.e. how does the LSTM compare with a standard MLP)?
  • Why not make use of spatial context? You could consider using a convolutional neural network or a ConvLSTM.
• I think you should use a larger dataset, or at least one that isn’t restricted to one month. There is seasonal variability to clouds, and you want your algorithm to be effective year-round.
• You are comparing your network with an operational cloud product. This is not a very fair comparison, as the operational cloud product is designed with different goals and principles (and physics) in mind than 2B-CLDCLASS-LIDAR was, whereas your method is directly trained on the 2B-CLDCLASS-LIDAR product.
  • You should spend more time breaking down these differences
  • Having a reasonable standard baseline (like a regular neural network or a random forest) would further help contextualize your results.

References

[1] Smith, Samuel L., et al. "Don't decay the learning rate, increase the batch size." arXiv preprint arXiv:1711.00489 (2017).