the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Estimation of twenty-four-hour continuous cloud cover using ground-based imager with convolutional neural network
Joo Wan Cha
Yong Hee Lee
Abstract. In this study, we aimed to estimate cloud cover with high accuracy using images from a camera-based imager and a convolutional neural network (CNN) as a potential alternative to human-eye observation on the ground. Image data collected at 1 h intervals from 2019 to 2020 at a manned weather station, where human-eye observations were performed, were used as input data. The 2019 dataset was used for training and validating the CNN model, whereas the 2020 dataset was used for testing the estimated cloud cover. Additionally, we compared satellite (SAT) and ceilometer (CEI) cloud cover to determine the method most suitable for cloud cover estimation at the ground level. The CNN model was optimized using a deep layer and detailed hyperparameter settings. Consequently, the model achieved an accuracy, bias, root mean square error (RMSE), and correlation coefficient (R) of 0.92, −0.13, 1.40 tenths, and 0.95, respectively, on the test dataset, and exhibited approximately 93 % high agreement at a difference within ± 2 tenths of the observed cloud cover. This result demonstrates an improvement over previous studies that used threshold, machine learning, and deep learning methods. In addition, compared with the SAT (with an accuracy, bias, RMSE, R, and agreement of 0.89, 0.33 tenths, 2.31 tenths, 0.87, and 83 %, respectively) and CEI (with an accuracy, bias, RMSE, R, agreement of 0.86, −1.58 tenths, 3.34 tenths, 0.76, and 74 %, respectively), the camera-based imager with the CNN was found to be the most suitable method to replace ground cloud cover observation by humans.
- Preprint
(1484 KB) - Metadata XML
- BibTeX
- EndNote
Bu-Yo Kim et al.
Interactive discussion
Status: closed
-
RC1: 'Comment on amt-2023-131', Anonymous Referee #1, 12 Aug 2023
General Comments:
Accurate cloud cover observations are crucial for analyzing the macro and micro characteristics of clouds and studying their impact on weather and climate. This study utilizes a camera-based imager and convolutional neural network (CNN) as a prospective alternative to ground-based human-eye observations. The optimized CNN model employed in this research yields favorable performance metrics, exhibiting relatively smaller errors compared to traditional alternatives, thereby demonstrating certain advantages and holding scientific value. However, in comparison to other studies that employ machine learning algorithms for cloud cover analysis (including the author's previous works), the improvements presented in this paper are relatively limited, and the level of innovation is less prominent. Therefore, it is recommended to reconsider the acceptance of this study after the following issues have been solved.
Specific Comments:
1) Compared to the study introduced in Kim et al. (2021b), the improvement in cloud cover analysis achieved by the algorithm presented in this study appears to be limited. Please provide a detailed explanation of the novelty and necessity of this study.
2) Figure 5&6: From these figures, it can be observed that the algorithm tends to underestimate cloud cover when compared to the observed results. Please provide an explanation for this phenomenon.
3) P2L4: satellites can’t be categorized into “ground-based methods”.
Citation: https://doi.org/10.5194/amt-2023-131-RC1 -
RC2: 'Reply on RC1', Anonymous Referee #2, 15 Aug 2023
This paper proposes a novel method to estimate ground-level cloud cover using images from a ground-based sky imager and convolutional neural network, as an alternative to human observations. The topic is of great value and the methodology is generally sound. The paper is well structured and well written. The study provides a feasible technical approach for automated monitoring of surface cloud cover. In summary, the following issues need further improvement:
（1）The input data of photos were sampled at 1 hour intervals, corresponding to the manual observations. It is unclear whether the sampling time is exactly the same as the time of manual observation. Clouds can change rapidly in a short period of time, which may lead to inaccurate sample data if there is a time discrepancy between the photos and observations.
（2）This study utilizes CNN to establish the model, with images as input and manual observations as true value labels. The method is relatively effective. However, there are two aspects of problems: 1) Manual observations also have errors, which are determined by human subjective judgments. Therefore, the methods developed based on such samples and labels inevitably still contain errors, which cannot be further improved through model tuning and data augmentation. It is recommended to try comparison with other conventional methods, and develop research methods to improve the accuracy of sample data;
2) The manual observations quantify the results into 0-10 tenths. From the observation perspective, the resolution of such cloud cover observations is slightly low. Using 1 tenth to 1.9 tenths may have a difference of 9 percentage points, which needs further improvement in the observation accuracy of cloud cover for refined observations and forecasts.
（3）Based on traditional image processing methods like NBR threshold, the accuracy is usually lower during sunrise and sunset due to the influence of scattering near the sun. This study also found larger errors during this period. What are the reasons? Is it because there are fewer manual observation samples?
Citation: https://doi.org/10.5194/amt-2023-131-RC2 -
AC2: 'Reply on RC2', Bu-Yo Kim, 31 Aug 2023
This paper proposes a novel method to estimate ground-level cloud cover using images from a ground-based sky imager and convolutional neural network, as an alternative to human observations. The topic is of great value and the methodology is generally sound. The paper is well structured and well written. The study provides a feasible technical approach for automated monitoring of surface cloud cover. In summary, the following issues need further improvement:
Thank you for reviewing this manuscript. Based on the reviewer's comments, we have added some content to the revised manuscript, as detailed below. We believe that the quality of the manuscript has been improved and the clarity of the manuscript has been enhanced after addressing the reviewer's comments.
（1）The input data of photos were sampled at 1 hour intervals, corresponding to the manual observations. It is unclear whether the sampling time is exactly the same as the time of manual observation. Clouds can change rapidly in a short period of time, which may lead to inaccurate sample data if there is a time discrepancy between the photos and observations.
We agree with the reviewer's comments. The KMA ground weather observation guidelines stipulate that observations should be conducted for every on the hour. However, the timing of observation may vary depending on the observer, which can lead to inconsistencies and potential inaccuracies in the collected data. Unfortunately, such information has not been documented, preventing us from determining the precise observation time.
（2）This study utilizes CNN to establish the model, with images as input and manual observations as true value labels. The method is relatively effective. However, there are two aspects of problems:
1) Manual observations also have errors, which are determined by human subjective judgments. Therefore, the methods developed based on such samples and labels inevitably still contain errors, which cannot be further improved through model tuning and data augmentation. It is recommended to try comparison with other conventional methods, and develop research methods to improve the accuracy of sample data;
2) The manual observations quantify the results into 0-10 tenths. From the observation perspective, the resolution of such cloud cover observations is slightly low. Using 1 tenth to 1.9 tenths may have a difference of 9 percentage points, which needs further improvement in the observation accuracy of cloud cover for refined observations and forecasts.
We agree with the reviewer's comments on 1) and 2). The KMA-observed cloud cover was recorded as 0-8 oktas or 0-10 tenths according to the WMO synoptic observation guidelines. The employed observation method, as also noted by the reviewer, has a low resolution of quantitative values; hence, the margin of error is a few % in the same cloud cover class. Therefore, we believe that it would be possible to estimate the cloud cover with high accuracy if the pixel information (with or without clouds) in the image is extracted and learned in terms of the percent cloud cover as the label. In this study, the data resolution was rather low because the OBS cloud cover was used as a label. Therefore, we have added the following sentence:
L301 “Depending on the characteristics of the data to be learned, it is possible to estimate the cloud cover in % instead of oktas and tenths; accordingly, the time resolution can also be estimated in minutes rather than hourly intervals.”
（3）Based on traditional image processing methods like NBR threshold, the accuracy is usually lower during sunrise and sunset due to the influence of scattering near the sun. This study also found larger errors during this period. What are the reasons? Is it because there are fewer manual observation samples?
We used the DL method to compensate for the shortcomings of the threshold method and the ML method. The estimation accuracy of the cloud cover in this study was better than that of the previously employed threshold method and ML method. However, the accuracy was still relatively low at sunrise/sunset time. This is because, as noted by the reviewer, the image was polluted due to the scattering of light near the sun, making it difficult to distinguish clouds from the sky. Therefore, we added the following sentence to the main text on possible methods to improve the accuracy of cloud cover estimation at sunrise/sunset time:
L223 “Moreover, in this study, images acquired at sunrise/sunset time accounted for 16.23% of all learning datasets. In other words, the images acquired at sunrise/sunset time were learned 2 to 3 times lower for each cloud cover class than images acquire at daytime and nighttime. Therefore, it is expected that the cloud cover estimation accuracy can be improved if more images are acquired and learned during sunrise/sunset time.”
Citation: https://doi.org/10.5194/amt-2023-131-AC2
-
AC2: 'Reply on RC2', Bu-Yo Kim, 31 Aug 2023
-
AC1: 'Reply on RC1', Bu-Yo Kim, 31 Aug 2023
General Comments:
Accurate cloud cover observations are crucial for analyzing the macro and micro characteristics of clouds and studying their impact on weather and climate. This study utilizes a camera-based imager and convolutional neural network (CNN) as a prospective alternative to ground-based human-eye observations. The optimized CNN model employed in this research yields favorable performance metrics, exhibiting relatively smaller errors compared to traditional alternatives, thereby demonstrating certain advantages and holding scientific value. However, in comparison to other studies that employ machine learning algorithms for cloud cover analysis (including the author's previous works), the improvements presented in this paper are relatively limited, and the level of innovation is less prominent. Therefore, it is recommended to reconsider the acceptance of this study after the following issues have been solved.
Thank you for reviewing this manuscript. Based on the reviewer's comments, we have added some content to the revised manuscript, as detailed below. We believe that the quality of the manuscript has been improved and the clarity of the manuscript has been enhanced after addressing the reviewer's comments.
Specific Comments:
1) Compared to the study introduced in Kim et al. (2021b), the improvement in cloud cover analysis achieved by the algorithm presented in this study appears to be limited. Please provide a detailed explanation of the novelty and necessity of this study.
Content related to novel attempts, comparison with the observation, and the necessity of our research has been added to the second paragraph of Section 2.1, first paragraph of Section 2.2, first paragraph of Section 3, and the last three sentences of Section 5. We also added the following content to Section 5 for greater emphasis:
L290 “In this study, a novel method was attempted to learn a DL model for cloud cover estimation. Compared to the datasets of previous studies (e.g., Fa et al., 2019; Kim et al., 2021b; Xie et al., 2020; Ye et al., 2022), more images were learned, and long-term estimated data were analyzed. Furthermore, the estimated results were compared with observational data from a real weather station and other remote observational data (i.e., from a satellite and ceilometer).”
2) Figure 5&6: From these figures, it can be observed that the algorithm tends to underestimate cloud cover when compared to the observed results. Please provide an explanation for this phenomenon.
We have revised the manuscript by adding the following sentences:
L209 “In general, the daytime and nighttime CNN cloud cover did not exhibit a large difference compared with the OBS cloud cover; however, the bias and RMSE were relatively large, and R was low during sunrise/sunset time.”
L215 “In particular, there were many cases where the CNN cloud cover was smaller than the OBS cloud cover in the 9 and 10 tenths classes during sunrise/sunset time. The mean SZAs of the two classes were 85.13° and 98.11°, respectively, and the error was relatively large when the sun moved completely above and below the horizon (i.e., at 7–8 LST and 19–20 LST, respectively).”
3) P2L4: satellites can’t be categorized into “ground-based methods”.
We have revised existing text as follows:
L33 “Remote and automatic observation as well as estimation of the cloud cover on the ground can be achieved using meteorological satellites (SATs), ceilometers (CEIs), and camera-based imagers.”
Citation: https://doi.org/10.5194/amt-2023-131-AC1
-
RC2: 'Reply on RC1', Anonymous Referee #2, 15 Aug 2023
Peer review completion
Interactive discussion
Status: closed
-
RC1: 'Comment on amt-2023-131', Anonymous Referee #1, 12 Aug 2023
General Comments:
Accurate cloud cover observations are crucial for analyzing the macro and micro characteristics of clouds and studying their impact on weather and climate. This study utilizes a camera-based imager and convolutional neural network (CNN) as a prospective alternative to ground-based human-eye observations. The optimized CNN model employed in this research yields favorable performance metrics, exhibiting relatively smaller errors compared to traditional alternatives, thereby demonstrating certain advantages and holding scientific value. However, in comparison to other studies that employ machine learning algorithms for cloud cover analysis (including the author's previous works), the improvements presented in this paper are relatively limited, and the level of innovation is less prominent. Therefore, it is recommended to reconsider the acceptance of this study after the following issues have been solved.
Specific Comments:
1) Compared to the study introduced in Kim et al. (2021b), the improvement in cloud cover analysis achieved by the algorithm presented in this study appears to be limited. Please provide a detailed explanation of the novelty and necessity of this study.
2) Figure 5&6: From these figures, it can be observed that the algorithm tends to underestimate cloud cover when compared to the observed results. Please provide an explanation for this phenomenon.
3) P2L4: satellites can’t be categorized into “ground-based methods”.
Citation: https://doi.org/10.5194/amt-2023-131-RC1 -
RC2: 'Reply on RC1', Anonymous Referee #2, 15 Aug 2023
This paper proposes a novel method to estimate ground-level cloud cover using images from a ground-based sky imager and convolutional neural network, as an alternative to human observations. The topic is of great value and the methodology is generally sound. The paper is well structured and well written. The study provides a feasible technical approach for automated monitoring of surface cloud cover. In summary, the following issues need further improvement:
（1）The input data of photos were sampled at 1 hour intervals, corresponding to the manual observations. It is unclear whether the sampling time is exactly the same as the time of manual observation. Clouds can change rapidly in a short period of time, which may lead to inaccurate sample data if there is a time discrepancy between the photos and observations.
（2）This study utilizes CNN to establish the model, with images as input and manual observations as true value labels. The method is relatively effective. However, there are two aspects of problems: 1) Manual observations also have errors, which are determined by human subjective judgments. Therefore, the methods developed based on such samples and labels inevitably still contain errors, which cannot be further improved through model tuning and data augmentation. It is recommended to try comparison with other conventional methods, and develop research methods to improve the accuracy of sample data;
2) The manual observations quantify the results into 0-10 tenths. From the observation perspective, the resolution of such cloud cover observations is slightly low. Using 1 tenth to 1.9 tenths may have a difference of 9 percentage points, which needs further improvement in the observation accuracy of cloud cover for refined observations and forecasts.
（3）Based on traditional image processing methods like NBR threshold, the accuracy is usually lower during sunrise and sunset due to the influence of scattering near the sun. This study also found larger errors during this period. What are the reasons? Is it because there are fewer manual observation samples?
Citation: https://doi.org/10.5194/amt-2023-131-RC2 -
AC2: 'Reply on RC2', Bu-Yo Kim, 31 Aug 2023
This paper proposes a novel method to estimate ground-level cloud cover using images from a ground-based sky imager and convolutional neural network, as an alternative to human observations. The topic is of great value and the methodology is generally sound. The paper is well structured and well written. The study provides a feasible technical approach for automated monitoring of surface cloud cover. In summary, the following issues need further improvement:
Thank you for reviewing this manuscript. Based on the reviewer's comments, we have added some content to the revised manuscript, as detailed below. We believe that the quality of the manuscript has been improved and the clarity of the manuscript has been enhanced after addressing the reviewer's comments.
（1）The input data of photos were sampled at 1 hour intervals, corresponding to the manual observations. It is unclear whether the sampling time is exactly the same as the time of manual observation. Clouds can change rapidly in a short period of time, which may lead to inaccurate sample data if there is a time discrepancy between the photos and observations.
We agree with the reviewer's comments. The KMA ground weather observation guidelines stipulate that observations should be conducted for every on the hour. However, the timing of observation may vary depending on the observer, which can lead to inconsistencies and potential inaccuracies in the collected data. Unfortunately, such information has not been documented, preventing us from determining the precise observation time.
（2）This study utilizes CNN to establish the model, with images as input and manual observations as true value labels. The method is relatively effective. However, there are two aspects of problems:
1) Manual observations also have errors, which are determined by human subjective judgments. Therefore, the methods developed based on such samples and labels inevitably still contain errors, which cannot be further improved through model tuning and data augmentation. It is recommended to try comparison with other conventional methods, and develop research methods to improve the accuracy of sample data;
2) The manual observations quantify the results into 0-10 tenths. From the observation perspective, the resolution of such cloud cover observations is slightly low. Using 1 tenth to 1.9 tenths may have a difference of 9 percentage points, which needs further improvement in the observation accuracy of cloud cover for refined observations and forecasts.
We agree with the reviewer's comments on 1) and 2). The KMA-observed cloud cover was recorded as 0-8 oktas or 0-10 tenths according to the WMO synoptic observation guidelines. The employed observation method, as also noted by the reviewer, has a low resolution of quantitative values; hence, the margin of error is a few % in the same cloud cover class. Therefore, we believe that it would be possible to estimate the cloud cover with high accuracy if the pixel information (with or without clouds) in the image is extracted and learned in terms of the percent cloud cover as the label. In this study, the data resolution was rather low because the OBS cloud cover was used as a label. Therefore, we have added the following sentence:
L301 “Depending on the characteristics of the data to be learned, it is possible to estimate the cloud cover in % instead of oktas and tenths; accordingly, the time resolution can also be estimated in minutes rather than hourly intervals.”
（3）Based on traditional image processing methods like NBR threshold, the accuracy is usually lower during sunrise and sunset due to the influence of scattering near the sun. This study also found larger errors during this period. What are the reasons? Is it because there are fewer manual observation samples?
We used the DL method to compensate for the shortcomings of the threshold method and the ML method. The estimation accuracy of the cloud cover in this study was better than that of the previously employed threshold method and ML method. However, the accuracy was still relatively low at sunrise/sunset time. This is because, as noted by the reviewer, the image was polluted due to the scattering of light near the sun, making it difficult to distinguish clouds from the sky. Therefore, we added the following sentence to the main text on possible methods to improve the accuracy of cloud cover estimation at sunrise/sunset time:
L223 “Moreover, in this study, images acquired at sunrise/sunset time accounted for 16.23% of all learning datasets. In other words, the images acquired at sunrise/sunset time were learned 2 to 3 times lower for each cloud cover class than images acquire at daytime and nighttime. Therefore, it is expected that the cloud cover estimation accuracy can be improved if more images are acquired and learned during sunrise/sunset time.”
Citation: https://doi.org/10.5194/amt-2023-131-AC2
-
AC2: 'Reply on RC2', Bu-Yo Kim, 31 Aug 2023
-
AC1: 'Reply on RC1', Bu-Yo Kim, 31 Aug 2023
General Comments:
Accurate cloud cover observations are crucial for analyzing the macro and micro characteristics of clouds and studying their impact on weather and climate. This study utilizes a camera-based imager and convolutional neural network (CNN) as a prospective alternative to ground-based human-eye observations. The optimized CNN model employed in this research yields favorable performance metrics, exhibiting relatively smaller errors compared to traditional alternatives, thereby demonstrating certain advantages and holding scientific value. However, in comparison to other studies that employ machine learning algorithms for cloud cover analysis (including the author's previous works), the improvements presented in this paper are relatively limited, and the level of innovation is less prominent. Therefore, it is recommended to reconsider the acceptance of this study after the following issues have been solved.
Thank you for reviewing this manuscript. Based on the reviewer's comments, we have added some content to the revised manuscript, as detailed below. We believe that the quality of the manuscript has been improved and the clarity of the manuscript has been enhanced after addressing the reviewer's comments.
Specific Comments:
1) Compared to the study introduced in Kim et al. (2021b), the improvement in cloud cover analysis achieved by the algorithm presented in this study appears to be limited. Please provide a detailed explanation of the novelty and necessity of this study.
Content related to novel attempts, comparison with the observation, and the necessity of our research has been added to the second paragraph of Section 2.1, first paragraph of Section 2.2, first paragraph of Section 3, and the last three sentences of Section 5. We also added the following content to Section 5 for greater emphasis:
L290 “In this study, a novel method was attempted to learn a DL model for cloud cover estimation. Compared to the datasets of previous studies (e.g., Fa et al., 2019; Kim et al., 2021b; Xie et al., 2020; Ye et al., 2022), more images were learned, and long-term estimated data were analyzed. Furthermore, the estimated results were compared with observational data from a real weather station and other remote observational data (i.e., from a satellite and ceilometer).”
2) Figure 5&6: From these figures, it can be observed that the algorithm tends to underestimate cloud cover when compared to the observed results. Please provide an explanation for this phenomenon.
We have revised the manuscript by adding the following sentences:
L209 “In general, the daytime and nighttime CNN cloud cover did not exhibit a large difference compared with the OBS cloud cover; however, the bias and RMSE were relatively large, and R was low during sunrise/sunset time.”
L215 “In particular, there were many cases where the CNN cloud cover was smaller than the OBS cloud cover in the 9 and 10 tenths classes during sunrise/sunset time. The mean SZAs of the two classes were 85.13° and 98.11°, respectively, and the error was relatively large when the sun moved completely above and below the horizon (i.e., at 7–8 LST and 19–20 LST, respectively).”
3) P2L4: satellites can’t be categorized into “ground-based methods”.
We have revised existing text as follows:
L33 “Remote and automatic observation as well as estimation of the cloud cover on the ground can be achieved using meteorological satellites (SATs), ceilometers (CEIs), and camera-based imagers.”
Citation: https://doi.org/10.5194/amt-2023-131-AC1
-
RC2: 'Reply on RC1', Anonymous Referee #2, 15 Aug 2023
Peer review completion
Bu-Yo Kim et al.
Bu-Yo Kim et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 228 | 58 | 17 | 303 | 6 | 8 |
- HTML: 228
- PDF: 58
- XML: 17
- Total: 303
- BibTeX: 6
- EndNote: 8
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1