In this study, a digital camera-based automatic cloud observation system (ACOS) was developed using a Canon EOS 6D camera to detect and calculate cloud cover for 24 h, as shown in Fig. 1. This system was developed by the National Institute of Meteorological Sciences (NIMS)/Korea Meteorological Administration (KMA) and A & D ⋅ 3D Co., Ltd. (Kim and Cha, 2020). The ACOS was installed at the Daejeon Regional Office of Meteorology (DROM; 36.37∘ N, 127.37∘ E), a manned observatory in which cloud cover observation by human eye is performed. The detailed ACOS specifications are listed in Table 1. The International Organization for Standardization (ISO) values of the complementary metal oxide semiconductor (CMOS) sensor employed in the camera are 100 (day)–25 600 (night), and the sensitivity is adjusted according to the image brightness. In this study, the camera shutter speed was set to 1/1000 s (day)–5 s (night), considering the long exposure for object detection required at night. The F-stop value was set to F8 (day)–F11 (night), and the sky-dome object was taken with a large depth of field (Peng et al., 2015; Dev et al., 2017). The camera lens was installed at a height of 1.8 m, similar to human-eye height, and a fisheye lens (EF 8–15 mm F/4L fisheye USM) was installed to capture the entire surroundings, including the sky and clouds, within a 180∘ FOV. To perform 24 h continuous observation, heating (below -2 ∘C) and ventilation devices were installed inside the ACOS body to facilitate image acquisition without manned management (Dev et al., 2015; Kim and Cha, 2020).

The image data captured by ACOS were processed by converting each RGB channel of each image pixel to a brightness of 0–255. Although the camera-lens FOV was 180∘, only pixel data within the zenith angle of 80∘ (FOV 160∘) were used. This condition was in consideration of the permanent masking area of the horizontal plane due to surrounding objects (buildings, trees, equipment, etc.) (Kazantzidis et al., 2012; Shields et al., 2019; Kim and Cha, 2020). For cloud cover calculation using the ACOS images, image data taken at 1 h intervals from 1 January to 31 December 2019 were used. The cloud cover was calculated using the statistical characteristics of the RGB brightness ratio (i.e., the red–blue ratio – RBR), difference (i.e., the blue–red difference – BRD), and luminance (Y), which vary for each image (Sect. 2.3), as well as supervised machine learning methods (Sect. 3). Here, Y was calculated as Y=0.2126R+0.7152G+0.0722B (Sazzad et al., 2013; Shimoji et al., 2016). The calculated cloud cover was compared with human-eye observation data from DROM. As the cloud cover was calculated as a percentage between 0 % and 100 %, the result was converted to an integer (tenth) between 0 and 10 (Table 2) for comparison with the human-eye-based cloud cover values. As the ACOS was installed at DROM, there were no location differences between observers; thus, the same clouds were captured (Kim and Cha, 2020). At DROM, night observations were performed at 1 h intervals during inclement weather (rainfall, snowfall, etc.), but otherwise at 3 h intervals. The night period varied with the season. Considering this, a total of 7402 images of concurrent human observations were collected, excluding missing cases, from the ACOS.

The entire collected dataset was randomly sampled without replacement. Overall, 50 % (3701 cases) of the total data elements were configured as a training set, 30 % (2221 cases) as a validation set, and 20 % (1480 cases) as a test set (Xiong et al., 2020). The training set was used to train the machine learning algorithms, and the prediction performance of each machine learning method was assessed using the validation set. Optimal hyper-parameters were set for each machine learning method through the training and validation sets. The results of each machine learning method were compared. In this process, the test set was input to the machine learning model that exhibited the best prediction performance, and the calculated results and human-eye observation data were compared. The accuracy, recall, precision, bias, root mean square error (RMSE), and correlation coefficient (R) were analyzed according to Eqs. (1)–(6); hence, the prediction performance of each machine learning method was determined and compared based on the human-eye observation data. 1Accuracy=TP+TNTP+TN+FP+FN2Recall=TPTP+FN3Precision=TPTP+FP4Bias=∑(M-O)N 5RMSE=∑(M-O)2N6R=∑M-M‾O-O‾∑M-M‾2∑O-O‾2 Here, TP, TN, FP, and FN are the number of true positives (reference: yes, prediction: yes), true negatives (reference: yes, prediction: no), false positives (reference: no, prediction: yes), and false negatives (reference: no, prediction: no), respectively. Further, M, O, and N are the cloud cover calculated by the employed machine learning method, the human-eye-observed cloud cover, and the number of data, respectively.

The data input to the machine learning algorithms for cloud cover calculation using the ACOS images were produced as follows. First, as the ACOS image was taken with a fisheye lens, the image was distorted. That is, objects at the edge were smaller than those at the center of the image (Chauvin et al., 2015; Yang et al., 2015; Lothon et al., 2019). Therefore, the relative size of each object in the image was adjusted through orthogonal projection distortion correction according to the method expressed in Eqs. (7)–(11) (Kim and Cha, 2020). 7r=(x-cx)2+(y-cy)2,8θ=asin(r/radi),9ϕ=asin((y-cy)/r),10x′=cx+r×θ×cos⁡(ϕ),11y′=cy+r×θ×sin⁡(ϕ), where r is the distance between the center pixel (cx, cy) of the original image and each pixel (x, y), θ is the solar zenith angle (SZA), “radi” is the image radius (distance between center and edge pixel of circular images), ϕ is the azimuth, and x′ and y′ are the coordinates of each pixel after distortion correction.

Second, surrounding masks such as buildings, trees, and equipment, as well as light sources such as the sun, moon, and stars, were removed from the image (building, tree, and equipment: masking was performed when the mean RGB brightness was less than 60 in the daytime on a clear day; light source: masking was performed when the mean RGB brightness exceeded 240). These objects directly mask the sky and clouds or make it difficult to distinguish them; therefore, they must be removed when calculating cloud cover (Yabuki et al., 2014; Kim et al., 2016; Kim and Cha, 2020). Third, the RBR, BRD, and Y frequency distributions were calculated using the RGB brightness of each pixel of image data subjected to pre-processing (i.e., masking removal and distortion correction). The class interval sizes of the RBR, BRD, and Y frequency distributions were set to 0.02, 2, and 2, respectively, and classes with frequencies of less than 100 were ignored. Statistical characteristics of the mean, mode, frequency of mode, kurtosis, skewness, and quantile (Q0–Q4: 0 %, 25 %, 50 %, 75 %, and 100 %) data obtained for each frequency distribution were used as input for each machine learning method. As input data for machine learning, time information (Julian day and hour) allowed differentiating seasons and day and night. Further, SZA should be considered because the colors of the sky and clouds change according to the position of the sun (Blazek and Pata, 2015; Cazorla et al., 2015; Azhar et al., 2021). As these image data features have different appearances under different conditions (cloud cover, day, night, etc.), they constitute an important variable in machine learning regression for cloud cover calculation (Heinle et al., 2010; Li et al., 2011).

Figure 2 shows distortion-corrected images for day and night cloud-free, overcast, and partly cloudy cases and the RBR, BRD, and Y relative frequency distributions. The frequency distributions were expressed as percentages over approximately 310 000 px excluding the masked area. Human-eye observations at DROM yielded cloud-free (Fig. 2a and g), overcast (Fig. 2b and h), and partly cloudy (Fig. 2c and i) case values of 0, 10, and 5 tenths, respectively. As for the RBR frequency distribution during the day, larger RBR distributions were observed for the overcast than cloud-free case, and bimodal distributions including both (i.e., overcast and cloud-free) distributions were obtained for the partly cloudy case. The variance was large in the partly cloudy case. With regard to the BRD frequency distribution, the blue-channel brightness increased with Rayleigh scattering, such that the cloud-free case with many sky pixels had larger BRD distribution than the overcast case (Ghonima et al., 2012; Kim et al., 2016). In contrast, the Y frequency distribution was relatively large for the overcast case, which involved many cloud pixels. Although the RBR frequency distributions at night and day were similar, the RBR was larger at night because the red-channel brightness increased under the influence of Mie scattering (Kyba et al., 2012; Kim and Cha, 2020). A negative BRD distribution was obtained from the cloud pixels. At night, there is no light source such as the sun. Therefore, in this study, RGB brightness close to black (0, 0, 0) was distributed in the cloud-free case, yielding small Y. As the images obtained through ACOS had different RBR, BRD, and Y frequency distribution classes and shapes for each case, it was necessary to train these data features (i.e., the mean, mode, frequency of mode, kurtosis, skewness, and quantile of each frequency distribution) on a machine learning model to calculate the cloud cover.

The method in which the relationship of the dependent variable to the independent variable is regressed by considering one independent variable only is called simple linear regression, and the method in which the change in the dependent variable is predicted based on the changes in two or more independent variables is called MLR. An MLR model with k independent variables predicts the dependent variable as shown in Eq. (12), using the least squares method which minimizes the predictor variable and the sum of squared errors (Fig. 3a) (Olive, 2017). In this study, we used the R “glm” package (Geyer, 2003). 12Yi=β0+β1X1i+β2X2i+⋯+βkXki,i=1,2,⋯,N, where βk are the population coefficients (i.e., parameters), and Xki is the kth predictor of the ith observation (a value that describes the variable Yi to be predicted). In this study, the independent variables were the RGB, BRD, and Y mean, mode, frequency of mode, skewness, kurtosis, quantile, as well as the Julian day, hour, and SZA. The dependent variable was the cloud cover observed by human eyes.

The kNN method involves non-parametric, instance-based learning and is one of the simplest predictive models in machine learning. The kNN algorithm finds the k-nearest neighbors to the query in the data feature space, as shown in Fig. 3b, and then predicts the query with distance-based weights (S. Zhang et al., 2018). That is, a set of independent variables is constructed as a cluster, and values corresponding to each neighbor are weighted according to the Euclidean distance and predicted (Martínez et al., 2019). In this study, the R “class” package (Ripley and Venables, 2021a) was used, and the hyper-parameter setting was k=15.

SVR is an extended method that can be used for regression analysis by introducing an ε-insensitive loss function to an SVM. As shown in Fig. 3c, a hyperplane consisting of support vectors that can classify the maximum margin for the distance between vectors is found (Gani et al., 2010; Taghizadeh-Mehrjardi et al., 2017). The optimal hyperplane is obtained by finding w and b that minimize the mapping function (Φ(w)), as shown in Eq. (13) (Meyer, 2001). The constraints are shown in Eq. (14). Then, as in Eq. (15), the kernel is applied and mapped to a higher dimension. Here, ε determines the threshold of margin, ξ is a slack variable to allow error, and C is the allowable cost that can violate the constraint of Eq. (14). In this study, the R “e1071” package (Meyer et al., 2021) was used, the SVR kernel was set as a radial basis function (RBF), and the hyper-parameters were set to epsilon (ε)=0.12, gamma (γ)=0.04, and cost (C)=5. 13Φ(w)=min12w2+C∑i=1nξi+ξi*,14wTxi+b-yi≤ε+ξi,yi-wTxi+b-yi≤ε+ξi*,ξi,ξi*≥0,15Kxi,xj=exp⁡-γxi-xj2, where subscript i and j are ith and jth data point, respectively, and γ is a parameter that controls the RBF kernel width.

An ANN is a mathematical model that mimics a neuron; i.e., the signal transmission system of a biological neural network. As shown in Fig. 3d, this model consists of an input layer that receives input data, an output layer that outputs prediction results, and an invisible hidden layer between the two layers (Rosa et al., 2020). The hidden node of the hidden layer acts like a neuron in a neural network and is composed of weight, bias, and an activation function. In this study, we used the R “nnet” package (Ripley and Venables, 2021b), which is based on feed-forward neural networks with a single hidden layer that can rapidly learn and predict while considering nowcasting. The hyper-parameters of this package were set as follows: size (number of hidden nodes) = 7, maxit (maximum number of iterations) = 700, and decay (weight decay parameter) = 0.05.

The RF method composes N decision trees by combining randomly selected variables from each node to grow a regression tree, as shown in Fig. 3e. An ensemble of the results of each decision tree is obtained, and hence a prediction result is provided (Wright and Ziegler, 2017). That is, in the RF ensemble learning method, every individual tree of the decision tree contributes to the final prediction (Shin et al., 2020; Kim et al., 2021). In this study, the R “Ranger” package (Wright et al., 2020) was used, and the hyper-parameters were set to num.trees (the number of trees) = 510, mtry (the number of variables randomly sampled from each node) = 7, and min.node.size (minimal node size) = 5.

The GBM uses boosting instead of bagging during resampling and ensemble processes. As shown in Fig. 3f, a model with improved predictive power is created by gradually improving upon the parts that the previous model could not predict while sequentially generating weak models. The final prediction is calculated from the weighted mean of these results (Friedman, 2001). In other words, gradient boosting updates the weights iteratively to minimize the difference from the function f(x) that predicts the actual observation using gradient descent (Ridgeway, 2020). In this study, the R “gbm” package (Greenwell et al., 2020) was used; the GBM kernel was set to a Gaussian distribution function; and the hyper-parameters were set to n.trees (number of trees) = 500, interaction.depth (maximum depth of binary tree) = 5, and shrinkage (learning rate) = 0.1.

Figure 4 shows the cloud cover prediction results obtained using the training set for each machine learning method. The hyper-parameters were optimized using the training and validation sets. Each box in the figure denotes the ratio (%) of the number of observations for each cloud cover in the DROM and the number of predictions for each cloud cover in the SVR model. The higher the frequency in the diagonal one-to-one boxes, the better the agreement between the observed and predicted cloud cover. In other words, the closer the diagonal one-to-one boxes are to red (i.e., 100 %), the higher is the agreement. For the training set, the highest human-eye observation data frequency by cloud cover was 26.80 % at 0 tenths; this was followed by 19.97 % at 10 tenths and 3.65 %–11.92 % at 1–9 tenths. For the SVR model, the 0- and 10-tenth frequencies were 71.88 % and 92.15 %, respectively, with the agreement being greatest among the machine learning models. As detailed in Table 3, the SVR accuracy, recall, and precision for all cloud cover were 0.94, 0.70, and 0.76, respectively, indicating the best prediction performance. The accuracy was in the range of 0.91–0.98 for each cloud cover, whereas recall and precision were in the ranges of 0.42–0.92 and 0.24–0.99, exhibiting low predictive power in the partly cloudy case. The bias was 0.07 tenths, the RMSE was 1.05 tenths, and R was 0.96. In the case of the RF model, the 0- and 10-tenth frequencies were 61.79 % and 80.65 %, respectively, being lower than those of the SVR model; however, the prediction for 1–9 tenths exhibited high agreement to within ±1 tenth. The accuracy, recall, and precision were 0.93, 0.67, and 0.76, respectively, which is lower than SVR model, but the bias and RMSE were the smallest at 0.02 and 0.71 tenths, respectively, and the R value was the highest at 0.98. However, for the validation set, the SVR model prediction performance (accuracy: 0.88, recall: 0.41, precision: 0.51, bias: 0.06 tenths, RMSE: 1.51 tenths, and R: 0.93) was better than that of the RF model. In other words, the RF model exhibited a tendency to overfit in this study. The accuracy of these results exceeds that of the classification machine learning method (0.60–0.85) presented by Dev et al. (2016) using day and night image data, and it is higher than or similar to the accuracy (0.91–0.94) achieved using the regression and deep learning machine learning methods proposed by Shi et al. (2019, 2021) for day and night image data. Apart from the SVR and RF methods, the machine learning methods exhibited similar frequency distributions; however, the accuracy, recall, precision, and R were lower and the RMSE values were higher in the order of GBM, kNN, ANN, and MLR. In particular, the MLR model had very poor predictive power (accuracy: 0.75, recall: 0.08, and precision: 0.78) for 0 tenths using the training set.

The relative importance of the input variable of the SVR method, which exhibited the best predictive performance in this study, is shown in Fig. 5. BRDQ4 had the highest relative importance at 8.54 %, whereas RBRmode had the lowest importance at 0.55 %. Among the RBR data features, RBRQ0 had the highest importance at 7.06 %, and, among the Y data features, YQ0 had the highest importance at 3.78 %. In terms of the cumulative relative importance, the BRD-, RBR- and Y-related data features contributed 38.25 %, 31.44 %, and 26.20 % of the total (100 %), respectively, to the cloud cover prediction, and the remaining data features contributed 4.10 %. The relationship between input data features is complex to determine the optimal hyperplane of the SVR model, and the variable importance is determined so that the cloud cover can be calculated with the smallest error using the observed cloud cover (Singh et al., 2020). Even if the BRD-related data features have the same RBR characteristics, they contribute to machine learning more comprehensively by day, night, and cloud presence depending on the BRD value; therefore, they are critical to cloud cover calculation. By contrast, the Y-related data feature is sensitive to the RGB brightness (especially the G brightness) in the image, but the difference in the Y characteristics according to the cloud cover during the day was not large; thus, their importance was relatively low. Although time information and SZA can provide information such as daytime, nighttime, and sunset/sunrise images, they have the lowest importance because they do not have statistical characteristics that can be used to directly calculate cloud cover. The importance of these data features may vary depending on the camera's sensor (Kazantzidis et al., 2012).

Figure 6 shows the total cases and seasonal scatter plots of the DROM cloud cover and the ACOS cloud cover prediction calculated from the SVR model using the test set. In the Korean Peninsula, the winter cloud cover is sparse (<5 tenths) as the weather is generally clear because of the Siberian air mass. In summer, the rainy season is concentrated under the influence of the Yangtze River and Pacific air masses, and the cloud cover is dense (>5 tenths) until fall because of typhoons (Kim and Lee, 2018; Kim et al., 2020a). Furthermore, the Korean Peninsula experiences a westerly wind, cumulus heat generated in the western sea moves inland, and the cloud cover changes rapidly and continuously (Kim et al., 2021). The cloud cover distributions calculated for all test set cases exhibited good agreement with the observed cloud cover, with accuracy, recall, and precision of 0.88, 0.42, and 0.52, respectively. Further, the bias, RMSE, and R were 0.04 tenths, 1.45 tenths, and 0.93, respectively. In fall, the bias, RMSE, and R were -0.12 tenths, 1.30 tenths, and 0.95, respectively, indicating that the difference between the observed and calculated cloud cover was small. In winter and summer, the RMSE was larger and R was lower than in the other seasons. This is because the cloud cover calculation error is large at sunrise and sunset (100∘ ≥ SZA > 80∘), i.e., where daytime (SZA ≤ 80∘) and nighttime (SZA > 100∘) intersect (Lalonde et al., 2010; Alonso et al., 2014; Kim and Cha, 2020).

For the test set daytime cases, the bias, RMSE, and R were 0.10 tenths, 1.20 tenths, and 0.95, respectively, and 0.08 tenths, 1.59 tenths, and 0.93, respectively, for the night data. However, for sunrise and sunset, these values were -0.22 tenths, 1.71 tenths, and 0.90, respectively. Relatively, the bias and RMSE were large and R was low. In spring and autumn, sunrise and sunset images were learned at similar times (sunrise: 06:00–07:00 LST, sunset: 18:00–19:00 LST); however, differences between the winter (sunrise: 07:00–08:00 LST, sunset: 17:00–18:00 LST) and summer (sunrise: 05:00–06:00 LST, sunset: 19:00–20:00 LST) results are apparent because sunrise and sunset occurred late or early and exhibited different features from the data features learned for those times (Liu et al., 2014; Li et al., 2019). That is, owing to the sunrise/sunset glow, high cloud cover calculation errors are obtained at sunrise/sunset, when it is difficult to distinguish between the sky and clouds because of the reddish sky on a clear day and the bluish cloud on a cloudy day (Kim et al., 2021). Therefore, for the test set, the bias, RMSE, and R for sunrise and sunset in spring and autumn were -0.24 tenths, 1.46 tenths, and 0.93, respectively. However, in winter and summer, the bias, RMSE, and R were -0.21 tenths, 1.93 tenths, and 0.86, respectively. Nevertheless, the results of this study surpass those of Kim et al. (2016) for daytime (08:00–17:00 LST; bias: -0.36 tenths, RMSE: 2.12 tenths, R: 0.87) and Kim and Cha (2020) for nighttime (19:00–06:00 LST; bias: -0.28 tenths, RMSE: 1.78 tenths, R: 0.91) cases. Shields et al. (2019) employed different day and night cloud cover calculation algorithms. In that approach, cloud cover calculation errors may occur at sunrise and sunset. Therefore, if a day and night continuous cloud cover calculation algorithm is considered, the calculation error for this discontinuous time period should be reduced (Huo and Lu, 2009; Li et al., 2019). Figure 7 shows the daily mean cloud cover results based on the observed and calculated cloud cover for the test set. For the observed and calculated cloud cover, a bias of 0.03 tenths, RMSE of 0.92 tenths, and R of 0.96 were obtained. The coefficient of determination (R2) was 0.92, and the result calculated from the SVR model constructed in this study explained approximately 92 % of the observed data in the test set.

Figure 8 shows the frequency distribution of the differences between ACOS and DROM by season and time. In this frequency distribution, the higher the 0-tenth frequency, the higher the agreement between the observed and calculated cloud cover. The highest 0-tenth frequency was obtained in winter (46.05 %) and the lowest in spring (35.58 %), but 41.69 % agreement was obtained for all seasons. Conditioned on the time of day, high agreement of approximately 44 % was obtained for both daytime and nighttime, but the lowest agreement (30.49 %) was obtained for sunrise and sunset. Previous studies obtained a difference of approximately 2 tenths from the observed cloud cover for the cloud cover calculated based on the ground-based imager data (Kazantzidis et al., 2012; Kim et al., 2016; Kim and Cha, 2020; Wang et al., 2021). When a difference of up to 2 tenths was allowed between the observed and calculated cloud cover, the agreement was 90.95 %, as detailed in Table 4. When a difference up to 1 tenth between both cloud cover results was allowed for all cases, the agreement was 79.05 %. When the difference was within 2 tenths, high agreement of 86.59 % to 94.41 % by season and by time was obtained. These results reveal greater agreement than those obtained by Cazorla et al. (2008), Kreuter et al. (2009), Kazantzidis et al. (2012), Kim et al. (2016), Krinitskiy and Sinitsyn (2016), Fa et al. (2019), Kim and Cha (2020), Xie et al. (2020), and Wang et al. (2021). In those works, 80 %–94 % agreement was achieved when the allowed difference between the observed and calculated cloud cover was 2 oktas (2.5 tenths) or 2 tenths for day, night, and day and night cases.