Preliminary verification for application of a support vector machine based cloud detection method to GOSAT-2 CAI-2

The Greenhouse Gases Observing Satellite (GOSAT) was launched in 2009 to measure global atmospheric CO2 10 and CH4 concentrations. GOSAT is equipped with two sensors: the thermal and near-infrared sensor for carbon observation (TANSO)-Fourier transform spectrometer (FTS) and TANSO-cloud and aerosol imager (CAI). The presence of clouds in the instantaneous field of view of the FTS leads to incorrect estimates of the concentrations. Thus, the FTS data suspected to have cloud contamination must be identified by a CAI cloud discrimination algorithm and rejected. Conversely, overestimating clouds reduces the amount of FTS data that can be used to estimate greenhouse gases concentrations. This is 15 a serious problem in tropical rainforest regions, such as the Amazon, where the amount of useable FTS data is small because of cloud cover. Preparations are continuing for the launch of the GOSAT-2 in fiscal year 2018. To improve the accuracy of the estimates of greenhouse gases concentrations, we need to refine the existing CAI cloud discrimination algorithm: Cloud and Aerosol Unbiased Decision Intellectual Algorithm (CLAUDIA1). A new cloud discrimination algorithm using a support vector machine (CLAUDIA3) was developed and presented in another paper. Visual inspection can use the locally optimized 20 standards for judging, although CLAUDIA1 and CLAUDIA3 use common thresholds all over the world. Thus, the accuracy of visual inspection is better than that of these algorithms in most regions, with the exception of snow and ice covered surfaces, where there is not enough spectral contrast to distinguish cloud. For the reason visual inspection can be used for the truth metric for the verification exercise. In this study, we compared between CLAUDIA1-CAI and CLAUDIA3-CAI for various land cover types, and evaluated the accuracy of CLAUDIA3-CAI by comparing the both of CLAUDIA1-CAI and 25 CLAUDIA3-CAI against visual inspection of the same CAI images in tropical rainforests. Comparative results between CLAUDIA1-CAI and CLAUDIA3-CAI for various land cover types indicated that CLAUDIA3-CAI had tendency to identify bright surface and optically thin clouds, however, misjudge the edges of clouds as compared with CLAUDIA1-CAI. The accuracy of CLAUDIA3-CAI was approximately 89.5 % in tropical rainforests, which is greater than that of CLAUDIA1-CAI (85.9 %) for the test cases presented here. 30 Atmos. Meas. Tech. Discuss., https://doi.org/10.5194/amt-2017-464 Manuscript under review for journal Atmos. Meas. Tech. Discussion started: 22 January 2018 c © Author(s) 2018. CC BY 4.0 License.

bands (Ishida et al., 2011a). Meanwhile, the FTS has a 2 μm band that contains many strong water vapour absorption bands.
Moreover, the CAI L2 cloud flag product may not be sensitive enough to detect clouds of sub-pixel size in ocean observations. To cope with these difficulties, the FTS data suspected to have cloud contamination are identified by two additional tests: the 2 μm band test and the CAI coherent test (Yoshida et al., 2010). Conversely, overestimation of clouds reduces the amount of the FTS data that can be used to estimate greenhouse gas concentrations. This is a serious problem in 5 tropical rainforest regions, such as the Amazon, where there is a small amount of suitable FTS data (approximately 3 % of the number of observations) because of cloud cover (Figs. 1, 2). For the reason we need to optimize thresholds between cloud and clear-sky because there are tradeoffs in maximizing accuracy while minimizing overlook and overestimate. To solve the problem, a new cloud discrimination algorithm (CLAUDIA3) using a support vector machine (SVM) (Vapnik and Lerner, 1963) was developed (Ishida et al., 2018). CLAUDIA3 can automatically identify the optimized thresholds only 10 using obvious clear-sky training data although CLAUDIA1 needs to optimize thresholds in manually. Verification was also performed by comparing with the MODIS cloud mask algorithm (Ackerman et al., 2010) and ceilometer data provided by Atmospheric Radiation Measurement (ARM) (Mather and Voyles, 2013) in the paper (Ishida et al., 2018). Furthermore the impact of different Support Vector generation procedures on cloud discrimination using CLAUDIA3 has also been evaluated in a previous study (Oishi et al., 2017). 15  The accuracy of CLAUDIA1-CAI was evaluated by comparing it with the MODIS/Aqua cloud mask data product 10 (MYD35) (Ackerman et al., 2010) because the MODIS cloud mask algorithm uses a larger number of bands for cloud Atmos. Meas. Tech. Discuss., https://doi.org/10.5194/amt-2017-464 Manuscript under review for journal Atmos. Meas. Tech. Discussion started: 22 January 2018 c Author(s) 2018. CC BY 4.0 License. discrimination than CLAUDIA1-CAI, and CLAUDIA1 was developed based on the MODIS cloud mask algorithm (Taylor et al., 2012;Ishida et al, 2011b). However, these comparisons cannot identify common weak points in the algorithms and another verification method is required. Visual inspection can use locally optimized standards for judging, whereas CLAUDIA1 and CLAUDIA3 use the common thresholds globally. Thus, the accuracy of visual inspection is better than that of these algorithms in most regions, with the exception of snow and ice covered surfaces, where there is not enough spectral 5 contrast to distinguish cloud. For the reason visual inspection can be used for the truth metric for the verification exercise.
Therefore, the accuracy of CLAUDIA1-CAI also has been evaluated by visual inspection in tropical rain forests (Oishi et al., 2014). In this study, we deal with the application of the CLAUDIA3 to GOSAT CAI data. And then, we compare between CLAUDIA1-CAI and CLAUDIA3-CAI for various land cover types, and evaluate the accuracy of CLAUDIA3-CAI by comparing the both of CLAUDIA1-CAI and CLAUDIA3-CAI against visual inspection of the same CAI images in tropical 10 rainforests.

Study area and data
The study area for various land cover types is the same as the previous study (Oishi et al., 2017) (Fig. 3).
The total forest area in the Amazon, Congo, and Southeast Asia rainforest basins is over 13 million km 2 , which 15 corresponds to one-third of the total global forest area (FAO and ITTO, 2011). The three most forest-rich countries (Brazil, Democratic Republic of Congo, and Indonesia) account for 57 % of the total global forest area (FAO and ITTO, 2011).
However, the total net emissions of carbon from tropical deforestation and land use were estimated to be 1.0 Pg-C/yr in the three rainforest basins (Baccini et al., 2012). In particular, Brazil and Indonesia have by far the highest and second highest deforestation rates, respectively (Fig. 4). Therefore, the study area for rainforests is Borneo and the Amazon (Fig. 5). 20 GOSAT returns to a similar footprint after 44 orbits (44 CAI paths) in three days. The satellite ground path of one orbit is divided into 60 equidistant CAI frames. We used the GOSAT CAI L1B product, which general users could download from the GOSAT User Interface Gateway (GUIG, https://data.gosat.nies.go.jp), for various land cover types on the beginning of the month from 2012 to 2014 in the same as the previous study (Oishi et al., 2017) (Table 2), and for rainforests (Table 3).
Currently GUIG has been changed to GOSAT Data Archive Service (GDAS, https://data2.gosat.nies.go.jp/index_en.html). 25 The spatial resolution of these products (pixel size at nadir) is 500 m, the image size is 2048 × 1355 pixels (approximately 1000 × 680 km). The CLAUDIA algorithm requires a land/sea mask and a surface albedo data. The CAI L1B product includes the Shuttle Radar Topography Mission's 15″ land/sea mask. For areas with latitudes higher than ±60°, the USGS Global Land 1-KM AVHRR Project mask is used. Surface albedo data at 1/30° resolution was generated from the CAI L1B data from 10 recurrent cycles by separating the land and water regions. This processing consists of three steps (Ishihara and 30 Nobuta, 2013): (1) calculate the minimum reflectance to remove cloud-contaminated pixels; (2) cloud shadow correction (Fukuda et al., 2013)

CLAUDIA1
CLAUDIA1-CAI comprises the calculation of clear-sky confidence levels (CCL) for every threshold test and their comprehensive integration (Ishida and Nakajima, 2009). Integrated-CCL of 0 means that the pixel is cloudy and 1 means that the pixel is cloud-free. Ambiguous pixels between cloudy and cloud-free are described by numerical values from 0 to 1. The threshold below which the integrated-CCL counts the pixel as cloud-free for GOSAT FTS L2 is 0.33, otherwise the pixel is 5 regarded as cloudy (Yoshida et al., 2010). The flow of the algorithm is shown in Fig

New cloud discrimination algorithm (CLAUDIA3)
CLAUDIA1 performs cloud discrimination by using thresholds set based on experience. The new cloud discrimination 5 algorithm (CLAUDIA3, Ishida et al., 2018) uses SVM to decide the thresholds objectively by using multivariate analysis.
SVM is a supervised pattern recognition method. First, it determines the following items using training samples of typical clear and cloudy pixels: 1) a decision function to discriminate between two classifications (clear and cloudy), 2) the thresholds, and 3) the support vectors, which are training samples specified by the decision function. The support vectors are decided in a high-dimensional feature space of the training samples. Next, it performs cloud discrimination by using the 10 decision function, thresholds, and support vectors it determined. CLAUDIA3 applies the kernel trick (Boser et al., 1992) to soft-margin SVM (Cortes and Vapnik, 1995). The kernel uses a second-order polynomial (Eq. (1)) , 2 where K is the kernel function, x i is the support vectors, and x is input data. The flow of CLAUDIA3-CAI is explained in Fig.   7. For CLAUDIA3-CAI, an integrated-CCL of 0.5 corresponds to the separating hyperplane of clear support vectors and 15 cloudy support vectors. In this study, we used two kinds of support vector: (1) support vectors generated by using MODIS data in February for cloud discrimination between November and April, and (2)   The analysis procedure consists of the following steps (Fig. 8).
2) Perform visual inspection of the pixels cut from the CAI L1B images.
We performed a visual inspection of the presence or absence of clouds in every pixel.
3) Perform cloud discrimination by using CLAUDIA1-CAI and CLAUDIA3-CAI. 10 For CLAUDIA1-CAI, we produced output images setting the integrated-CCL threshold to 0.33. For CLAUDIA3-CAI, we produced output images setting the integrated-CCL threshold to 0.5.

4) Compare output with visual inspection.
We coloured the images by comparing the visual inspection images with the output images pixel-by-pixel. In this study, "accuracy" is defined as the ratio of the number of pixels for which the standard image and output from the cloud discrimination algorithm agree to the total number of pixels in the input image. "Overlook" is defined as the ratio of the number of pixels judged clear in the output and cloudy in the standard image to the number of pixels that were judged cloudy in the standard image. "Overestimate" is defined as the ratio of the number of pixels judged cloudy in the output and 15 ( 4 ) 5 Figure 9 shows the monthly average accuracy, overlook, and overestimate for an integrated-CCL threshold of 0.33 for CLAUDIA1-CAI and 0.5 for CLAUDIA3-CAI. We used the CLAUDIA1-CAI result as the standard image.

Results for various land cover types
In Australia and Algeria, Overlook was greater than Overestimate; there was tendency that CLAUDIA3-CAI judged clear, despite CLAUDIA1-CAI judged cloudy. 10 In Japan, Borneo, Canada, and Alaska, Overestimate was greater than Overlook; there was tendency that CLAUDIA3-CAI judged cloudy, despite CLAUIDA1-CAI judged clear.
In Thailand and Mongolia, there was seasonal variation. In Thailand, Overlook was greater than Overestimate from March to May, and Overestimate was greater than Overlook from June to February. In Mongolia, Overestimate was greater than Overlook from February to March, and Overlook was greater than Overestimate from April to January. 15 Figure 10 compares the output images of CLAUDIA1-CAI and CLAUDIA3-CAI for select cases in each region.
In Australia and Algeria, CLAUDIA3-CAI could identify bright surface, however, there were a few oversights of the edges of clouds.
In Japan, CLAUDIA3-CAI misjudged vegetation areas as clouds.
In Borneo, CLAUDIA3-CAI could identify optically thin clouds. 20 In Canada and Alaska, they were snow or ice covered scenes. Since the CAI is not equipped with any thermal infrared bands, cloud discrimination based on the temperature at the top of clouds is not feasible. Accordingly, it is difficult to discriminate between ice or snow and clouds. The difference or coincidence between CLAUDIA1-CAI and CLAUIDA3-CAI was attributed to this source of error.
In Thailand, CLAUDIA3-CAI could judge smokes as non-clouds, despite CLAUDIA1-CAI judged clouds, however, there 25 were oversights of optically thin clouds and the edges of clouds on 3 April 2013. Furthermore CLAUDIA3-CAI misjudged clear muddy rivers and boundaries between land and water as cloudy. This was also reported about CLAUDIA1-CAI in   Figure 11 compares the visual inspection images and the output images for four select cases in the Amazon: low cloud cover, high cloud cover, small scattered clouds, and optically thin clouds. We used the visual inspection result as the standard image.

Results in the Amazon
CLAUDIA3-CAI produced fewer overlooked clouds but slightly more overestimated clouds than CLAUDIA1-CAI did.    Table 4 shows the results for an integrated-CCL threshold of 0.33 for CLAUDIA1-CAI and 0.5 for CLAUDIA3-CAI, and 5 Table 5 shows the results for an integrated-CCL threshold of the maximum accuracy values in Fig. 12 (CLAUDIA1-CAI: 0.75, CLAUDIA3-CAI: 0.5). There was no notable change in the accuracies with the season or location. When the integrated-CCL threshold was 0.33 for CLAUDIA1-CAI and 0.5 for CLAUDIA3-CAI, the accuracies were 87.0 % and 92.0 %, respectively. When the accuracy of CLAUDIA1-CAI was higher than that of CLAUDIA3-CAI, optically thick clouds covered a wide area of the input images. Furthermore, when the integrated-CCL threshold was 0.75 for CLAUDIA1-10 CAI and 0.5 for CLAUDIA3-CAI, the accuracy was the highest, at 88.3 % and 92.0 %, respectively. In the both cases, the accuracy of CLAUDIA3-CAI was higher than that of CLAUDIA1-CAI.   Figure 13 compares the results of the visual inspection images and the output images for two select cases in Borneo: small scattered clouds and optically thin clouds. We used the visual inspection result as the standard image. The comparison of the results for Borneo is similar to that for the Amazon. Figure 14 shows the average accuracy, overlook, and overestimate of all 5 data for all cases in Borneo. These results indicate that the most suitable integrated-CCL thresholds are 0.85 for the CLAUDIA1-CAI and 0.35 for CLAUDIA3-CAI in Borneo. Since curved lines of overestimate and overlook intersect as same as the Amazon cases, CLAUDIA3-CAI can appropriately determine the boundary between cloud and clear-sky.   Table 6 shows the results for an integrated-CCL threshold of 0.33 for CLAUDIA1-CAI and 0.5 for CLAUDIA3-CAI, and 10 Table 7 shows the results for an integrated-CCL threshold of the maximum accuracy values in Fig. 14  and 0.35 for CLAUDIA3-CAI, the highest accuracies of 87.5 % and 88.8 %, respectively, were obtained. In both cases, the accuracy of CLAUDIA3-CAI was greater than that of CLAUDIA1-CAI.

Discussions and conclusions
Comparative results between CLAUDIA1-CAI and CLAUDIA3-CAI for various land cover types indicated that CLAUDIA3-CAI had tendency to identify bright surface and optically thin clouds, however, misjudge the edges of clouds as compared with CLAUDIA1-CAI. There are tradeoffs in maximizing accuracy while minimizing overlook and overestimate. 5 Thus, it is sufficient to change the integrated-CCL threshold according to purpose. Furthermore, CLAUDIA3-CAI misjudged vegetation areas as clouds in Japan. It is necessary to add clear training data of Japanese vegetation areas for CLAUDIA3.
The averaged accuracy of CLAUDIA3 used with GOSAT CAI data (CLAUDIA3-CAI) was approximately 89.5 % in tropical rainforests, which was greater than that of CLAUDIA1-CAI (85.9 %) for the test cases presented here. This is 10 mainly because, in contrast to CLAUDIA1-CAI, CLAUDIA3-CAI can detect optically thin clouds and the edges of clouds, which prevents cloud-contaminated FTS-2 data from being processed as cloud-free FTS-2 data in the greenhouse gas concentration calculations. However, CLAUDIA3-CAI tends to overestimate the surroundings of clouds, which are judged to be cloudy despite being clear. Thus, CLAUDIA3-CAI is not expected to increase the amount of the FTS-2 data that can be used to estimate greenhouse gas concentrations in tropical rainforests. Conversely, CLAUDIA3-CAI may be able to detect 15 optically thin clouds that cannot be detected by visual inspection.
CLAUDIA3-CAI misjudged clear muddy rivers and boundaries between land and water as cloudy in the same manner as CLAUDIA1-CAI. This has three possible causes: (1) insufficient training data for muddy rivers to distinguish the differences in the spectral reflectance properties of muddy water and other water; (2) deviation of the positions in each CAI band owing to the band-to-band registration error; and (3) insufficient resolution of the surface albedo data. The surface albedo data was 20 generated at 1/8° resolution by separating the land and water region. If the border pixels between land and water regions were mixed pixels, the albedo data of 1/8° areas that include the mixed pixels would be included. To decrease this effect, higher resolution surface albedo data are needed. For boundaries between land and water, the resolution of surface albedo Atmos. Meas. Tech. Discuss., https://doi.org/10.5194/amt-2017-464 Manuscript under review for journal Atmos. Meas. Tech. Discussion started: 22 January 2018 c Author(s) 2018. CC BY 4.0 License. data is being investigated because it may be the main problem; the misjudged regions and grid pattern of albedo data match.
CLAUDIA3-CAI is more sensitive to differences between land and water than CLAUDIA1-CAI because there is a large difference in the structure of support vectors between land and water. However, generating higher resolution surface albedo data from CAI L1B data for 10 recurrent cycles cannot completely remove clouds in the minimum reflectance calculation.
To solve this, initially we need to confirm whether 500 m resolution albedo data should be used. If necessary, we will 5 develop a new method for generating surface albedo data. For example, simple cloud discrimination could be added to calculate the minimum reflectance, and if it is a cloud-contaminated pixel then the pixel is replaced by a minimum reflectance pixel, which is calculated form the same month in several years.
Although we used MODIS data as training images to generate support vectors in this study, the MODIS data and CAI data depend on observation conditions. In future work, we will use CAI data as training images to perform cloud discrimination 10 for CAI data. Furthermore, we will verify CLAUDIA3-CAI by using global CAI data with an alternative method. For instance, comparison with satellite LiDAR data, such as CALIPSO, because it is impossible to perform visual inspection of global data and visual inspection is also not itself perfect. Addressing these points will make CLAUDIA3-CAI more reliable for GOSAT-2 CAI-2 cloud discrimination.