Improved cloud mask algorithm for FY-3 A / VIRR data over the northwest region of China

Abstract. The existence of various land surfaces always leads to more difficulties in cloud detection based on satellite observations, especially over bright surfaces such as snow and deserts. To improve the cloud mask result over complex terrain, an unbiased, daytime cloud detection algorithm for the Visible and InfRared Radiometer (VIRR) on board the Chinese FengYun-3A polar-orbiting meteorological satellite is applied over the northwest region of China. The algorithm refers to the concept of the clear confidence level from Moderate Resolution Imaging Spectroradiometer (MODIS) and the unbiased structure of the CLoud and Aerosol Unbiased Decision Intellectual Algorithm (CLAUDIA). Six main channels of VIRR centered at the wavelengths of 0.455, 0.63, 0.865, 1.595, 1.36, and 10.8 μm are designed to estimate the degree of a pixel's cloud contamination judged by the clear confidence level. Based on the statistical data set during four months (January, April, July, and October) in 2010, seasonal thresholds are applied to improve the accuracy of the cloud detection results. Flags depicting snow and water are also generated by the specific threshold tests for special surfaces. As shown in image inspections, the cloud detection results over snow and deserts, adopting the proposed scheme, exhibit better correlations with true-color images than the VIRR official cloud mask results do. The performance of the proposed algorithm has been evaluated in detail for four seasons in 2011, using cloud mask products from MODIS and the ground-based observations. The evaluation is based on, overall, 47 scenes collocated with MODIS and 96 individual matchups between VIRR and the ground-based observations from two weather stations located in the research region. The quantitative validations suggest that the estimations of clear-sky regions have been greatly improved by the proposed algorithm, while a poor identification of the cirrus clouds occurs over deserts.


Introduction
Distinguishing cloudy from clear-sky areas has always been a challenging job for satellite remote sensing.The presence of clouds contaminates radiometric observations (Kidder and Vonder Haar, 1995), which may cause errors in the retrieval of atmospheric compositions and parameters, especially for the researches on aerosols (Kaufman et al., 2005).Considering the large effect of clouds on radiance, it is necessary to provide techniques to separate clouds from clear-sky areas.
Many cloud detection algorithms have been proposed, depending on the purposes of the observations and specifications of the imager.The International Satellite Cloud Climatology Project (ISCCP) presented statistical-threshold tests for visible and infrared radiance to differentiate cloudy and clear scenes over various climate regions (Rossow, 1989;Rossow and Garder, 1993).The Advanced Very High Resolution Radiometer (AVHRR) Processing scheme Over cLouds, Land, and Ocean (APOLLO) used two visible and three infrared bands to screen clouds (Saunders and Kriebel, 1988;Gesell, 1989;Kriebel et al., 2003).Clouds from AVHRR (CLAVR) used sequential decision-threshold tree tests to identify the pixels, which were classified into 2 × 2 global area-coverage pixel arrays over cloud-free, mixed, and cloudy regions (Stowe et al., 1991(Stowe et al., , 1994)).The dynamic threshold cloud masking (DTCM) algorithm used dynamic thresholds to screen clouds over land for AVHRR data and correctly identified a comparable or higher number Published by Copernicus Publications on behalf of the European Geosciences Union.
X. Wang et al.: Improved cloud mask algorithm for FY-3A/VIRR data over the northwest region of China of cloud-contaminated pixels compared with the CLAVR scheme (Di Vittorio and Emery, 2002).The cloud mask algorithm for the 36-channel Moderate Resolution Imaging Spectroradiometer (MODIS) was determined by the final confidence flags based on five groups of threshold tests (Ackerman et al., 1998).Based on the infrared tri-spectral algorithm, the cloud thermodynamic phase-discrimination method applied to MODIS data introduced three additional bands located at 0.65, 1.63, and 1.90 µm to improve the accuracy of phase retrieval (Baum et al., 2000).Ishida and Nakajima (2009) proposed an unbiased cloud detection scheme, based on a neutral concept, to restructure the final confidence flags of the MODIS cloud mask algorithm (MOD35) and applied the CLoud and Aerosol Unbiased Decision Intellectual Algorithm (CLAUDIA) to a virtual imager to investigate the efficiency of existing and future spaceborne imagers (Nakajima et al., 2011).The Visible Infrared Imager Radiometer Suite (VIIRS) cloud mask, derived from the automated analysis data, provided a critical data product for the NPOESS (National Polar-orbiting Operational Environmental Satellite System) program (Hutchison et al., 2005).The cloud screening algorithm for Environmental Satellite -Medium Resolution Imaging Spectrometer (ENVISAT-MERIS) multispectral imagers was based on the extraction of meaningful physical features to increase cloud detection accuracy (Gómez-Chova et al., 2007).For FengYun-3A/Visible and InfRared Radiometer (FY-3A/VIRR), the multiple-feature (single channel or multiple channels in combination) thresholds, which were determined by a dynamic histogram method or forward modeling results, were applied into the official cloud mask scheme (Yang et al., 2011).A daytime cloud detection algorithm with good performance has also been adopted for FY-3A/VIRR recently.It was an automatic daytime cloud mask technique based on the multispectral threshold synthesis method inherited from MODIS.By adding the difference between the 1.38 and 1.6 µm bands, it provided improved results for high-cloud detection (He, 2011).
To acquire more accurate cloud masking results, the potential impact of various land surfaces should also be considered in the algorithm.Especially bright surfaces, such as snow and deserts, with large reflectance similar to that of clouds could probably be classified as cloudy under clearsky conditions.Focusing on improving the cloud detection over bright surfaces, researchers have tried several methods.For discrimination of snow-covered areas, the Normalized Difference Snow Index (NDSI) has been widely used for many sensors.Defined by the two bands centered at 0.57 and 1.65 µm, NDSI was introduced for Landsat Thematic Mapper (TM) images (Dozier, 1984(Dozier, , 1989)).For the Vegetation (VGT) sensor, the Normalized Difference Snow/Ice Index (NDSII) was proposed, which was based on the reflectance values of red and mid-infrared spectral bands (Xiao et al., 2001).NDSI was also applied in the cloud detection algorithm of MODIS (Ackerman et al., 1998).By adding the cloud-phase identification, cloud detection over snow/ice for MODIS was developed (King et al., 2004).With the 1.6, 2.2, and 1.38 µm observations from MODIS, two multispectral enhancement techniques were proposed for distinguishing between regions of clouds and snow cover (Miller et al., 2005).Six extra tests were also applied for MODIS to identify clouds over desert regions in daytime imagery, which included a 13.9 µm and a 6.7 µm single brightness temperature test, a 1.38 µm reflectance test, and three brightness temperature difference tests (11 minus 12 µm, 3.7 minus 11 µm, and 3.7 minus 3.9 µm) (Ackerman et al., 1998).To improve the performance of the cloud mask in the vicinity of desert areas, data from 0.4 µm were also used based on the strong contrast between highly reflective clouds and less reflective cloudfree desert regions (Hutchison and Jackson, 2003).For watercovered regions, the Normalized Difference Vegetation Index (NDVI) was an effective discrimination method.Defined by the red and near-infrared bands, NDVI was widely applied in vegetative studies (Nemani and Running, 1989;Defries and Townshend, 1994;Carlson and Ripley, 1997).For water-covered regions, the value of NDVI was always negative compared with clouds and other primary land surfaces.Based on this fact, NDVI was also adopted to discriminate water areas (Giglio et al., 2003;Lunetta et al., 2006;Gao et al., 2007).
In this paper, an unbiased daytime cloud detection algorithm based on CLAUDIA is applied to FY-3A/VIRR data over the northwest region of China.For multispectral threshold tests, it is necessary to provide a method to regroup the results of each single test to produce the final result.According to the regrouping schemes, prevailing algorithms are designed into two main categories: clear conservative (such as APOLLO and MOD35) and cloud conservative (such as IS-CCP).The CLAUDIA algorithm provides a new concept to recombine the individual threshold test result.It refers to the multiple threshold method employed in the MODIS cloud mask scheme but reconstructs the calculation method for the final confidence flags to realize neutral results (Ishida and Nakajima, 2009).With the unbiased technique, the mixed pixels, in which clouds and surfaces are both included, are not directly identified as clouds or clear-sky areas.Instead, the clear confidence levels are applied.This technique provides a more accurate cloud mask result that is not biased to either clear or cloudy.With those advantages, the concept of the unbiased algorithm is introduced into the cloud detection for FY-3A/VIRR in our project, to improve the official cloud mask product of VIRR over the northwest region of China.Over the regions with the complex terrain, specific threshold tests are also added into the improved algorithm to separate the special clear-sky surfaces such as snow and water into distinct categories.The seasonal thresholds used in this paper are determined based on the statistical data of 2010.
This paper includes five sections.Section 2 describes the background and data we used.Section 3 provides details of the threshold experiments and descriptions of the improved cloud mask algorithm for FY-3A/VIRR.The cloud detection results and validations of the proposed algorithm are then presented in Sect. 4. Finally, Sect. 5 provides a summary and discussion.

Background
The existence of bright land surfaces, which share similar radiative characteristics with clouds, always leads to errors in cloud detection.Over snow-covered regions, the high reflectance and cold temperature make it difficult to distinguish clouds from clear-sky areas.A similar problem exists when the underlying surface is a desert.Thus, the cloud detection algorithm cannot only depend on the single wavelength threshold test.Instead, it is necessary to add more effective techniques to avoid the false identifications.In this study, the region from 34 to 40 • N and from 73 to 82 • E, which lies in the northwest region of China, was chosen for cloud detection experiments.The various land surfaces over this region (including snow/ice, desert, water, and land) always lead to errors when screening clouds.In this paper, the cloud detection results have been significantly improved by our combination of the unbiased algorithm and specific threshold tests for special surfaces over the studied region.

Data
FY-3A, China's new generation polar-orbiting meteorological satellite, is capable of a wide range of spectral detection, from ultraviolet, visible, infrared, to the microwave spectrum.It operates at the altitude of 831 km.The local time of the descending node is 10:05:00 UTC.An on-board sensor, VIRR, which has ten bands in the wavelengths from 0.44 to 12.5 µm, the scanning range of ±55.4 • and a resolution of 1.1 km, is one of its key instruments (Dong et al., 2009).
In this study, six main channels of VIRR L1 (level 1) data are used for the cloud detection algorithm (Table 1).The performance of the improved cloud mask algorithm is examined through comparisons with the VIRR official cloud mask product, the MODIS cloud mask product (MOD35), and the ground-based observations from the Meteorological Information Comprehensive Analysis and Process System (MICAPS).The VIRR official cloud mask product is generated by the multiple-feature (single channel or multiple channels in combination) threshold tests determined by a dynamic histogram method or forward modeling results (Yang et al., 2011).As one of the basic products for VIRR, it is released in hierarchical data format version 5 (HDF5), and can be obtained from the website of the National Satellite Meteorological Center (online at: fy3.satellite.cma.gov.cn).MICAPS is an interactive computer system that integrates all meteorological, satellite, and radar data into one computer workstation (Li et al., 2010).Data used for validations include the

Methodology
The cloud detection algorithm is mainly based on the CLAUDIA algorithm, which adopts the threshold concept in MOD35 but reconstructs the scheme for calculating the final confidence flag (Ishida and Nakajima, 2009;Nakajima et al., 2011).According to the channel settings of VIRR, new groups are proposed to estimate the unbiased final clear confidence level.Additional steps are also added for cloud discrimination over special surfaces.

Traditional threshold definition and tests
The thresholds in this study are determined by the statistical data obtained from original observations of VIRR onboard FY-3A in 2010.In this analysis, four months (January, April, July, and October) are selected to represent conditions in four seasons.The results are classified into five different types: snow, water, desert, land (except desert and snow), and clouds.
The traditional thresholds are determined based on the statistical results of the single channel or index.Figure 1 illustrates the concept of the estimation of the clear confidence level for the traditional threshold test.The definitions of the three thresholds are quite similar to those of MODIS: the low limit, the high limit, and the threshold T .The low and high limits define the cloudy conditions (confidence level 0) and the clear conditions (confidence level 1).These two parameters are determined by the minimum and maximum values of the overlapping intervals between the two categories according to the statistical data.The value of T determines the pass or fail threshold within two limits.It is assigned by the technique applied in the delineation of rain areas (Lovejoy and Austin, 1979).We define a loss function f as  where A and B represent the total statistical numbers of pixels from two types or categories, respectively.The value of T is the threshold to determine which type the pixel belongs to.
After the classification by T , A b is the number of A incorrectly classified as B according to T , and B a is the number of B incorrectly classified as A. As the value of T changes, the loss function f varies.When the loss function f achieves a minimum such that the sum of the incorrect classified ratio reaches the lowest value, the threshold value can be defined as the final T .

Single reflectance tests
In the visible and near infrared bands, optically thick clouds usually have greater reflectance compared with the underlying surface.This is now used as an effective method to identify cloudy areas.The channels at 0.66 and 0.88 µm, the most common bands used in this spectral range, have exhibited good performance in cloud masking.However, reflectance tests may falsely identify bright surfaces, such as deserts and snow-covered areas, as clouds.The statistical results presented here provide the reflectance distributions of the different surface types and clouds (Fig. 2). Figure 2a illustrates the histograms of the reflectance at channel 0.58-0.68µm.The reflectance of snow and clouds covers a broad range, while other types show a relatively narrow distribution.Figure 2b shows the ranges of reflectance covered by the five different types and indicates where their median values lie (shown by slash marks).The large overlap between clouds and snow generally results in difficulties in distinguishing them.However, for land, desert, and water, there is less overlap, so these surface types can be mostly differentiated from clouds and snow by the threshold tests.Thus, two major categories can be identified as follows: Cloud (including clouds and snow), and Clear (all the others).All single reflectance thresholds are determined on the basis of the classification criteria.Figure 2b also shows where the three specific thresholds are located for cloud types (clouds and snow) vs. clear types.The two black lines represent the low and high limits.The T value is indicated by a red line.
Because it is near a strong water vapor absorption region, the band 1.38 µm test is quite effective at detecting thin cirrus clouds (Gao et al., 1993).Due to the sufficient water vapor in the lower atmosphere, the channel at 1.38 µm receives little scattered solar radiance from the surface or low-level clouds.Radiance increases when cirrus clouds are located above almost any atmospheric water vapor.Taking this into consideration, the reflectance threshold test at 1.38 µm of VIRR is applied primarily to separate cirrus clouds from clear-sky areas and thick clouds.The single-reflectance thresholds used in this study are shown in Table 2.

NDSI and NDVI tests
To separate snow and water into distinct categories, NDSI and NDVI thresholds are used respectively in the cloud detection algorithm.This section focuses mainly on the threshold tests of these two indices.
While the reflectance of snow cover and clouds are very similar at wavelengths below about 1 µm, they diverge in the near infrared and achieve a maximum difference at wavelengths between about 1.55 and 1.75 µm (Rees, 2006).Thus, the normalized difference snow index (NDSI) was proposed and developed for the discrimination between snow and clouds/surface (Dozier, 1984(Dozier, , 1989)).Here, the NDSI used for VIRR is defined as where r 1 and r 6 are the reflectances at channels 1 (0.58-0.68 µm) and 6 (1.55-1.64µm).The statistical histogram shows that snow and clouds could be distinctly separated by NDSI (Fig. 3a).Compared with clouds, snow typically has higher NDSI values, corresponding to its radiative properties.Table 3 lists the specific thresholds.The normalized difference vegetation index (NDVI) is defined with the radiance measured by two bands in the red and near infrared as where r 1 and r 2 are the reflectances at channels 1 (0.58-0.68 mum) and 2 (0.84-0.89 µm). Figure 3b illustrates the distribution of NDVI.Because the reflectance of the visible channel is greater than that of the near-infrared channel, the NDVI values for water are negative.For land or desert, NDVI values are greater than or around zero.This difference can be used to mask water regions.Table 3 provides the thresholds for four seasons.For April, because there was no overlap between water and the other types in its category, the pixels covered with water could be picked up only by the value of the low limit.

Linear threshold definition and tests
To pick up the residual cloudy pixels over the bright desert, channel 7 (0.43-0.48 µm) and channel 4 (10.3-11.3µm) are used together in the algorithm.For the wavelength at channel 7, there is a strong contrast between the more highly reflective clouds and the less reflective cloud-free desert.Combined with channel 4, where the cloud has a lower brightness temperature than clear sky, this contrast separates cloudy pixels from bright desert surfaces.
Figure 4 shows the scatter diagrams for these two channels.The cloudy pixels can be clearly discriminated from the clear-sky surfaces, especially from the bright desert.According to the distribution in Fig. 4, a linear threshold is introduced here to discriminate the residual cloudy pixels from the clear category.The linear form is determined by two factors: the slope and the intercept.As mentioned in Sect.3.1, when the loss function f achieves a minimum, the line defined by the two factors is chosen to be the final linear threshold.Figure 4 and Table 4 provide the linear thresholds and the loss function values for different seasons.

Unbiased confidence flag
CLAUDIA classifies individual tests into two groups based on the trend observed in each test, which is distinct from other prevailing approaches of classification based on the wavelength domain or the primary target of each test (Ishida and Nakajima, 2009).We applied CLAUDIA to this study.
To realize unbiased cloud detection, two groups need to be formed: tests that tend to be clear conservative (Group 1), and tests that exhibit trends of being cloud conservative (Group 2).The criteria for classifying individual tests are based on the method proposed by Ishida and Nakajima (2009).According to the criteria, the single threshold tests of VIRR can be classified into two groups: Group 1 (clear conservative) includes the channels 1 and 2 reflectance tests; and Group 2 (cloud conservative) contains only the reflectance test for channel 10.The estimation of the clear confidence level for the individual threshold test has been described above (see Sect. 3.1).When the observed values are higher than the high limit (or lower than the low limit), probably due to clear sky, the confidence level is assigned to be 1, whereas, if the observed values are lower than the low limit (or higher than the high limit), with cloudy properties, the confidence level is recorded at 0. The confidence level is set at 0.5 when the observation equals the value of T .Observations between the low and high limits are divided into two parts: higher than T or lower; and the confidence level between 0 and 1 is determined by two linear interpolations.
The final confidence flags are determined by the equations proposed in CLAUDIA, which are quite different from MOD35.According to the classification criteria of CLAU-DIA, there are two groups for the proposed algorithm: group 1 includes data from channel 1 and 2 of VIRR that exhibit the trend of clear conservative; group 2 includes the observations from channel 10 of VIRR that exhibit the trend of cloud conservative.The most significant characteristic of CLAUDIA is that the cloud conservative equation is used to combine the confidence level of individual threshold tests from the clear conservative group (group 1).The cloud conservative group (group 2) employs the clear conservative equation conversely.The contradictory combination of the group characteristics and equations could probably cancel out the possible bias.Thus, the final confidence flags of the proposed algorithm for VIRR can be calculated as follows where q 1 , q 2 , and q 10 represent the values of individual confidence level at channels 1, 2, and 10, respectively, Q 1 and Q 2 are the values of confidence level for groups 1 and 2, and Q final is the final confidence flag.When finishing these steps, the pixels can be divided into the two categories defined above: cloud or clear types (see Sect. 3.1.1).

Special surface mask
Three specific tests are set for cloud screening over special surfaces: NDSI, NDVI, and combined two-channel tests.Based on the values of the final confidence level, the pixels are classified into two categories: Cloud types (Q final between 0 and 0.5) and Clear types (Q final between 0.5 and 1).The pixels included in the Cloud category are selected for the NDSI test, and the remaining pixels are passed onto the other two tests.These tests, which are excluded from the calculation of the final confidence level, are designed on the basis of the characteristics of the special surfaces.Based on the classifications of the unbiased algorithm, it could be more accurate to apply these tests independently to separate clearsky special surfaces into distinct categories.In addition, it provides flags for the special surfaces instead of the overall clear-sky classification, which could be convenient for users of the cloud detection results, to select proper regions for their research purposes and targets.Figure 5 shows the flow chart of the improved cloud detection algorithm.

Cloud mask results and comparisons
The validation of the cloud detection algorithm is difficult because of the lack of observations, which could only be obtained from ground truth measurements of lidar or weather stations.In this section, we focus on the validation of the proposed algorithm over complex terrain by visual image inspection and quantitative analysis.Visual image inspection can be applied to identify obvious problems in cloud masking www.atmos-meas-tech.net/6/549/2013/with the knowledge of cloud and surface spectral properties, but provides poor quantitative evaluation.The quantitative analysis in this study can be obtained by the comparison with the collocated MODIS cloud mask product and the groundbased observations from MICAPS.

Image analysis
The visual image analysis for the performance of the proposed cloud mask algorithm is attained through the comparison with the true-color image of VIRR and its official cloud mask product.The VIRR official cloud mask product contains four confidence levels: high confidence of clouds (cloudy), low confidence of clouds (uncertain), low confidence of clear sky (probably clear), and high confidence of clear sky (clear).The improved cloud mask result provides the specific value of confidence level for each pixel, except pixels covered by water, snow, and residual clouds over deserts.For comparison, the cloud mask image is also divided into four levels by the value of the final confidence level: above 0.75 (clear), between 0.5 and 0.75 (probably clear), between 0.25 and 0.5 (uncertain), and less than 0.25 (cloudy).
Figure 6 shows the cloud mask result over the desert region located from 37.6 to 40 • N and from 77.6 to 80.8 • E. Data were obtained on 6 March 2011, at 05:25:00 UTC. Figure 6a is the gray-scale image of VIRR channel 4 (10.3-11.3µm). Figure 6b is the true-color VIRR image composed of channel 1 (red), channel 9 (green), and channel 7 (blue).Figure 6c is the cloud mask result obtained from the algorithm proposed in this study.Figure 6d is the image of VIRR official cloud mask product.Compared with the VIRR official cloud mask product, the cloud detection result from the proposed algorithm is in good agreement with the infrared image and the true-color image.The proposed cloud mask scheme provides more accurate areas covered by clouds, especially for the cloud located at the left bottom of the image.By applying the two-channel linear threshold test, the residual clouds (labeled as light blue) have been detected, and the cloud detection result has been greatly improved over the bright desert.Apparently, the official cloud mask result overestimates the amount of clouds, and cannot identify the clear-sky pixels between the clouds.
Figure 7 shows a scene over a snow-covered region from 34.6 to 37 • N and from 74.5 to 77.7 • E, on 22 January 2011 at 05:35:00 UTC. Figure 7a is the VIRR gray-scale image of channel 6 (1.55-1.64µm).The settings of the other three images are the same as those in Fig. 6.The difference between the reflectance of snow cover and clouds achieves a maximum at wavelengths between about 1.55 and 1.75 µm, and snow shows a lower reflectance (Rees, 2006).Based on this fact, the dark-colored areas in Fig. 7a  be identified as snow-covered regions.Combined with the true-color image, it is apparent that the region is largely covered by snow.Compared with the VIRR official cloud mask product, the proposed cloud detection algorithm has greatly improved the result over the snow-covered region.The snowcovered areas (shown in yellow), which should be masked as clear-sky pixels, have been modified by the proposed cloud mask scheme.The official product provides incorrect distributions of clouds and underestimates the areas of clear-sky surface.This could probably introduce errors into the subsequent researches.

Comparison with cloud mask products from MODIS
The MODIS cloud detection algorithm is probably the most comprehensive cloud detection scheme, which employs information from 19 out of 36 channels.To provide quantitative validation, the MODIS cloud mask product (MOD35), as a high-quality cloud mask product, is taken as "truth" for the evaluation of the proposed cloud mask algorithm.In order to collocate MODIS with VIRR data, the time difference is limited within 5 min.For the different spatial resolutions between these two observations, the distances between the pixels from VIRR and MODIS are calculated as below: where S represents the distance between two pixels, lat 1 , lat 2 , lon 1 , and lon 2 are the latitudes and longitudes of two pixels, respectively.The minimum distance is then applied to collocate MODIS with VIRR.However, the distance should not exceed 1 km.For quantitative validation, three indices are applied for the comparison with MOD35: the probability of detection (POD), the false-alarm ratio (FAR), and the hit rate (HR) (Cheng and Brown, 1995;Behrangi et al., 2010;Karlsson and Dybbroe, 2010).The definitions (referring to notations in Table 5) in this study are as below: where a represents the number of pixels identified as cloudy by both VIRR and MODIS.d represents the number of pixels identified as cloud-free by both VIRR and MODIS.b and c The POD and FAR provide the accuracy of the cloud detection algorithm in determining either cloudy or clear-sky conditions.The high POD and low FAR values represent good-quality cloud detection results.The HR value is the measurement of the overall efficiency of the cloud masking algorithm.All of the defined indices are calculated based on the total collocated data set for the validation of the proposed algorithm.For MOD35 and the official products of VIRR, the cloudy pixels include high confidence of clouds (cloudy) and low confidence of clouds (uncertain); the clear-sky ones contain low confidence of clear sky (probably clear) and high confidence of clear sky (clear).For the proposed algorithm, the pixels with the confidence level between 0 and 0.5 and the residual cloudy pixels over deserts are classified as clouds for quantitative analysis, while the remaining pixels belong to the cloud-free category.Table 6 summarizes the accumulated results for POD, FAR, and HR during four seasons of 2011.The desert (37.6-40 • N, 77.6-80.8• E) and snowcovered (34.6-37 • N, 74.5-77.7 • E) regions are selected for the validation.
For the scenes over snow-covered regions, the values of FAR for cloudy pixels have been greatly improved by the proposed cloud mask algorithm.However, the values of POD for cloudy pixels provided by the proposed scheme are lower than those of the official product.When focusing on the cloud-free pixels, the proposed algorithm has significantly increased the values of POD, but the scores of FAR are not as low as expected.The analysis for the POD and FAR over desert scenes shows similar trends.The HR scores for the proposed scheme are practically identical or even better than those for the official product, except for the summer months over deserts and the winter months over snow.Figure 8 shows two cases during the seasons with poor HR scores.Figure 8a-d illustrates the snow-covered scene on 12 December 2011 (VIRR at 05:25:00 UTC, MODIS at 05:20:00 UTC). Figure 8a is the VIRR gray-scale image of channel 6 (1.55-1.64µm); Fig. 8b is the MODIS cloud detection product; Fig. 8c and d are the results from the proposed algorithm and the official VIRR product, respectively.Considering the difference between the reflectance of snow cover and clouds at wavelengths between about 1.55 and 1.75 µm, the dark-colored areas in Fig. 8a could probably be identified as snow-covered regions.Apparently, the snow-covered regions have been identified as clouds by the official product of VIRR.For the MODIS cloud detection product, the snow-covered regions in the left bottom of the image have been correctly classified as clear-sky pixels.However, there are still some snow-covered pixels in the right bottom of the image that are identified as cloudy ones.For the proposed algorithm, the low POD for clouds and the high FAR for clear pixels are probably caused by the underestimation of clear-sky areas over snow-covered surfaces for MODIS.Figure 8e-h provides a scene over deserts on 24 August 2011 (VIRR at 05:00:00 UTC, MODIS at 05:05:00 UTC). Figure 8e is the true-color VIRR image composed of channel 1 (red), channel 9 (green), and channel 7 (blue); Fig. 8f-h represent the cloud detection results from MODIS, the proposed algorithm, and the official VIRR product.For the region in the upper left part of the image, both of the MODIS and VIRR official cloud detection products provide practically identical estimations that uncertain clouds existed in this area.The poor identifications of the cirrus clouds by the proposed algorithm result in low POD scores of clouds and high FAR scores for clear pixels over deserts.In addition, the false identifications of the special terrain, in the right bottom of the image by MODIS, could be another reason for the poor detection of the proposed algorithm.

Comparison with ground-based observations from MICAPS
Ground-based observations from MICAPS can also be compared with the cloud detection results.at these weather stations, which is presented as the ratio of clouds in the naked-eye observed sky, can be used for the validation.According to the rules of ground measurement, if the average height of the clouds is 4 km and the angle of view is 10 • , the area for cloud observation covers approximately 1617 km 2 (Malberg, 1973).Thus, for the cloud detection image based on satellite observations, the percentage of clouds is calculated in the region centered at the weather station with a radius of 22.7 km.Considering the passing time of FY-3A, the ground-based observations at 14:00:00 LT are used for the validation.Table 7 lists the comparisons.
Compared with the ground-based observations, the proposed cloud mask results yield mostly good correlations except for some cases where the estimated amount of clouds is generally smaller than the observations at weather stations.For ground-based observations, the curvature of the Earth may result in overestimation of low clouds with a small horizontal angle because openings between clouds can be overlooked.For satellite observations, the presence of thin cirrus clouds may result in underestimation due to the poor identification of them in space (Malberg, 1973).For the overall 96 matchups in 2011, the proposed cloud detection algorithm and ground-based observations agree on 92.0 % of the time with the existence of clear (23/25), and 78.9 % of the time with the presence of clouds (56/71).Figure 9 shows the two cases with poor correlations centered at the Bachu and Hetian sites with a 4 • × 4 • array.For the case on 24 November 2011, the infrared image (Fig. 9a) reveals that the Bachu site was covered by cirrus clouds.By adding the linear threshold test, parts of the residual cirrus clouds over deserts have been screened out, but the proposed cloud mask scheme still underestimates the amount of cirrus clouds.For the case on 16 February 2011, the high, thin cirrus clouds located to the southeast of the site are also not identified by the proposed algorithm.For the northwest regions of China, the atmosphere is well known for its dryness especially over desert regions.The lack in water vapor in this area may cause difficulties for the identifications of the thin cirrus clouds by 1.38 µm observations (Ben-Dor, 1994).In addition, the cases that have shown poor correlations with ground-based observations are mostly from the transitional months between seasons.The thresholds during transitional seasons without modifications may also introduce errors into the cloud detection results.

Conclusions
An unbiased, daytime cloud detection algorithm was proposed for clear-sky and cloudy pixels discriminations over the northwest region of China for VIRR on board the Chinese FengYun-3A polar-orbiting meteorological satellite.This cloud detection algorithm provided the degree of a pixel's cloud contamination using the clear confidence level from 0 (cloudy) to 1 (clear), and also generated flags for the special surfaces such as snow or water in the research region.Six main channels (0.455, 0.63, 0.865, 1.595, 1.36, and 10.8 µm) of VIRR were designed to improve the cloud detection results during four seasons over the complex terrain of the focused region.
The algorithm was based on the unbiased scheme employed in CLAUDIA, which provided a neutral concept to combine the cloud detection result from the individual threshold test.According to the set of VIRR channels, new groups were designed for our algorithm to acquire the unbiased final confidence level of pixels.Additional tests for the identifications of the special surfaces were also applied in our algorithm including the NDSI and NDVI tests for snow and water flags, and a new two-channel linear threshold test for the extraction of the residual cloudy pixels over deserts.In order to provide accurate cloud detection results, seasonal thresholds were used in our algorithm, which were determined based on the statistic data set from four months (January, April, July, and October) in 2010.
Through image inspections, we found that the cloud detection results over snow and deserts adopting the proposed scheme exhibited better correlations with true-color images than the VIRR official cloud mask results did.The performance of the proposed algorithm was also evaluated in detail, during four seasons in 2011, based on the overall 47 scenes collocated with the cloud mask products from MODIS and the 96 individual matchups between VIRR and the groundbased observations from two weather stations in the research region.Comparisons with corresponding cloud detection results from MODIS showed significantly higher POD scores of clear pixels for the proposed algorithm than those for the official VIRR scheme.The HR scores were mostly higher for the proposed algorithm, except for several months with somewhat lower HR values due to the poor identifications of cirrus clouds.The quantitative validation with the groundbased observations suggested that the results from the proposed algorithm and the observations from two weather stations agreed on 92.0 % of the time with the existence of clear (overall 25 clear days), and 78.9 % of the time with the presence of clouds (overall 71 cloudy days).The poor correlations were mostly during the transitional seasons and under conditions with thin cirrus clouds.The poor identification of the cirrus clouds over deserts was probably due to the dryness of the atmosphere in the research region.The identification of the high cirrus clouds under dry atmospheric conditions is a difficult problem in cloud detection, which requires further investigations.The determination and validation of the thresholds during the transitional seasons, which were not considered in this paper, are also potential areas in which to improve the cloud detection results.This topic has to be investigated in our future studies as well.

Fig. 1 .
Fig. 1.Concept of the confidence levels with three threshold values.

Fig. 2 .
Fig. 2. (a) Histograms and (b) ranges of the reflectance distributions of the different surface types and clouds at channel 1 (0.58-0.68 µm) of VIRR during April in 2010; for (b) the locations of the median values shown as slash marks, the low limit and high limit for clear skies vs. clouds and snow shown as black lines, T shown as red line.

Fig. 5 .
Fig. 5.The flow chart of the improved cloud detection algorithm.

Fig. 9 .
Fig. 9. Two cases with poor correlations for comparisons with ground-based observations centered at the Bachu and Hetian sites.The red rectangles represent the weather stations; the gray-scale images of VIRR channel 4 (10.3-11.3µm) (a) on 24 November 2011 at 06:05:00 UTC for the Bachu site and (b) on 16 February 2011 at 06:05:00 UTC for the Hetian site; (c) and (d) the corresponding cloud mask results obtained from the algorithm proposed in this study.

Table 1 .
FY-3A/VIRR Channels used for the cloud detection algorithm in this study.

Table 2 .
Threshold values of single reflectance tests during four seasons.

Table 3 .
Threshold values of NDVI and NDSI tests during four seasons.

Table 4 .
The definition values of the linear threshold test during four seasons.

Table 5 .
The notations used for the definitions of POD, FAR, and HR scores.

Table 6 .
Accumulated results of the proposed algorithm for POD, FAR, and HR validation scores during four seasons.Corresponding values are given for the VIRR official products in brackets.
Two weather stations located in the research region are used for comparisons: the Hetian site (79.93 • E, 37.13 • N), and the Bachu site (78.57• E, 39.8 • N).The total amount of clouds observed

Table 7 .
Comparisons between ground-based observations and the VIRR cloud mask results of the proposed algorithm.