Cloud masking of Earth observation measurements is an important and often crucial part of various remote sensing retrievals. This includes, but is not limited to, the retrieval of cloud and aerosol microphysical parameters, the estimation of cloud cover, ocean color retrievals, and in general, algorithms which include atmospheric correction schemes. Cloud masking algorithms differ widely in their complexity, computational requirements, and assumptions about what a cloud is and which physical process is exploited for their detection. Implementation of particular algorithms are often application specific, which makes the cloud masks as well application specific and generally complicates the inter-comparison of results from different cloud masks.

This paper emphasizes the application of Bayesian methods for the cloud masking of the complete 9.5 year time series of the Medium Resolution Imaging Spectrometer (MERIS) and the Advanced Along-Track Scanning Radiometer (AATSR) on-board the Environmental Satellite (ENVISAT) and is part of the European Space Agency (ESA) Cloud CCI (Climate Change Initiative) project . Thus, the requirements for the cloud masking scheme, which is described in Sects.  to , are robustness, accuracy, and computational efficiency. Several possible applications of the Bayesian method are discussed in Sect.  . Results for MERIS and AATSR are discussed separately but with a focus on their combination within the Synergy product, in which daytime AATSR measurements are mapped on the MERIS swath and their mutual overlap is used. The Synergy data set in combination with one of the presented cloud detection schemes will be used for the retrieval of cloud microphysical parameters using the FAME-C algorithm which was described by . The development within Cloud CCI is ongoing and finalization of the actual algorithm is planed for the near future.

Major challenges of cloud detection are validation, the correct classification of scenes with clouds for mountainous regions and over snow- and ice-covered areas, and the distinction between clouds and optically thick aerosol plumes such as dust storms. These points are discussed in more detail in Sect. .

Common approaches to cloud masking are hierarchies of thresholds e.g.,, complex statistical models e.g.,, or other Bayesian approaches e.g.,. A classification scheme for Bayesian cloud masks which helps to clearly distinguish the various approaches to Bayesian cloud masking is introduced in Sect.  and, in addition, a short overview of the relevant literature using such schemes is given.

The results presented here are computational highly efficient and are very well suited for the processing of large numbers of data, which makes these results very well suited for future application to the Ocean Land Colour Instrument (OLCI) and the Sea and Land Surface Temperature Radiometer (SLSTR) on-board the Sentinel-3 satellite and its operational follow-ups.

Bayes' theorem can be used to reverse joint probabilities. It is appealing to apply it to cloud masking since its theory is widely adopted, its implementation on a computer system is straightforward, and its results are probabilities which can be directly interpreted. The theorem allows the computation of the probability P(C,F) that a particular measurement with feature F is affected by a cloud when the occurrence probabilities P(F,C) and P(F,C¯) of the feature under cloudy and non-cloudy conditions are known. Here, P(a,b) denotes the occurrence probability of a under the condition of the occurrence of b.

With C being the case that a measurement is affected by clouds and F being a set of features associated with that measurement, P(C,F) can be expressed as P(C,F)=P(C)P(F,C)P(F)=P(C)P(F,C)P(C)P(F,C)+P(C¯)P(F,C¯), where P(C) is the background probability of cloudiness and C¯ is the negation of C, which states that a measurement is not affected by clouds. The occurrence probability of the feature P(F) can be expressed in terms of the joint probabilities P(F,C) and P(F,C¯), because cloudiness and non-cloudiness are the only two considered classes for each measurement.

Evaluating Bayes' theorem involves only a few arithmetic operations so that a specific implementation can be very fast and efficient, which is of importance when large numbers of data are to be processed. Additional computations involve the feature F and the a priori joint probabilities P(F,C) and P(F,C¯), which are discussed in the following sections.

With an appropriate set of thresholds, one can convert the probability P(C,F) into a cloud mask. For instance, any probability strictly higher than 50 % could be interpreted as cloud, but other thresholds or more classes can be used. This is discussed in more detail in Sect. , but this choice clearly depends on the target application and is independent of the Bayesian approach. Other applications, such as the construction of a cost function as described by , are also viable alternatives.

Estimating the value of the background probability P(C) is not discussed in detail in this paper and for all following applications a value of 0.5 is used. This choice basically states that for each measurement an equal probability of it being cloudy or not cloudy is assumed. This assumption is of course valid neither on a global nor local scale, and a rich body of knowledge about the spatial and temporal distribution of cloud occurrence probabilities exists. Such knowledge, typical in the form of external climatologies, could be used to estimate P(C) but would eventually shift derived climatologies towards the external one, which would then effectively lead to circular arguments. This point might be of lower importance for some applications, i.e., operational processing by weather services, but within Cloud CCI climatological data sets will be derived from the full MERIS and AATSR time series and circular arguments are best to be avoided. Since a decision for the actual value of P(C) has to be made, its impact on derived data sets should be investigated and communicated to potential users. In general, the background probability can be a function of external or auxiliary data like position or time of year. In the general case, the estimation of the joint probabilities P(F,C) and P(F,C¯) should be consistent with P(C).

For the special case of P(C)=0.5, Eq. () simplifies to P(C,F)=P(F,C)P(F,C)+P(F,C¯). Setting up a particular Bayesian cloud mask algorithm involves several decisions, such as specifying the measurement feature F and choosing a technique to estimate P(F,C) and P(F,C¯), which allows us to group the various possible approaches to Bayesian cloud masking into distinct subgroups. This natural grouping allows to clearly separate the presented approach from other algorithms and is discussed in Sect. . In addition, a short overview about the relevant literature is given.

Several papers on Bayesian approaches to cloud masking have been published in the past and fundamental differences between the various algorithms are often buried in the technical details of the particular paper. A nomenclature which aims to clearly separate different approaches to Bayesian cloud masking is discussed in the following.

Let the feature F from Eq. () be a set of nF real numbers F=(F1,…,FnF)∈RnF, where the Fi are typically determined from measurements M∈RnM, auxiliary data A∈RnA, and external data E∈RnE. The components Fi are computed from prescribed feature functions fi, which generally depend on all of the above introduced classes of data: Fi=fi(M,A,E). In the case of the MERIS and AATSR Synergy, the set of measurements M includes radiances and brightness temperatures for a single collocated pixel. Auxiliary A data are available with negligible computational cost, such as time stamps, geolocation, and data flags. External data may be a function of the available measurements M and auxiliary data A and their procurement are by definition associated with non-negligible computational cost. This category essentially introduces significant external knowledge about the measurement and common examples are online radiative transfer (RT) simulations, nontrivial interpolation in numerical weather prediction (NWP) data, or the use of climatologies.

Let us call the feature set F independent when it is only a function of the measurements M and auxiliary data A and dependent when external data E are additionally exploited. Both classes can be further subdivided with respect to weak and strong dependence to describe F even more precisely. A weakly dependent feature set could, for example, depend on interpolation in NWP data, which is of negligible computational cost, while a strongly dependent feature set could depend on online RT with non-negligible numerical cost. Strongly independent feature sets would then depend only on measurements M, while weakly independent feature sets could in addition depend on auxiliary data A.

This paper focuses on Bayesian cloud masks based on strongly independent features. Only MERIS and AATSR measurements and trivial functions operating on them are used to construct the feature set. This class of features allows to implement a numerically highly efficient algorithm with simple opportunities to parallelization and vectorization. With no dependency on external data, the algorithm can be used in non-operational environments where the acquisition of NWP data can require significant effort. In general, there is no obvious reason why the techniques which are discussed in the following sections are limited to the independent case.

The second major branch in Bayesian cloud masking schemes involves the computation of the joint probabilities P(F,C) and P(F,C¯). The classical approach aims at the direct computation of these two joint probabilities, while the naive approach treats the components Fi of the feature set F as statistically independent and decouples the joint probabilities on F into a product of joint probabilities of the Fi: P(F,C)=∏iP(Fi,C). One can either construct the feature set very carefully, such that this strong assumption holds (e.g., follow and their discussion on cloud texture and cloud top temperature), or simply accept its violation and the possible effects on the cloud masking scheme. Formally proving the statement of Eq. () seems to be only possible for a rather limited class of features.

Computing the joint probabilities in the classical approach can be greatly simplified by assuming an analytic form and estimating its parameters. Depending on the assumed form, for instance multivariate Gaussian e.g., see, the resulting cloud mask could be called classical Gaussian. As for the naive approach, it will be difficult to formally prove the validity of such assumptions.

The classical and naive approaches can be mixed when one or more subsets of the Fi are treated as statistically independent, such that the decoupling of P(F,C) and P(F,C¯) becomes partial. For this class of Bayesian cloud masks we propose using the terms “mostly naive” when the majority of features are decoupled and “mostly classical” when the majority of features are not decoupled.

This paper is mainly concerned with the discussion of the classical and naive approach with an emphasis on the classical one. In conclusion, this paper is mostly concerned with the application of classical Bayesian cloud masks based on strongly independent features. As it will be shown later in the paper, the classical approach gives better results for the cloud masking in our scenario and the strongly independent feature set was chosen to allow the implementation of a very fast algorithm.

Cloud detection methods based on Bayesian probabilities have been used for cloud masking in the past, and a short overview is given now but without the attempt to fully outline them. used Bayesian probability with strongly dependent features to derive a cost function for a 1D-Var retrieval of cloudiness. The classification is based on the exploitation of microwave and infrared channels and, in addition, external data from NWP simulations. used Advanced Very High Resolution Radiometer (AVHRR) channels, derived channels such as reflectance ratios and brightness temperature differences, and textural measures to construct strongly independent features. It was found that textural measures are most important for nighttime measurements. The joint probabilities were separated by assuming a multivariate Gaussian form and were expressed in terms of mean values and associated covariances. used nighttime thermal infrared measurements at 3.7, 11, and 12 µm to construct a mostly classical Bayesian cloud mask. Textural features were assumed to be independent from measurements in thermal channels and were separated when computing the joint probabilities. P(C) was estimated from NWP data and the algorithm was discussed with an emphasis on operational NWP centers such that these feature are likely only weakly dependent. discussed a mostly classical Bayesian algorithm with strongly dependent features for the 3.9, 11, and 12 µm channel of the SEVIRI instrument. External knowledge is introduced by NWP data, and a fast radiative transfer model and textural features were separated from spectral features assuming independence. discussed a naive Bayesian cloud mask with strongly dependent features for the AVHRR instrument. A surface type classification using external MODIS data was used to maximize the detection rate. CALIPSO lidar measurements were used as truth data to compute histograms from which the occurrence probabilities for each feature were estimated.

Channels of the MERIS and AATSR instruments cover the spectral range from 412 nm to 12 µm and are referenced in this paper by their central wavelength, while for MERIS the unit of nm and for AATSR the unit µm is used. Figures  and  show examples of possible features for two particularly interesting scenes over Greenland and in the vicinity of the Korean peninsula. Each figure shows an RGB image, various single channels, and a selection of trivial functions which combine two channels. Both figures include a panel with results of the non-Bayesian Synergy cloud mask, which is briefly discussed in Sect. . Figure  shows a scene over Greenland with its center located at 59∘31′12′′ W and 79∘0′0′′ N with high and low clouds over a large ice- or snow-covered region. Figure  shows a scene in the vicinity of the Korean peninsula with its center located at 125∘52′12′′ E and 37∘45′36′′ N. This scene shows a pronounced dust storm mixed with a deck of clouds.

Strongly independent features are constructed using a single channel or any combination of channels in a trivial function. Such combinations have been called derived channels in the literature e.g.,. Considered here are all basic arithmetic operations (+,-,×,/) and, in addition, the index function dx(a,b)=(a-b)/(a+b), which can be used to create indices such as the normalized difference vegetation index see, the normalized difference snow index see, or other general channel indices. Even when well-known and generally accepted combinations of channels and indices are used, it is unclear whether a specific combination is the best possible candidate for the particular data set and one has to rely on the experience of the involved experts.

In contrast to approaches based on expert knowledge, an objective measure for any given set of feature functions is exploited to numerically search for the best possible set of feature functions. Maximizing the Hanssen–Kuipers skill score see with respect to a given validation data set is an appropriate metric for this problem. It is also sometimes referred to as a Hanssen–Kuipers discriminant and is essentially the difference of the hit rate and the false alarm rate of the cloud mask with respect to a validation data source. It covers the range of -1 to +1, with +1 being a perfect representation of the validation source. From now on, only the term “skill score” is used.

Validation of cloud masks for MERIS and AATSR on-board ENVISAT is a difficult task since no generally accepted and available set of truth data exists. A generally used approach is to generate truth data by means of manual classification of images by human experts or the use of data from ground-based stations. Converting a ground truth to a pixel-by-pixel truth can be complicated, and possibly insufficient spatial coverage can limit the applicability of that approach. Consequently, most approaches for generating truth data for MERIS and AATSR are based on the manual classification of sample data by human experts e.g.,. Such data sets can be called artificial truth because, although they are used as if they were truth, it is arguable whether such data sets are in fact truth.

To demonstrate the feasibility of the Bayesian approach, results from the Synergy cloud mask (see and Sect.  for a brief description) were chosen as a source of artificial truth data; it is therefore assessed whether Bayesian cloud masks can reproduce this Synergy cloud mask. The major advantage of this approach is that large numbers of artificial truth data can be created without significant effort. Clearly, all shortcomings of this seeding algorithm will be present in this data set and will limit the success of the application of the Bayesian technique.

Optimizing the choice for a particular set of feature functions is not straightforward, since this problem is noncontinuous with a varying number of free parameters. First, the number of feature functions has to be set. Then, for each feature, a feature function from the pool of considered functions has to be selected. The identity function, all four basic arithmetic operations, and the index function are considered as feature functions. As a last step, the input channels for each feature function must be set. Depending on the chosen functions and channels, a maximum of 2×nF channels can be included in the computation of a feature set with nF elements.

Then, for a particular feature set, the prerequisites for computing the joint probabilities must be carried out, which is described in detail in Sect. . Once this step is completed, the Hanssen–Kuipers skill score for the selected set of validation data can be computed.

The only numeric optimization procedure that we are aware of, which is generally applicable to this situation, is a random search in the huge search space spanned by this outlined procedure. This is quite a different approach to that of a human expert, who would likely start an educated search but might not attempt to cover the whole search space. The number of possible combinations depends on the number of chosen features and the number of available channels (22 in the case of the MERIS and AATSR Synergy) and can be estimated using the binomial coefficient. In the simplest case, where merely the identity function is used, no channel is used more than once, and four features are to be selected, the search space spans 224=7315 elements. When only functions of two channels are to be selected and re-selected and channels can be used multiple times, then the search space consists of 5×(222-22)4=23104≈1.2×1012 entries. The enormous size of the search space makes it difficult to completely cover it by a search, but the random search can be allowed to run appropriately long such that a result of sufficient quality is obtained. One can expect that a large number of different sets of feature functions will essentially exhibit very similar classification skills. The considered feature functions are not symmetric under a change of the parameter order, but the overall classification result might be approximately symmetric. This alone would decrease the search space by a factor of approximately 16. In addition, the classification results might be only weakly dependent with respect to the feature function itself; i.e., the index function dx(a,b) might be as effective as a ratio a/b, which would decrease the effective size of the search space.

The proposed random search might not be able to cover the complete search space, but with a sufficiently long runtime one will be able to find solutions with a sufficiently high skill score. In addition, unusual combinations of channels might be found which would not be considered in an educated search by a human expert. The features shown in Figs.  and  are frequently found in searches when results from the non-Bayesian Synergy cloud mask are used as artificial truth.

The physical meaning of a certain feature set and why it might be better or worse than a different one is not discussed here and is also not within the scope of this paper. This knowledge is very useful for educated searches but is not necessarily needed in this setup. However, for the experienced expert it might be only slightly surprising which channels are found to be successful by the optimization scheme. There is also no apparent reason why human experts should not compete with the optimization scheme in order to find an optimum set of features. This is especially important for applications where only a small fraction of the search space can be tested using the optimization approach.

Implementing such a search strategy is straightforward. A generator of random feature functions must be implemented and each of these instances can be tested for its skill score with respect to the artificial truth. This procedure is easily parallelizable, and one could store only the results with a higher skill score rather than some predefined value. At any given time during an ongoing search, one can sort these results and evaluate the top results.

The background joint probabilities P(F,C) and P(F,C¯) could be computed in various ways, but here only the frequentist approach based on sample data is considered. A sufficiently large number of already-classified measurements are converted into their corresponding set of features, and probability density histograms are produced, from which the probabilities are estimated. In the naive Bayesian approach, as many one-dimensional histograms as there are features are needed, while in the classical Bayesian approach a single nF-dimensional histogram is used. When these histograms are stored in a computer system, the handling of any reasonable number of one-dimensional histograms poses no specific problem, while an array of dimension nF grows rapidly in memory with increasing number of bins nB. For the sake of simplicity, the same number of bins is assumed for each particular dimension. With four bits per float and twenty bins per feature, one would need 0.6 Gb to store a single histogram for four features but already about 4883 Gb for seven features. This limits the practical number of features for the classical Bayesian approach to about four to six at the time of writing this paper.

However, the main argument of and against the use of the classical approach is that one has generally not enough truth data available to robustly derive the histograms in a completely frequentist way. This can be a valid point for real truth data, which are limited in principle, but not so much for artificial truth data. Here, the number of available data is merely a function of the available human labor for manual classification or computational resources when an existing cloud masking scheme is used to produce artificial truth data.

Both left panels of Fig.  and show results of two-dimensional histograms for MERIS and AATSR Synergy data. For both cases, almost 1 million spectra were used to compute both histograms. Shown is the difference between the histograms for C and C¯. Both choices of features recreate the Synergy cloud mask reasonably well with a skill score of about 0.76. The cloud masking setup is discussed in detail in Sect. . The main point here is that with enough data points these histograms can be computed. The two-dimensional case was chosen since this is simple to visualize.

Both right panels of Figs.  and  show remarkably similar histograms with just barely smaller skill score values of about 0.75, but only 1000 randomly selected measurements from the original data set were used to produce these histograms. A simple Gaussian smoothing filter was applied to both histograms and each Gaussian smoothing factor was chosen such that the skill score as a function of the Gaussian smoothing was maximized. This is the first main result of this paper. This numerical experiment shows that, at least for some sets of feature functions, the nF-dimensional histograms can be approximated by using very few data points and an appropriate Gaussian smoothing factor. The best smoothing factor for both cases is slightly different and is obtained from optimization. More detailed results are shown in Sect. . In addition to the previously discussed parameters, e.g., the construction of features, classical Bayesian cloud masks are defined by the number of bins used in the histograms and the chosen Gaussian smoothing parameter, which is discussed in Sect. .

It should be noted that this is an extreme case and we do not propose to use so few data points to construct cloud masks for real-world applications. These two examples merely show how well this approach operates and that a surprisingly small number of data might be sufficient to explore the application of classical Bayesian cloud masks.

The Gaussian smoothing approach works reasonably well and is so far only justified by its actual success for a particular problem, where in fact sufficient numbers of artificial truth data are available. Its general application to situations with limited numbers of such data is therefore not very well justified. However, numerical experiments with the available data have shown that this approach yields remarkably good results. Other functional kernels have not been tested, but the Gaussian approach seems sufficient since the convoluted histograms yield nearly the same skill score as the original histograms. Success of this approach is likely based on the fact that the smoothing procedure distributes data to neighbor bins but does not strongly change the defining spectral features of the measurements. That is, it implicitly creates data which could represent different viewing geometries or situations with slightly varying optical parameters. Hence, this approach is not justified by first principles but rather with working examples which strengthen our expectations that this approach will work reasonably well for any other set of features.

The Synergy cloud mask is discussed in detail by and is implemented as an external processor for the BEAM toolbox . It is based on radiative transfer simulations covering all spectral bands of MERIS and AATSR and statistical analysis of classified data by human experts. Within the frame of the ESA Cloud CCI project phase 1, the years 2007–2009 of the MERIS and AATSR time series were processed. The derived cloud cover (or cloud number) was assessed in several validation exercises, e.g., compared to cloud numbers from the GEWEX CA database , which consists of a number of data sets with gridded and monthly mean cloud number derived from a variety of satellite instruments. Results of global mean cloud number are in line with GEWEX cloud numbers . The cloud mask product from the years 2007 to 2009 can be used as a large source of artificial truth data for the synergy data set.

When the computation of P(F,C) and P(F,C¯) is based on the frequentist approach and artificial truth data, then three major applications of the technique become feasible. Results from existing algorithms can be reproduced using the Bayesian technique, which could potentially speed up and simplify the cloud masking of large numbers of data. With the Synergy cloud mask from Sect.  as an example, this procedure is discussed in the following Sect. . When the existing algorithm is reproduced reasonably well, one can use this technique to further enhance the algorithm, which is discussed in Sect. . A simple example in which data classified by a human expert are used to set up a Bayesian cloud mask is discussed in Sect. .

The application of the classical and naive Bayesian cloud masking technique to MERIS, AATSR, and their Synergy was discussed in detail. Bayesian cloud masks based on independent features are numerically highly efficient and are very well suited for the fast processing of large numbers of data. This technique will be applied to a reprocessing of the 9.5 year time series of MERIS and AATSR measurements within ESA's Cloud CCI project.

Details of the actual implementation of the Bayesian cloud mask for Cloud CCI are not part of this paper. The algorithm is implemented in Python and is based on the multiprocessing, SciPy, and NumPy libraries . Effective parallelization is achieved trough separation of CPU bound and input / output (I/O) tasks. Processing time per orbit is largely dominated by I/O and the actual time spend in the Bayesian scheme is 1 order of magnitude smaller than the total run time. Currently, the scheme supports the classical and naive approach for independent Bayesian cloud masks. The final set of features for processing the complete Cloud CCI period of 9.5 years will be determined in the near future before starting the generation of level 3 data.

Sufficient numbers of artificial truth data and the frequentist approach can be used to estimate multidimensional histograms for the estimation of background joint probabilities. Gaussian smoothing of appropriate width can be used to drastically reduce the actual numbers of truth data needed to compute histograms for the classical Bayesian approach. This post-processing step greatly simplifies our ability to further explore the classical Bayesian approach.

Due to restrictions of modern computer hardware, the practical limit for the classical Bayesian approach is reached with six to seven features. This does not actually restrict its applicability, since trivial feature functions can be used which combine any number of measurements into a single feature.

It was found that classical Bayesian cloud masks with four strongly independent features are the best choice for the cloud masking of MERIS, AATSR, and their Synergy measurements when the Synergy cloud mask is used as a benchmark. The classical approach gave significantly better results then the naive approach. MERIS and the MERIS–AATSR Synergy give very similar results in terms of cloud classification, while AATSR alone shows significantly smaller skill scores. The MERIS Oxygen-A absorption channel was found to be present in the best results when the set of selected feature functions and channels was numerically optimized.

The broad spectral range and the number of available channels within the Synergy data set can be used to set up Bayesian cloud masks with very similar classification skill but based on different combinations of channels. This can be used to design cloud masking schemes which are robust against partially missing data.

It was shown how Bayesian cloud masks can be used to reproduce the results of existing algorithms, improve existing algorithms and how to set up new classification schemes based on manual classification by human experts. Reproducing existing algorithms offers the perspective of increased numerical efficiency and processing robustness. The approach based on manual image classification is straightforward for the human expert. Classified scenes can be stored and revisited if the produced cloud masks show misclassifications in certain areas or weather conditions. When errors are not traceable to errors in the manual classification, additional scenes can be added to the set of artificial truth data to increase the chance of correct classification.

The presented results for MERIS and AATSR can be used to implement an accurate and highly efficient cloud masking scheme for OLCI and SLSTR on-board the upcoming Sentinel 3 satellite. Especially the additional oxygen absorption channels from the OLCI instrument might be used within an improved and numerically efficient cloud classification algorithm.

Although this paper is focused on strongly independent Bayesian cloud masks, there is no apparent reason which prevents the application of the introduced techniques to the case of dependent Bayesian cloud masks. It is straightforward to include external information such as clear sky radiance estimators or NWP fields in the proposed optimization strategy for the construction of features. The application of Gaussian smoothing to derived histogram fields is independent of external information and can be used to reduce the numbers of needed truth data. To actually assess the added value of the external data, one must assure that the quality of the truth data is sufficient. In the case of MERIS and AATSR, one likely needs a reasonable large set of manually classified data.