The accurate identification of the presence of cloud in the ground scenes observed by remote-sensing satellites is an end in itself. The lack of knowledge of cloud at high latitudes increases the error and uncertainty in the evaluation and assessment of the changing impact of aerosol and cloud in a warming climate. A prerequisite for the accurate retrieval of aerosol optical thickness (AOT) is the knowledge of the presence of cloud in a ground scene.
In our study, observations of the upwelling radiance in the visible (VIS),
near infrared (NIR), shortwave infrared (SWIR) and the thermal infrared
(TIR), coupled with solar extraterrestrial irradiance, are used to determine
the reflectance. We have developed a new cloud identification algorithm for
application to the reflectance observations of the Advanced Along-Track Scanning
Radiometer (AATSR) on European Space Agency (ESA)-Envisat and Sea and Land
Surface Temperature Radiometer (SLSTR) on board the ESA Copernicus Sentinel-3A
and -3B. The resultant AATSR–SLSTR cloud identification algorithm (ASCIA)
addresses the requirements for the study AOT at high latitudes and utilizes
time-series measurements. It is assumed that cloud-free surfaces have
unchanged or little changed patterns for a given sampling period, whereas
cloudy or partly cloudy scenes show much higher variability in space and
time. In this method, the Pearson correlation coefficient (PCC) parameter is
used to measure the “stability” of the atmosphere–surface system observed
by satellites. The cloud-free surface is classified by analysing the PCC
values on the block scale
The large trends in warming over the Arctic in recent decades has received much attention from the global and regional climate change research community (Wendisch et al., 2017; Cohen et al., 2014). A number of studies using global observations and climate models confirm this phenomenon, called Arctic amplification, and provide evidence that its impact extends beyond the Arctic (Kim et al., 2017; Cohen et al., 2014). Though the attribution of the origins of this phenomenon is controversially discussed (Serreze and Barry, 2011; Pithan and Mauritsen, 2014), cloud cover is well known to play a role in the Arctic surface–atmosphere radiation balance (Kellogg, 1975; Curry et al., 1996). The accurate identification of Arctic clouds in the ground scenes of remote-sensing measurements made from space is therefore of intrinsic importance. However, cloud identification and screening over the Arctic is a challenging task, since all developed cloud detection methods encounter many obstacles originating from the unique atmosphere and surface conditions in the Arctic (Curry et al., 1996). The Arctic clouds are mostly optically thin and low with no remarkable contrast in the commonly used visible or thermal or microwave measurements to the underlying surface covered with highly reflecting snow and ice. For example, snow and ice are also cold like clouds: the lack of strong thermal contrast is a limitation in the retrieval of clouds in the thermal infrared (Rossow and Garder, 1993; Curry et al., 1996).
In addition to the importance of clouds to Arctic amplification, errors in the identification of cloud within a ground scene are also one of the major sources of error in retrievals of a variety of data products for both satellite and ground-based measurements at high latitude. For example, the interference of cloud contamination in the aerosol optical thickness (AOT) retrieved by passive satellite remote sensing is a well-known issue (Shi et al., 2014; Várnai and Marshak, 2015; Christensen et al., 2017; Arola et al., 2017). This limits the reliability and usefulness of the AOT products in the assessment of the direct or indirect impact of aerosols in the Earth's energy balance, in particular over the Arctic. To avoid the uncertainty included in AOT products due to significant misclassification of heavy aerosol load as thin clouds (which have similar reflectance properties), the development of an adequate cloud identification algorithm is a prerequisite (Martins et al., 2002; Remer et al., 2012; Wind et al., 2016; Mei et al., 2017a, b; Christensen et al., 2017).
One recent approach to detect cloud-free snow and ice over high latitudes used the spectral shape of clear snow, ISTO (Istomina et al., 2010). The latter analyses the spectral behaviour of each ground scene and identifies clear snow or ice scenes from Advanced Along-Track Scanning Radiometer (AATSR) measurements. Thresholds of the reflectance were empirically determined in seven spectral channels from the VIS to TIR. Defining a reliable threshold which can guarantee a successful separation of cloud and cloud-free regions for the wide range of atmospheric conditions and surface types is a challenging task. This is because of the similarity between spectral reflectance of cloud and snow/ice (Lyapustin et al., 2008). In spite of progress made by this approach, adequate discrimination of thin cloud above ice or snow is an inherent limitation of such threshold-based techniques.
The European Space Agency (ESA) standard cloud product from AATSR is another
example of an existing cloud data product over the Arctic. This operational
cloud mask is called the Synthesis of ATSR Data Into Sea-Surface Temperature
(SADIST) and is based on the latitudinal thresholds for various cloud types
(Ghent et al., 2017). SADIST was initially developed for cloud screening over
the ocean (Zavody et al., 2000). Birks (2007) modified this method to apply it over land. Later,
Kolmonen et al. (2013) reported that the cloud flags included in AATSR
product are noticeably restricted and using this cloud product results in
aerosol episodes not being observed. SADIST is known to misclassify ice,
cloud and open ocean in polar regions. Bulgin et al. (2015) developed a
Bayesian approach in ESA's Climate Change Initiative (CCI) project to
overcome this limitation (Hollmann et al., 2013). Sobrino et al. (2016)
reviewed different cloud-clearing methods including the AATSR operational
cloud mask in the framework of Synergistic Use of The Sentinel Missions For
Estimating And Monitoring Land Surface Temperature (SEN4LST) project. They
highlighted the potential uncertainty in different versions of this product,
which result in these errors being propagated in subsequent data products.
For example, the AATSR operational cloud mask falsely detects cloud in
To avoid the uncertainty arising from the similarity of spectral characteristics of snow, ice and clouds, we decided to develop an algorithm based on a different strategy, namely the use of time series measurements. The use of abrupt changes of TOA reflectance in time with the aim of cloud identification has been reported previously (Gómez-Chova et al., 2017; Lyapustin et al., 2008). An early example of this idea was proposed for low to midlatitudes by Rossow and Garder (1993) in the International Satellite Cloud Climatology Project (ISCCP). This method later evolved as a part of the MultiAngle Implementation of Atmospheric Correction (MAIAC) algorithm (Lyapusitn et al., 2008). MAIAC is mainly designed for use with observations over land (low to middle latitudes), where the aim is to simultaneously retrieve aerosol and surface properties. However, it has also been utilized by another study to identify snow grain size over Greenland (Lyapustin et al., 2009). Although further optimization for the Arctic region is required and reported, a better performance in comparison to Moderate Resolution Imaging Spectroradiometer (MODIS) cloud mask is reported by Lyapustin et al. (2009).
The central assumption used in these algorithms for cloud identification is
that clear-sky reflectance is different to that of clouds, which exhibit, in
comparison, high variation as a function of time (Lyapustin et al., 2008;
Gómez-Chova et al., 2017). Knowledge of cloud-free scenes within a given
time period is achieved from knowledge of the variability of the measured
TOA reflectance. Covariance analysis is used to estimate the spatial
coherence. This has a long history in remote-sensing studies using time
series measurements (Leese et al., 1970; Lyapustin et al., 2008). The
covariance computation assumes changes in the textural patterns of the
observed scene, which originate from natural and man-made features such as
topography, lakes or urban areas (Lyapustin et al., 2008). The use of the
covariance analysis, which accounts for geometrical structures, minimizes
issues originating from illumination variation and results in the same
algorithm being applicable over both dark and bright surfaces (Lyapustin et
al., 2008). For these reasons we decided to use the Pearson correlation
coefficient (PCC) as a function of covariance value for cloud detection over
the Arctic. However, Lyapustin et al. (2008) reported that, in spite of
a relatively good performance, the covariance itself is not alone adequate for
cloud identification in the case of homogeneous surfaces or thin clouds.
Therefore, we decided to use a combination of a PCC analysis and the
reflectance of solar radiation at 3.7
Another argument in favour of the use of time series analysis is the availability of multiple images by the AATSR and Sea and Land Surface Temperature Radiometer (SLSTR) sensor over the Arctic. For AATSR the revisit time is 3–4 days over midlatitudes (Kolmonen et al., 2016) but more frequent at higher latitudes, which decreases to 2 days over the Arctic (Soliman et al., 2012; Mei et al., 2013). In addition to multiple imagery over the Arctic, the shorter time interval between satellite overpasses over the same scene provides images with less variability in the observed cloud-free areas which the algorithm looks for. For the two SLSTR, the revisit time is 0.9 days at the equator (Coppo et al., 2010) and this time becomes even shorter at higher latitudes due to orbital convergence.
The AATSR–SLSTR cloud identification algorithm (ASCIA) has been developed as
a part of research activities to meet the scientific objectives of
Collaborative Research Centers, CRC/Transregio 172 “ArctiC Amplification:
Climate Relevant Atmospheric and SurfaCe Processes, and Feedback Mechanisms
(AC)
A full description of this new cloud identification and its application to AATSR data is presented in the following sections of this paper. First, a brief data description is presented in Sect. 2. The theory and methodology used in our new ASCIA are discussed in detail in Sects. 3 and 4. We evaluated the performance of ASCIA by comparison of the cloud identification with (i) the ESA standard cloud product for AATSR level 2 nadir cloud flag, (ii) the data obtained by applying ISTO to AATSR data, (iii) the MODIS cloud mask, (iv) the surface synoptic observations (SYNOP) and (vi) the AErosol RObotic NETwork (AERONET). The results of the comparisons with these five different sources of cloud data are reported in Sect. 5. A discussion and set of conclusions, drawn from the study, are presented in Sect. 6.
The AATSR sensor flown on board polar-orbiting Envisat was primarily designed
for measuring sea-surface temperature (SST) with accuracy higher than
0.3 K. As the successor of ATSR-1 and ATSR-2 on
European Remote Sensing-1, ERS-1 and ERS-2, AATSR delivered data from March
2002 until Envisat failed in 2012
(
The conical imaging geometry of AATSR yields the dual-viewing capability of
this sensor. Each scene is imaged twice. The first measurement of the ground
scenes is in the forward direction at a viewing angle of 55
SLSTR on board Sentinel-3A was launched on 16 February in 2016 as the
successor to AATSR to provide the continuity of long-term SST measurements.
The Sentinel-3B satellite, which contains an identical payload, was also
launched by a Rockot/Breeze-KM launch vehicle from the Plesetsk Cosmodrome in
northern Russia, on 25 April 2018. The design of the SLSTR instrument has
some significant improvements with respect to ATSR (Coppo et al., 2010). For
example, the swath widths of single view and dual view were increased from
500 to 1420 and 750 km respectively. This yields global revisit times of
1.9 days at the equator with two satellites and 0.9 day with one satellite.
There are measurements of two additional channels in the SWIR, at the
wavelengths of 1.37 and 2.25
The SYNOP have been provided by the World Meteorological Organization (WMO) for the purpose of mapping weather information around the world. However, the availability of the data is limited in the Arctic due to the coverage of SYNOP stations in this region. For example, there is almost no observation in the central parts of the Arctic Circle as is shown in Fig. 3. The SYNOP measurements, which are made by an observer or an automated device are available in a standardized layout of numerical code which is called FM-12 by WMO (1995). The SYNOP reports include a variety of meteorological parameters such as temperature, barometric pressure, visibility, etc. as well as cloud amount, which are observed at synoptic hours simultaneously throughout the globe. We used SYNOP cloud fraction, which has a temporal resolution of 1–3 h, to evaluate the performance of our newly developed ASCIA over the Arctic region.
SYNOP network coverage over the Arctic, the dark-blue points indicate the location of SYNOP stations.
The use of SYNOP measurements to validate a cloud identification algorithm,
or for that matter the cloud predicted by a climate model, the fact that the
SYNOP cloud fraction is reported using the okta scale, has to be
appropriately taken into account. Converting discrete okta values, which
range from 0 (completely clear sky) to 8 (completely obscured by clouds), to
continuous percentages has been done in different ways by climatologists. A
common assumption is that 1 okta equals 12.5 % of cloud coverage (Boers
et al., 2010; Kotarba, 2009). For use in this study it was necessary to
estimate the error or uncertainty in the okta in measurements. It is assumed
that the man-made nature of cloudiness okta estimation has errors of
Calculation of cloudiness in percentage for corresponding okta values.
AERONET is a network of approximately 700 ground-based sun photometers
established by National Aeronautics and Space Administration (NASA) and
PHOtométrie pour le Traitement Opérationnel de Normalisation
Satellitaire (PHOTONS). This globally distributed network aims to provide
long-term and continuous measurements of AOT, inversion products and
perceptible water in diverse aerosol regimes (Holben et al., 1998). The high
temporal resolution of 15 min for these data and expected low accuracy of
AERONET data are categorized and available at three levels: level 1.0 (unscreened), level 1.5 (cloud screened and quality controlled) and level 2.0 (quality assured). The data used in this work are selected from level 1.5 to validate cloud identification results from newly developed ASCIA. More details on the validation procedure are discussed in Sect. 5.2.
The PCC was proposed by Pearson (1896) and is used in this study as an indicator of the correlation between sequential AATSR measurements. The PCC is also known as the Pearson product-moment correlation coefficient (PPMCC). It is a standard dimensionless statistical parameter commonly used to measure the strength and direction of the linear association between a pair of variables (Benesty et al., 2009). This parameter has extensively been used in many studies which pursue pattern analysis and recognition.
Our PCC analysis separates the surface reflectance at a given viewing angle,
which is stable over short time periods, from the cloud reflectance, which is
highly variable over a short time period. To describe the computational
procedure developed, we assume
For this aim, the use of all seven channels (0.55, 0.66,
0.87, 1.6, 3.7, 11 and 12
A second question in PCC analysis (after wavelength selection) is the definition
of the optimal size of the block of the ground scene for PCC calculation. In
an early version of the current algorithm, we set up
The reflectance part of TIR channels at 3.7 and 3.9
To do this, we use the method described in Meirink and van Zadelhoff (2016) and Musial et al. (2014), where the reflectance of 3.7
Theoretical reflectance values in the 3.7
Simulated and observed reflectance values at 3.7
The ASCIA implementation is initiated by preparing a time series of data. A time span of 1 month for the ground scene was selected. Hagolle et al. (2015) indicated that in Sentinel-2 measurements with revisit time of 5 days, most of the given scenes would be observed cloud-free at least once a month. Consequently, we also assume that every scene of AATSR measurements, which have a higher revisit time of 3 days, will be cloud-free at least once a month.
Depending on the latitude and the time of year, the number of downloaded data
varies from 10 to 50 or more over the same scene. AATSR provides more data
over higher latitudes, which increase in spring and summer due to longer
polar days and solar illumination. The AATSR L1b data are already provided as
gridded and calibrated
In the first step, a PCC analysis for a block of ground scenes (
The schematic flow chart of ASCIA.
After computing the first binary cloud flag at block level using the last
measurement and one previous image, ASCIA keeps the result in memory and
repeats the procedure with the second-to-last data. This procedure is iterated until
the last measurement of the data series is involved. The final binary blocks
are imported in the second step to identify cloudy scenes based on
thresholds defined differently for blocks with low and high PCC value. We
note that, the snow/ice reflectance in the 3.7 For the high PCC For PCC
Land classification criteria in the cloud-free scene.
In our method, the PCC analysis constrains the procedure and the strict decision
is only made within low PCC blocks. The loss of some clear scenes in low PCC
blocks is an unavoidable side effect of using these strict criteria, in
particular over land scenes, which have low PCC and high 3.7
Examples of the results of ASCIA on AATSR observations on the scenes
over Greenland
Although characterized as land, a scene may include soil, different types of vegetation cover or even melting snow. The latter mixes with soil and becomes dark enough to be filtered out from the snow class. Sea ice is distinguished from water on the basis of its greater brightness; one scene might be white enough to be considered as ice. However, melting or broken ice, as well as new ice, would not be labelled as ice. Snow over sea ice is not distinguished from pure sea ice and both of them are labelled as sea ice. This also means that ice over land is also marked as snow as well as pure snow.
A representative example of the block level (
In this study, we applied our recently developed ASCIA to identify cloud in
the scenes using AATSR L1b (TOA reflectance) and SLSTR L1b gridded data. The
input file to the process chain is one scene of the AATSR L1b product. The output
comprises five classes of surface types, including snow/ice, sea ice, water,
cloud and land. The procedure of surface classification is explained in
Sect. 4. The location and time of selected case studies are used to show that
the identification of cloud by our new ASCIA is adequate. The AATSR data are
selected from several years starting from 2006, during strong Arctic haze
episode, which originated predominantly from agricultural fires burning in
eastern Europe. The event has been reported previously (Law and Stohl, 2007).
A second episode in 2008 is also considered, for which validation data are
available from SYNOP stations. Three months of data from March, May and July
have been acquired over Greenland and Svalbard to assess the performance of
ASCIA in a wide range of solar zenith angles (60–85
As we discussed in Sect. 1, misclassification of thin cirrus cloud with clear
snow is reported to be an unresolved problem of ISTO approach. Two
representative scenarios of this problem are illustrated in Figs. 6 and 7
over Greenland and Svalbard respectively, in which thin cloud is detected as
clear snow by the ISTO method, whereas ASCIA confirmed the presence of cloud.
Over a homogeneous surface such as Greenland, the second step of ASCIA is
decisive. The lack of structural patterns on the surface leads to low PCC values
in the first step and consequently an overestimation of cloudy scenes. However,
the reflection part of 3.7
The ESA cloud product from L2 data overestimates cloud, which leads to a loss of clear snow and ice scenes. The tendency of this product to flag clear scenes as cloud is also visible in Figs. 6 and 7. The results in Fig. 8 show undetected clouds as another problem of the AATSR level 2 cloud product, which happens frequently at high solar zenith angles. To have a better understanding of this misclassification, we validated the AATSR L2 nadir cloud flag against SYNOP measurements and the results are described in Sect. 5.2.
Poor performances for cases over the Arctic with high solar zenith are
observed in all of the results using ISTO method. Figure 8 is an example for
Svalbard in March 2008. Over a highly variable surface type, such as
Svalbard, the reflection at 3.7
Figure 10 shows one example of a haze event over Svalbard on 3 May 2006. Both
the ESA and ISTO cloud products showed good results for this case with the
exception of thin-cloud scenes, which are falsely labelled as clear snow by
ISTO. The appropriate design and application of PCC analysis over
1.6
The only season in which all three approaches detected clouds with similar success was in July, as shown in Fig. 9. Although ASCIA shows an overall better performance, in particular for thin clouds, the required computational time for cloud detection and surface classification is higher than for the two other methods.
We also compared our results with those from the MODIS cloud identification algorithm, used for masking cloudy scenes. As an example, Fig. 11 shows the AATSR scene over Svalbard on 14 July 2008, where a large part of the sea ice is covered with thin clouds, which have a small signature in the visible channels. The middle panel shows the MODIS cloud mask for the same area. Although there is a small time difference of 15 min between MODIS and AATSR overpasses, we see that scenes identified as cloudy by ASCIA correspond well with those of MODIS.
Figure 12 shows another example over the north-west of Greenland on 18 May 2008. The thin and broken clouds are well detected over the snow cover by ASCIA, as well as the clouds over the southern part of the scene, which is covered with snow and ocean. As we can see from the comparison between ASCIA and MODIS cloud scene identification, cloudy scenes in the northern part of scene are not captured by the MODIS product, but the presence of clouds is seen in the RGB image in the left panel. We observed other cases with similar differences, especially for thin and broken clouds. There are two potential sources of these differences: (1) time differences, which are 10 min in this case, or (2) an inadequate performance of the MODIS cloud mask over bright surfaces covered by snow and ice.
Due to the loss of Envisat and thus AATSR data in 2012 and the need for long
time series of consistent data, we tested ASCIA on the AATSR successor SLSTR
as well. Figure 13 shows some results over Svalbard on 18 April 2017. Due to
the smaller swath width of AATSR compared to SLSTR, ASCIA is not applied to
the full coverage of SLSTR and the selected scene is cropped to have a
similar coverage of
In this section, we present a quantitative validation of our ASCIA results by
making comparisons with simultaneous ground-based SYNOP and AERONET
measurements. The ESA standard cloud product is also compared with these
validation data sets. The difference in spatial and temporal resolutions of
the new cloud identification data sets and the data sets used to validate
this data set have to be taken into account. To define the optimal maximum
temporal difference between SYNOP and satellite data, other comparable
validation activities used different temporal intervals like 10 min
(Werkmeister et al., 2015), 15 min (Musial et al., 2014), 1 h (Dybbroe et
al., 2005) and 4 h (Meerkötter et al., 2004). The investigation and
results in the previous publications indicate that temporal differences in
validation of satellite retrievals against SYNOP depend on meteorological
conditions. Allowing only a small temporal difference between measurement
data sets (here, SYNOP and ASCIA) ensures an optimal temporal overall but can
introduce a significant sampling error due to the small number of scenes for
validation (Bojanowski et al., 2014). According to Bojanowski et al. (2014) a
temporal difference of 90 min between measurement data sets (SYNOP measurements
at a temporal resolution of 3 h and satellite retrievals) minimizes the sampling error.
However, a potentially longer temporal difference will introduce an error
which should be considered along other sources of uncertainty (different
viewing perspective, different spatial footprint, etc.). In this study, the
maximum allowed temporal difference between the ASCIA retrievals and SYNOP
measurements is less than
Density plot of occurrences of the CFC by ASCIA as a function of SYNOP.
Histogram of CFC differences (blue is ASCIA minus SYNOP; red is ESA cloud product minus SYNOP).
CFC in percent by ASCIA (red), SYNOP (blue) and ESA Cloud Product (green) for 100 scenarios in March, May and July 2008 over Svalbard and Greenland. Light-blue error bars show the range of percentage values for each okta from SYNOP measurements.
In Fig. 14 we present the relation between the calculated cloud fractional
cover (CFC) from ASCIA and SYNOP measurements and density plot of occurrences
of the CFC by ASCIA as a function of SYNOP, following the idea of Werkmeister
et al. (2015). The two data sets have a correlation coefficient of
A summary of the comparison of ASCIA and ESA cloud products with SYNOP measurements used to validate these products.
We also validated ASCIA cloud identification results with AERONET level 1.5 measurements, which are cloud screened. The procedure for this validation takes place in two steps: (1) covering AERONET-observed AOT to a cloud flag (AOT is provided in AERONET only in cloud-free conditions) and(2) validation of ASCIA with AERONET cloud flag. In 86.1 % of 36 studied scenes over Svalbard, both ASCIA and AERONET confirm the presence of clouds.
A new cloud detection algorithm, called ASCIA, has been developed for use at
high altitudes above bright surfaces to generate stand-alone products and for
subsequent use in the retrieval of AOT over the Arctic. ASCIA has been
developed for use with the data from the European instrument AATSR on the ESA
Envisat (2002 to 2012) and SLSTR on ESA Sentinel-3A or -3B. ASCIA initially
employs a time series analysis of PCC to identify cloud presence, the
stability and cloud-free conditions on the block scale of scenes (
The results of applying the newly developed ASCIA are compared and validated
against five existing products and methods over the Arctic: (1) SYNOP
measurements, (2) AERONET measurements, (3) one of the existing methods based
on the spectral shape of clear snow, (4) AATSR L2 nadir cloud flag, (5) MODIS
cloud product. The validation resulted in an overall agreement of 96 %
(within
The validation results indicate that the current ESA AATSR L2 nadir cloud
flag often falsely identifies clouds over snow/ice, except during summer. The
comparison between the ESA AATSR L2 cloud product and SYNOP measurements
resulted in an agreement of 68 % (within
The AATSR, SLSTR, MODIS and AERONET data are publicly available:
AATSR:
SJ designed and developed the algorithm, performed the analyses, validated the results and prepared the manuscript. LM, MV and JPB supervised the research project and contributed to the writing and revision of the manuscript. VR provided general advice and valuable discussions on the paper and the work. RH provided the SYNOP data.
The authors declare that they have no conflict of interest.
This article is part of the special issue “Arctic mixed-phase
clouds as studied during the ACLOUD/PASCAL campaigns in the framework of
(AC)
We gratefully acknowledge the funding by the Deutsche Forschungsgemeinschaft
(DFG, German Research Foundation) – project number 268020496 – TRR 172,
within the Transregional Collaborative Research Center “ArctiC
Amplification: Climate Relevant Atmospheric and SurfaCe Processes, and
Feedback Mechanisms (AC)