Validation of the Cloud_CCI (Cloud Climate Change Initiative) cloud products in the Arctic

The role of clouds in the Arctic radiation budget is not well understood. Ground-based and airborne measurements provide valuable data to test and improve our understanding. However, the ground-based measurements are intrinsically sparse, and the airborne observations are snapshots in time and space. Passive remote sensing measurements from satellite sensors offer high spatial coverage and an evolving time series, having lengths potentially of decades. However, detecting clouds by passive satellite remote sensing sensors is challenging over the Arctic because of the brightness of snow and ice in the ultraviolet and visible spectral regions and because of the small brightness temperature contrast to the surface. Consequently, the quality of the resulting cloud data products needs to be assessed quantitatively. In this study, we validate the cloud data products retrieved from the Advanced Very High Resolution Radiometer (AVHRR) post meridiem (PM) data from the polar-orbiting NOAA-19 satellite and compare them with those derived from the ground-based instruments during the sunlit months. The AVHRR cloud data products by the European Space Agency (ESA) Cloud Climate Change Initiative (Cloud_CCI) project uses the observations in the visible and IR bands to determine cloud properties. The ground-based measurements from four high-latitude sites have been selected for this investigation: Hyytiälä (61.84^∘ N, 24.29^∘ E), North Slope of Alaska (NSA; 71.32^∘ N, 156.61^∘ W), Ny-Ålesund (Ny-Å; 78.92^∘ N, 11.93^∘ E), and Summit (72.59^∘ N, 38.42^∘ W). The liquid water path (LWP) ground-based data are retrieved from microwave radiometers, while the cloud top height (CTH) has been determined from the integrated lidar–radar measurements. The quality of the satellite products, cloud mask and cloud optical depth (COD), has been assessed using data from NSA, whereas LWP and CTH have been investigated over Hyytiälä, NSA, Ny-Å, and Summit.

The Cloud_CCI COD results for liquid water clouds are in better agreement with the NSA radiometer data than those for ice clouds. For liquid water clouds, the Cloud_CCI COD is underestimated roughly by 3 optical depth (OD) units. When ice clouds are included, the underestimation increases to about 5 OD units. The Cloud_CCI LWP is overestimated over Hyytiälä by ≈7 g m⁻², over NSA by ≈16 g m⁻², and over Ny-Å by ≈24 g m⁻². Over Summit, CCI LWP is overestimated for values ≤20 g m⁻² and underestimated for values >20 g m⁻². Overall the results of the CCI LWP retrievals are within the ground-based instrument uncertainties. To understand the effects of multi-layer clouds on the CTH retrievals, the statistics are compared between the single-layer clouds and all types (single-layer + multi-layer). For CTH retrievals, the Cloud_CCI product overestimates the CTH for single-layer clouds. When the multi-layer clouds are included (i.e., all types), the observed CTH overestimation becomes an underestimation of about 360–420 m. The CTH results over Summit station showed the highest biases compared to the other three sites. To understand the scale-dependent differences between the satellite and ground-based data, the Bland–Altman method is applied. This method does not identify any scale-dependent differences for all the selected cloud parameters except for the retrievals over the Summit station. In summary, the Cloud_CCI cloud data products investigated agree reasonably well with those retrieved from ground-based measurements made at the four high-latitude sites.

1 Introduction

Over the past decades, a lack of knowledge of cloud properties in the melting Arctic has limited our understanding of their role in climate change (Devasthale et al., 2020). Clouds influence the earth's radiation budget by scattering the incoming solar radiation and absorbing and emitting outgoing longwave radiation in the thermal infrared. Clouds at different heights and temperatures also have different effects on the incoming shortwave (SW) and outgoing longwave (LW) radiation. The net amount of solar radiation reaching the earth's surface depends on the optical properties of clouds and their horizontal and vertical geometrical extent. For example, the low-level cloud fraction from 2000 to 2016 increased over the sea ice regions of the Beaufort and East Siberian seas during the summertime and resulted in an increasing ice melt. This behavior strengthens the ice–albedo feedback in the Arctic (Huang et al., 2021). At least since the beginning of the 1990s, Arctic surface temperature has been increasing more rapidly than at lower latitudes, a phenomenon called Arctic amplification (AA; Wendisch et al., 2017, and references therein). Long-term data records of cloud properties are required to understand the Arctic clouds and their impacts on AA's causes and consequences. Cloud properties, measured at the sparsely located ground-based sites can be complemented by satellite retrievals to improve the coverage of observations. Improved knowledge of the spatial and temporal variation in cloud micro-physical properties (e.g., cloud optical depth, COD; liquid water path, LWP) and macro-physical properties (e.g., cloud fraction, CF; cloud top height, CTH) in the Arctic improves our understanding of cloud processes and radiation balance (e.g., He et al., 2019; Devasthale et al., 2020; Lelli et al., 2023). Cloud data products from satellite remote sensing provide daily global coverage and high spatial resolution over the Arctic. In addition, these cloud data products can help to evaluate climate models and assess anomalies in the reanalysis data products (Liu and Key, 2016; Shaw et al., 2021). Thus the accuracy of these cloud data products needs to be assessed and validated in the Arctic. However, retrievals of cloud properties from measurements by passive remote sensing instruments over snow and ice are challenging and thus error-prone. These errors are mainly because of the low spectral contrast between the cloud and the surface in the visible spectral region and temperature inversions which influence the radiance observed in the near-infrared and thermal infrared spectral regions (see Dybbroe et al., 2005; Marchand, 2016, and references therein). In addition, multi-layer clouds occur in the Arctic, leading to issues in interpreting the cloud parameters (e.g., CTH) and resulting in biases. The lack of validation studies for the satellite cloud data products at high latitudes may have limited their use by the scientific community undertaking research in the Arctic. There are some exceptions, where TROPOspheric Monitoring Instrument (TROPOMI) products are compared with ground-based instruments and other satellite products (Compernolle et al., 2021). Other studies which have used the cloud cover (Schweiger, 2004; Boccolari and Parmiggiani, 2018; Philipp et al., 2020) have investigated cloud trends and variability over the Arctic. In addition, some studies, such as Sporre et al. (2016), have compared LWP and CTH retrievals from the Moderate Resolution Imaging Spectroradiometer (MODIS) and Visible Infrared Imaging Radiometer Suite (VIIRS) with the measurements of the Atmospheric Radiation Measurement (ARM) mobile facility at the high-latitude site in Hyytiälä. The results from this study showed an LWP difference of <15 g m⁻² and CTH differences of <500 m between satellite and ground-based observations. However, the CTH differences are more than 1000 m for the clouds located above 6000 m. Earlier versions of the MODIS measurements were compared with ARM measurements over the North Slope of Alaska (NSA), where the authors identified cloud detection problems at high viewing angles (>55^∘) (Berendes et al., 2004). Another study by Liu et al. (2017) examined active satellite observations in the Arctic compared to two ground-based stations, NSA and Eureka, and found from the cloud fractions that the satellite observations show 25 %–40 % fewer near-surface clouds (<0.5 km).

In this study, we validate the satellite cloud retrieval using selected cloud parameters, i.e., cloud mask, COD, LWP, and CTH, from the Cloud Climate Change Initiative (Cloud_CCI) dataset with ground-based cloud data products. The Cloud_CCI is a recent global dataset of cloud properties with good coverage in the Arctic. The Cloud_CCI data products are retrievals from the measurements made by the Advanced Very High Resolution Radiometer (AVHRR) on a series of satellite platforms (Stengel et al., 2020). It is part of the European Space Agency's program to provide climate data records within the United Nations Convention on Climate Change framework. In this study, we use the data products retrieved from the AVHRR on NOAA-19 daytime measurements made during the sunlit months, specifically April, May, June, July, August, and September for the time period 2010–2018. NOAA-19 flies in a sun-synchronous orbit in an ascending node, having an approximate Equator crossing time of 13:30 local time when it was launched. Previously, validation of the Cloud_CCI dataset has typically focused on latitudes outside of the Arctic (see Jeanneret et al., 2018). In this study, ground-based measurements from one sub-Arctic site at Hyytiälä, Finland (61.84^∘ N, 24.29^∘ E), and three Arctic sites at NSA (North Slope of Alaska), USA (71.32^∘ N, 156.61^∘ W); Ny-Ålesund, Svalbard (78.92^∘ N, 11.92^∘ E); and Summit, Greenland (72.59^∘ N, 38.42^∘ W) are used to validate the Cloud_CCI data product. All these sites have similar ground instrumentation. The measurements made at Hyytiälä were part of the Biogenic Aerosols – Effects on Clouds and Climate (BAECC) campaign and are available for 2014. In contrast, those at NSA and Summit provide a relatively long-term data record from 2010–2018 for LWP. For CTH over NSA and Summit, data records from 2012–2016 and 2010–2014, respectively, were used. The ground-based data record from Ny-Ålesund (Ny-Å) used in this study covers the years 2016–2018. The Hyytiälä and NSA ground-based data are from the ARM research facilities. The ARM program, initiated by the US Department of Energy, provides atmospheric measurements from targeted campaigns and also long-term measurements at some sites. ARM cloud data products have been used to validate satellite cloud retrievals (Rutan et al., 2001; Ji and Shi, 2012; Sporre et al., 2016). The data from Summit were obtained from the Integrated Characterization of Energy, Clouds, Atmospheric state, and Precipitation at Summit (ICECAPS) a project funded by the US National Science Foundation. For Ny-Å, measurements are taken from the AWIPEV Atmosphere Observatory. While some measurements at AWIPEV, e.g., radiosondes, surface radiation, and meteorology, started more than 30 years ago, cloud observations have been enhanced in 2016 as part of the Transregional Collaborative Research Center TRR 172 “ArctiC Amplification: Climate Relevant Atmospheric and SurfaCe Processes, and Feedback Mechanisms (AC)³” (Wendisch et al., 2022). In particular with the installation of a cloud radar, enhanced vertically resolved cloud observations became possible (Nomokonova et al., 2019a).

The satellite retrievals are often subject to a scale bias (i.e., scale-dependent bias) compared to the measurements from ground-based instruments. They tend to overestimate (underestimate) the high (low) values of COD, LWP, and CTH (Sporre et al., 2016). Consequently, in the present study, we investigated the scale bias between the selected Cloud_CCI data products and those from the ground-based measurements in the Arctic. For this purpose, we use the Bland–Altman (BA) approach (Altman and Bland, 1983; Bland and Altman, 1986; Knobelspiesse et al., 2019). The BA approach uses the normalized bias for the paired mean values between the ground-based measurements and the Cloud_CCI data for the selected cloud parameter.

In Sect. 2 of the paper, we introduce the Cloud_CCI cloud data products and then the corresponding data product from the ground-based observational sites. Section 3 describes the method used to assess and validate the Cloud_CCI data products, i.e., the statistical techniques. The results and their discussion are presented in Sect. 4. In the last section, we summarize the findings of this study.

2 Data sources

2.1 CCI data

As described above, we use the Cloud_CCI dataset retrieved from the measurements of AVHRR which flew on NOAA-19. The interested reader is referred to the detailed publication about the data by Stengel et al. (2020). The Cloud_CCI retrievals use the AVHRR radiance measurements centered at the wavelengths 0.6, 0.8, 3.7, 11, and 12 µm. The version 3.0 Level-3U daily products used in this study contain selected Level-2 data (satellite pixel level data) on a global 0.05^∘ longitude–latitude grid (Stengel et al., 2020). The Cloud_CCI products are retrieved using the Community Cloud retrieval for CLimate (CC4CL) retrieval system (Sus et al., 2018; McGarragh et al., 2018). It comprises three major steps involving the following:

• A cloud mask that identifies the cloudy and clear scenes is determined using an artificial neural network (ANN). The ANN uses the AVHRR radiances trained with the co-located Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) COD to identify the presence or absence of cloud. In addition, a detection algorithm for cirrus clouds is used based on the brightness temperature differences in the IR band radiances at 11 and 12 µm and defined threshold values as proposed by Pavolonis et al. (2005).

• A cloud-phase mask is used to determine liquid water and ice clouds using an ANN approach. The ANN cloud-phase mask is similar to the cloud mask ANN, but it is trained with the CALIOP cloud-top-phase data. The cloud-phase retrieval provides binary information, liquid or ice.

• An optimal estimation algorithm (Rodgers, 1976) is applied to the measured AVHRR radiances using the retrieved cloud mask and cloud-phase information to generate the cloud data products. The ancillary information, such as the land bidirectional reflectance distribution function (BRDF), surface temperature, and surface emissivity, is taken from the MODIS C6 (Schaaf et al., 2010), ERA-Interim (Dee et al., 2011), and Cooperative Institute for Meteorological Satellite Studies (CIMSS) databases, respectively (Seemann et al., 2008). Cloud properties retrieved using the optimal estimation algorithm are cloud top pressure, cloud effective radius (CER), and COD in the SW region. These are used to calculate cloud liquid and ice water paths (Stephens, 1978). The COD and LWP are daytime products. One of the key features of the Cloud_CCI dataset comprises the uncertainties provided at the pixel level inferred from the implemented optimal estimation theory (Stengel et al., 2020). The Cloud_CCI data have been validated by comparing them to independent spaceborne references, especially for non-polar regions (Stengel et al., 2020; Stapelberg et al., 2019). The main focus of these studies was global-scale evaluation against space-based reference data as well as global-scale comparisons to existing datasets of similar types. The results presented in the article at hand extend the evaluation efforts by using high-quality, ground-based reference measurements in the Arctic.

2.2 Ground-based measurements

This study uses ground-based data from Hyytiälä, NSA, Ny-Å, and Summit stations shown in Fig. 1. Although of limited duration, the campaign data products, measured as part of the BAECC campaign (2014) over Hyytiälä, provide relatively dense records with similar instrumentation and are directly comparable to measurements at the NSA (Verlinde et al., 2016), Ny-Å (Nomokonova et al., 2019a), and Summit (Shupe et al., 2013) sites. Due to the availability of data products from the ground-based measurements, the CCI cloud mask (CM) and COD are validated at NSA, whereas CCI LWP and CTH are validated at all four sites.

Figure 1 Arctic polar stereographic map showing the selected ground-based sites.

2.2.1 Cloud fraction and optical depth

The cloud fraction (CF) and cloud optical depth over NSA are obtained from the radiative flux analysis product. This value-added product is based on the shortwave irradiance from the pyranometer to identify clear and cloudy skies (Riihimaki et al., 2019). In the presence of clear skies, the shortwave irradiance, measured at the ground, is larger than that for cloudy skies. An empirical fit function is then used to calculate the continuous clear-sky estimates and cloud optical property data products (Long and Ackerman, 2000; Riihimaki et al., 2019). The CF is derived using the methodology described in Long et al. (2006), which assumes that increasing cloud cover increases the diffuse irradiance relative to the direct irradiance (Riihimaki et al., 2019). The SW COD is retrieved using the algorithm from Barnard and Long (2004), which has been adapted for the identification of optically thin clouds (Barnard et al., 2008). The algorithm enables COD for liquid and ice clouds to be retrieved. The algorithm differentiates the liquid and ice clouds based on the fixed asymmetry parameter of 0.87 and 0.80, respectively, as suggested by Fu (1996) and Barnard et al. (2008). The retrieved SW COD is only valid for overcast conditions, i.e., cloud fractions greater than 0.95 (Min and Harrison, 1996). The absolute differences in sky cover amount derived from the SW CF estimates and total sky imager (TSI) retrievals agree within 10 %, indicating a good level of agreement (Long et al., 2006; Barnard et al., 2008).

2.2.2 Liquid water path

The LWP at Hyytiälä is retrieved from the three-channel microwave radiometer instrument (MWR3C) data product. The center frequencies of the microwave radiometer (MWR) are 23.83, 30, and 89 GHz, which coincide with the peak absorption frequencies for the liquid water (Cadeddu, 2021). The use of the three channels makes the retrieval sensitive to the small LWP values expected in dry environments, such as larger parts of the Arctic (Cadeddu et al., 2009; Cadeddu, 2021). The retrieval uses an ANN approach as described by Cadeddu et al. (2009). The LWP at NSA is retrieved from MWR-measured brightness temperatures at 23.80 and 31.40 GHz frequencies, while at Summit the same technique is applied to the same channels in addition to a 90.0 GHz channel. For these two sites, a physically based radiative transfer model for monochromatic light called MonoRTM uses the observed measurements to create the MWR RETrieval (MWRRET) product (Turner et al., 2007; Gaustad et al., 2011). The MWRs make measurements every 28 s, and the random uncertainties associated with the retrieval for MWR3C and MWRRET are ≈15 and ≈20 g m⁻², respectively (Turner et al., 2007; Cadeddu, 2021). For Ny-Å, LWP is retrieved from the Humidity And Temperature PROfiler (HATPRO) of the Alfred Wegener Institute for Polar and Marine Research (Nomokonova et al., 2019b, a). HATPRO is an MWR providing brightness temperature measurements at seven frequencies between 22.24 and 31.40 GHz and at seven frequencies between 51.26 and 58.00 GHz with a temporal resolution of about 1 s. LWP is retrieved from a linear regression algorithm (Nomokonova et al., 2019a). The uncertainty is estimated to be about 20–25 g m⁻² (Rose et al., 2005). In the analysis, HATPRO LWP has been excluded in cases where the quality flag was set, i.e., indicating for example precipitation, and in cases where an additional check indicated that the MWR brightness temperatures of one frequency channel were spectrally inconsistent with the other channels of the same band.

2.2.3 Cloud top height

CTH at Hyytiälä and NSA is measured using the Ka-band ARM Zenith Radar (KAZR) Active Remote Sensing of Clouds (KAZR-ARSCL) data product, which blends the radar observations with micropulse lidar observations (Clothiaux et al., 2001) to provide, among other parameters, a detailed vertical cloud mask with cloud boundaries. The lidar provides supporting information on the cloud boundaries, although for optically thick clouds the radar typically provides information on the cloud top due to attenuation of the lidar signal. The CTH over Summit is derived from an earlier version of the ARSCL algorithm but uses only observations from a millimeter cloud radar (MMCR), which was the precursor for the KAZR and is an operationally similar radar. Both radars used here are 35 GHz, Ka-band radars, which are commonly used for cloud detection because the oxygen and water vapor absorption at these frequencies is at a local minimum (Clothiaux et al., 2001; Kollias et al., 2007). The reflectivity in the range of −50 to 20 dBZ for the hydrometeor layer up to 10 km or higher is accurately detected by the cloud radar (Clothiaux et al., 2000). The accuracy of the measurement for a stratocumulus or altocumulus cloud with liquid particle size in the range of 3–5 µm at −50 dBZ is such that the detection limit for a hydrometeor layer is smaller than a condensed water content of ≤0.01 g m⁻³ and hydrometer layers in the range 0.01–0.03 g m⁻³ are measurable (Noonkester, 1984; Heymsfield et al., 1991; Clothiaux et al., 2001). The radar products have a vertical resolution of 45 m and a time resolution of 2–4 s. For Ny-Å, the Cloudnet classification product (Illingworth et al., 2007; Nomokonova et al., 2019a) was used to determine CTH. To this end, measurements at the AWIPEV Atmosphere Observatory, i.e., cloud radar reflectivity, Doppler velocity, and ceilometer attenuated backscatter, are jointly analyzed with numerical weather prediction data. The resulting classification profiles have a temporal (vertical) resolution of 30 s (20 m) and provide information on the presence of cloud liquid droplets, ice, melting ice, and drizzle/rain in each radar height bin up to a height of about 12 km. The CTH value corresponding to the height above ground level is converted to the height above mean sea level to match satellite CTH values.

3 Method

One of this study's essential aspects is to match the satellite and the ground-based instruments spatio-temporally. We follow the spatial co-location strategy from Stengel et al. (2020) i.e., 5 km radius. However, in comparison to Stengel et al. (2020), we relaxed the temporal co-location from 3 to 5 min and averaged the ground-based data over this 5 min interval. We expect the satellite and ground-based instruments to view the same cloud even at this relaxed interval, bearing in mind that the Arctic atmosphere is less dynamic than at the lower latitudes (Kay et al., 2016; Edel et al., 2019). For the quality criteria, the pixels for the selected Cloud_CCI cloud data are not qualified for validation if the cost values are too high in the optimal estimation retrieval. The exact limits for each Cloud_CCI cloud parameter for the iteration process in the optimal estimation can be found in McGarragh et al. (2018). The validation uses scatterplots and linear least-squares regression coefficients to interpret the results. The reliability of the regression coefficients is examined using the t distribution (Phillips, 1986). In the following text, the terms overestimation and underestimation refer to the median bias.

Bland–Altman plots

Validation or assessment studies often use scatterplots to infer the difference or bias between the space and ground measurements. This method has its limitations (Martonchik et al., 2004; Van Harten et al., 2018; Seegers et al., 2018). For example, linear regression models do not provide information on scale-dependent biases. In addition to using scatterplots, we use Bland–Altman (BA) plots to further investigate the behavior of the bias. The BA approach used here plots the differences between the paired measurements, normalized by the root mean square of the standard deviations versus the measured paired mean. This plot enables scale-dependent and scale-independent biases to be identified (Altman and Bland, 1983; Bland and Altman, 1986). There have been a few relevant studies that have used this approach effectively (Knobelspiesse et al., 2019; McKinna et al., 2021). The steps of our BA analysis are as follows:

• Calculate the paired mean (M) for all matched observations for the chosen cloud parameter,

  $$\begin{array}{}\text{(1)}& M_i={\displaystyle \frac{X_i+Y_i}{\mathrm{2}}},\end{array}$$

  where M_i is the paired mean for the ith observation, X_i is the ground ith observation, and Y_i is the satellite ith observation.

• Calculate the normalized differences (D) for all matched observations for the chosen cloud parameter,

  $$\begin{array}{}\text{(2)}& D_i={\displaystyle \frac{X_i-Y_i}{{\left({\mathit{\sigma }}_{\mathrm{c}i}^{\mathrm{2}}+{\mathit{\sigma }}_{\mathrm{a}i}^{\mathrm{2}}\right)}^{\mathrm{1}/\mathrm{2}}}},\end{array}$$

  where D_i is the normalized difference for the ith observation, σ_ci is the Cloud_CCI pixel uncertainty for the ith observation in the optimal estimation, and σ_ai is the standard deviation observed in the ground-based measurements for the temporally averaged period for the ith observation.

• Plot M vs. D, and calculate the linear regression coefficients (Pearson coefficient, r) with the significance testing (t test). The corresponding formula for r is

  $$\begin{array}{}\text{(3)}& r={\displaystyle \frac{{\sum }_{i=\mathrm{1}}^n(X_i-\overline{x})(Y_i-\overline{y})}{\sqrt{{\sum }_{i=\mathrm{1}}^n(X_i-\overline{x}{)}^{\mathrm{2}}}\sqrt{{\sum }_{i=\mathrm{1}}^n(Y_i-\overline{y}{)}^{\mathrm{2}}}}},\end{array}$$

  where $\overline{x}$ is the mean of ground observations, $\overline{y}$ is the mean of satellite observations, and n is the total number of observations.

• Check whether the Pearson coefficient from the above step is insignificant at the 95 % confidence level. BA bias (B) and limits of agreement (LOAs) are defined in this case. The BA bias is the mean normalized difference (D), and LOA denotes the 2σ region for the observed BA bias.

  $$\begin{array}{}\text{(4)}& {\displaystyle }B=\sum _{i=\mathrm{1}}^n{\displaystyle \frac{D_i}{n}},\text{(5)}& {\displaystyle }\text{LOA}=\left[B-\mathrm{2}\times B_{\mathrm{s}},B+\mathrm{2}\times B_{\mathrm{s}}\right],\end{array}$$

  where B_s is the standard deviation in D_i.

• If the significance test fails, then B and LOA are considered meaningless and are not calculated. In this case, the bias is dependent on the paired mean, which implies that the bias changes significantly with the paired mean, i.e., a scale-dependent or variant case.

• We assume that an ideal distribution of normalized differences would have a bias of 0 % and 95 % of the values lie within $\pm \mathrm{2}\times {\left({\mathit{\sigma }}_{\mathrm{c}i}^{\mathrm{2}}+{\mathit{\sigma }}_{\mathrm{a}i}^{\mathrm{2}}\right)}^{\mathrm{1}/\mathrm{2}}$. The obtained results from the BA plots (B, LOA) are interpreted by comparing them to this assumed ideal range.

4 Results and discussion

4.1 Cloud mask

The initial step in any cloud retrieval scheme is to apply a cloud mask to the satellite observations. This classifies the pixels as cloudy or cloud-free. If a cloud is identified in a pixel, cloud data products are retrieved. The Cloud_CCI, as discussed before, uses an ANN trained with CALIOP data to identify the cloudy and cloud-free pixels. Figure 2 shows the plot of SW CF vs. SW COD from NSA, stratified by Cloud_CCI cloud mask information. The COD from NSA is divided into three ranges as follows, less than 3 optical depth (OD) units (optically thin), 3–10 OD units, and more than 10 OD units. The black and red colors show the cloudy and clear conditions, respectively, as described in the Cloud_CCI data. The shaded region of Fig. 2 comprises ground-based measurements having CF values of 50 %–100 %. We choose the 50 %–100 % values of CF to investigate whether the Cloud_CCI cloud mask is accurate. For these data, we expect the Cloud_CCI cloud mask to be cloudy when the NSA CF is at least 0.5. The hit percentage for a particular COD range is defined as the number of Cloud_CCI cloudy pixels that occurs in the shaded region (NSA CF >0.5) divided by the total number of pixels in the shaded region. The miss percentage is defined as the number of Cloud_CCI clear pixels that occurs in the shaded region (NSA CF >0.5) divided by the total number of pixels in the shaded region. In this study, we use the hit and miss percentages to quantify the success of the cloud mask. In Fig. 2, the hit percentage is ≈75 % for the optically thin clouds, whereas, for the optical thick clouds, it is as high as 90 %. For the clouds having optical depths greater than 10 OD units, the hit percentage is greater than 95 %. The miss percentage is around 25 % for the optically thin clouds, i.e., for COD ≤3. The miss percentage was reduced to almost 10 % for the clouds, whose optical depths are between 3 and 10 OD units. This relatively high miss percentage for the thin clouds is due to the high cloud detection sensitivities (minimum COT required to detect 50 % of clouds) over polar regions during the AVHRR–CALIOP matchups (Karlsson and Håkansson, 2018). This could also be due to the smaller cloud elements (smaller than AVHRR field of view) that make the cloud optically not thick enough to be sensed by the AVHRR sensor. Another possible explanation for the miss percentages is the presence of fog or clouds that form as a result of a temperature inversion in the boundary layer. Such conditions are more frequent in the Arctic, compared to other regions (Tjernström et al., 2019; Wang et al., 2022). For optically thick clouds whose COD is more than 10 OD units, the miss percentage is as small as 5 %. In summary, the CCI cloud mask shows cloudy for 90 % of the cases when the ground-based COD ≥3 OD units.

Figure 2 NSA cloud fraction (CF) vs. NSA cloud optical depth (COD) for different COD ranges for the cloudy (black) and clear sky (red) stratified by Cloud_CCI measurements at NSA during the daytime in the period 2010–2018. The shaded grey region is the acceptance range, i.e., the range used to analyze the Cloud_CCI cloud mask accuracy. The value of “n” represents the number of samples in the shaded region for that particular COD range. The percentage values written on the plot correspond to the hit (black) and miss percentages (red). More information is found in the text.

4.2 Cloud optical thickness and liquid water path

The validation of Cloud_CCI COD is performed for daytime cases at NSA (ground-based COD is only available at NSA) because the retrieved COD from the instruments uses visible channels. Cloud_CCI COD is classified as either liquid water or ice, as explained in the Cloud_CCI data product. The statistical analysis (i.e., the determination of regression coefficients and BA biases) of the comparison of COD is calculated for the three possible types of COD: all types (liquid water phase + ice phase), the ice phase, and the liquid water phase. Figure 3 shows the regression lines and BA biases for all types and liquid-water-phase types because scale independence is not observed for the ice-phase type. From Fig. 3 and Table 1, it can be seen that the COD retrieved by Cloud_CCI for the liquid-water-phase clouds shows a better agreement compared to the NSA measurements than for the ice-phase clouds. The regression coefficients are improved significantly when only liquid-water-phase clouds are considered, i.e., when the regression line for the liquid-water-phase clouds is closer to the 1:1 line. The Cloud_CCI COD product underestimates the COD: by a bias of 3 OD units for water-phase clouds and 9 OD units for ice-phase clouds. In general, liquid water clouds whose COD is less than 30 OD units are closer to the 1 : 1 line, and for ice-phase clouds, the Cloud_CCI COD values are much lower than the NSA measurement values (red color points below the 1 : 1 line). In summary, the Cloud_CCI product underestimates COD by around 5 OD units. There is no scale dependence for COD observed for the liquid-water-phase clouds. The BA bias for liquid-water-phase clouds is close to 0 (−0.88) and becomes almost twice that magnitude (−1.64) with the inclusion of ice-phase clouds (i.e., the all-case scenario). The LOA for the liquid water clouds is also closer to the ideal range (the shaded region) and becomes wider when augmented with the ice-phase clouds. The percentage of cases outside the ideal bias range is also better for liquid-water-phase clouds than ice-phase clouds. In the all-case type, 40 % of the measurements lie outside the ideal range, and most of these are due to the ice-phase clouds below 30 OD units where an underestimation by Cloud_CCI is clearly visible. This value is reduced to 32 % when only liquid-water-phase clouds are considered.

Figure 3 (a) Ground-based COD vs. Cloud_CCI COD daytime data products plotted for the period 2010 to 2018 at NSA. The data points are classified by liquid water (black) and ice (red) phases as given by the Cloud_CCI cloud product. The thick black line is the 1:1 line. The regression lines for the liquid-water-phase cases (dashed) and for all cases (liquid water + ice; dotted) are shown as well. The error bars represent the standard deviation in the 5 min averaging period of the ground-based measurements and the Cloud_CCI pixel uncertainty. The zoomed-in plot represents the samples below 40 units. (b) Bland–Altman plot showing the paired mean vs. the normalized difference for the liquid-water-phase cases (black dots) and for ice cases (red dots). The thick dashed line indicates the BA bias for the liquid cases with the thin dashed lines indicating the corresponding limits of agreement (LOAs). Correspondingly, the dotted lines indicate the results including all cases (liquid water + ice). The shaded area indicates the assumed ideal range. Table 1 lists the corresponding statistics.

Table 1 Statistics related to Fig. 3, NSA vs. Cloud_CCI COD data for all cases (liquid water + ice), the ice phase, and the liquid water phase for the period 2010–2018. SE represents the standard error. LOA and BA bias represent the limit of agreement and Bland–Altman bias, respectively, as calculated from Eqs. (4) and (5), respectively. The last row represents the percentage of values outside the assumed ideal range.

LWP is further analyzed when the cloud phase is liquid according to the Cloud_CCI product. Figure 4 and Table 2 show that Cloud_CCI generally overestimates the LWP when compared with the ground-based radiometer measurements. The LWP comparison for Hyytiälä, NSA, and Ny-Å reveals similar results: for Hyytiälä and NSA, LWP is overestimated by 7 and 16 g m⁻², respectively (bearing in mind the number of samples over Hyytiälä is quite small). At Ny-Å, LWP is overestimated by about 24 g m⁻². The comparison statistics for the measurements made at Hyytiälä are generally better than those at other sites. The overestimation of LWP is most probably due to the uncertainty in the CER retrievals. The CER, especially in the Arctic, is not necessarily constant with respect to altitude within the cloud. The inhomogeneity of the CER cannot be accounted for in the retrieval. The LWP statistics for Summit are inferior to those at the other three sites. As expected (see, for instance, Bennartz et al., 2013), very low values of LWP are observed over the Greenland ice shield. Unlike the other three sites, a scale-dependent bias is observed approximately at about 20 g m⁻², which can be observed from the BA plot (where the regression line intersects the zero line). Thus, the median biases are calculated for pixels when LWP is ≤20 g m⁻² and for LWP >20 g m⁻². At LWP values ≤20 g m⁻², we observe an overestimation of ≈5 g m⁻² in the Cloud_CCI data, whereas for LWP >20 g m⁻², there is a negative bias i.e., underestimation of about 13 g m⁻², which, according to the BA plot, is strongly driven by the lower range of LWP values (20 g m${}^{-\mathrm{2}}\le \text{LWP}\le \mathrm{50}$ g m⁻²). Because of this scale dependence, LOAs and BA biases are not calculated for Summit. Scale independence was observed over Hyytiälä, NSA, and Ny-Å, leading to the consideration of BA bias and LOAs. These LOAs are much closer to the defined ideal range where the BA bias is less than 1. The percentage of points outside the ideal range is also small for all the sites i.e., 11 % for Hyytiälä, 29 % for NSA, 30 % for Ny-Å, and 9 % for Summit. In previous studies (Sporre et al., 2016), the LWP determined from passive satellite-based remote sensing depends on the solar zenith angle (SZA) and larger biases are observed at higher SZAs. For the Cloud_CCI LWP data product, there is no sign of change in statistics when separating the days based on SZA (not shown). The biases observed at all the sites are smaller than the uncertainty in the ground-based LWP retrievals.

Figure 4 Similar to Fig. 3 but for LWP over Hyytiälä (2014), NSA (2010–2018), Ny-Å (2016–2018), and Summit (2010–2018) for all liquid cases according to the Cloud_CCI product. Table 2 lists the corresponding statistics. Note that the dashed line in the BA plot for Summit is the regression line.

Table 2 Statistics as in Table 1 but related to Fig. 4, showing ground-based vs. Cloud_CCI LWP data for all liquid cases according to the Cloud_CCI product.

4.3 Cloud top height

The comparison is separated into single-layer and multi-layer conditions using the lidar–radar classification. A single-layer cloud is defined when the ground-based instruments showed only one cloud layer throughout the averaged 5 min interval. Similarly, if two or more cloud layers are observed, the scene is defined as a multi-layer cloud. For Ny-Å, multi-layer clouds are assumed if at least two vertical bins (i.e., 40 m) are cloud-free between the cloud layers, while for the other sites one bin (i.e., 45 m) is at least cloud-free between the cloud layers. Figure 5 shows the scatterplots and BA plots for the CTH data. The statistics (Tables 3 and 4) are calculated separately for all cloud types, defined as (i) the sum of single- and multi-layer clouds and (ii) single-layer clouds. The regression line for all cloud types is much closer to the 1:1 line for Hyytiälä (slope =0.87) than those for NSA (0.68), Ny-Å (0.64), and Summit (0.32). The CTH values for the single-layer clouds are in better agreement with the ground-based measurements than those for the multi-layer clouds. In general, the single-layer CTH is overestimated at all sites. When single-layer clouds are combined with multi-layer clouds, i.e., all cloud types, the Cloud_CCI CTH underestimates the ground-based CTH, implying an underestimation of the CTH of multi-layer clouds. This overestimation for single-layer clouds and underestimation for multi-layer clouds is similar for all sites except for Summit where the underestimation of CTH for multi-layer clouds is not so large. In brief, the overestimation for the single-layer CTHs at Hyytiälä, NSA, and Ny-Å are ≈240 m, ≈197 m, and ≈187 m, respectively, while CTHs of all clouds are underestimated by ≈180, ≈208, and ≈172 m at Hyytiälä, NSA, and Ny-Å, respectively (Tables 3 and 4). This difference between the overestimation of single-layer CTH and underestimation of all-cloud CTH over Hyytiälä, NSA, and Ny-Å is around 420, 405, and 360 m, respectively. This suggests that the presence of multi-layer clouds adds an underestimation by about 360–420 m in the Cloud_CCI CTH retrievals. We attribute this underestimation of the CTH of multi-layer clouds to the reduced sensitivity of the passive spaceborne instruments to detect high-altitude thin clouds, which are often comprised of ice. A thick cloud deck underneath thin high-altitude clouds will contribute significantly to the observed radiance at the top of the atmosphere. Similarly to the analyses of the LWP, the analysis of the CTH data from the Summit station (located at ca. 3.2 km a.m.s.l.) showed different statistics and high retrieval uncertainties for the Cloud_CCI CTH as compared to the values at other stations (Table 4). Over Summit, scale-dependent bias is observed with an overestimation of CTH values below 5000 m and an underestimation above 5000 m (derived from the intersection of the regression line with the zero line in the BA plot). This led us to report the median biases for these two separate cases. For clouds whose CTH is below 5000 m over the Summit, the overestimation of CTH for single-layer clouds is around 380 m. In contrast to the other sites, no distinct underestimation of the multi-layer CTH is observed over Summit. For CTH values above 5000 m, CTH is largely underestimated by Cloud_CCI, i.e., by 1100 m for single-layer clouds and by 1371 m for all cloud types (single- and multi-layer clouds).

Figure 5 Similar to Fig. 3 but for CTH over Hyytiälä (2014), NSA (2012–2016), Ny-Å (2016–2018), and Summit (2010–2014). Tables 3 and 4 list the corresponding statistics. The data points are classified by single-layer (black) and multi-layer (red) clouds as identified from the ground-based observations. Left: the regression lines for the single-layer cloud cases (dashed) and for all cloud cases (single- and multi-layer; dotted) are shown as well. Right: the BA bias and limits of agreement (LOAs) are shown for single-layer cloud cases (dashed) and all cloud cases (dotted). Note that the dashed (dotted) line in the BA plot for Summit represents the regression line for single-layer cloud (all cloud) cases instead.

Table 3 Statistics as in Table 1 but related to Fig. 5 showing ground-based vs. Cloud_CCI CTH data over Hyytiälä (2014) and NSA (2012–2016).

Table 4 Statistics as in Table 1 but related to Fig. 5 showing ground-based vs. Cloud_CCI CTH data over Ny-Å (2016–2018) and Summit (2010–2014).

From the BA plots over Hyytiälä, NSA, and Ny-Å, no observable scale-dependent bias for the CTH retrievals is noticed, leading to a meaningful LOA. The LOA over Hyytiälä is much closer to the defined acceptable range and within the defined acceptable range for single-layer CTHs. Compared to Hyytiälä, over NSA and Ny-Å, the LOA is wider for both considered types, i.e., all-cloud CTHs and single-layer CTHs, indicating more significant differences between satellite and ground-based CTH values. These differences are clearly identifiable in the BA plot, where 32 % of such cases fall outside the defined acceptable range for single-layer clouds (Table 3). The number of cases falling outside the defined acceptable range for single-layer clouds is 12 % over Hyytiälä. In contrast, for multi-layer clouds, the samples outside the defined acceptable range are about 40 %, 50 %, and 60 % over Hyytiälä, NSA, and Ny-Å, respectively. For Summit, a high number of values (>60 %) are outside the defined acceptable range, resulting from many CTH values that are underestimated. The BA bias and LOA are not calculated for Summit since they showed scale dependence.

Previous studies from Holz et al. (2008) and Sporre et al. (2016) for MODIS and VIIRS satellite retrievals observed similar patterns of overestimation of CTH for the single-layer clouds and underestimation for multi-layer clouds. These studies did not include high-Arctic sites such as NSA and Summit. The plausible reason reported in those studies is the differences in the cloud optical thicknesses between different cloud layers, which then influence the accuracy of the multi-layer CTH retrievals.

5 Conclusions

The Climate Change Initiative (CCI) global cloud products (Cloud_CCI) are one of the publicly available climate data records. Cloud_CCI makes use of the measurements from the AVHRR post meridiem (PM) dataset, which extends over more than 3 decades. To our knowledge, this is the first study to validate Cloud_CCI data products by comparison with measurements from ground-based instruments in the Arctic (>60^∘ N) and one of few to validate satellite cloud products over the Arctic.

The comparisons of the Cloud_CCI and ground-based cloud data during the sunlit months, available for Hyytiälä, NSA, Ny-Å, and Summit, in most cases, showed high Pearson correlation coefficients (PCCs) ≥0.6, having a 99 % confidence level. The Cloud_CCI cloud mask indicates cloudy for more than 90 % of the time in the presence of optically thick clouds (COD >10), having CF >0.5 as indicated from ground-based observations. Of the clouds, 10 % are missed by Cloud_CCI for cases with COD in the range of 3 to 10. We conclude that the CCI cloud mask is reasonably well validated for optically thick clouds over NSA. Such clouds are found in the oceanic regions of the Arctic, such as the Barents Sea, Laptev Sea, Kara Sea, East Siberian Sea, and the Chukchi Sea, where optically dense clouds are present most of the time during the sunlit months (Kay et al., 2016; Huang et al., 2021). As a result of the warming Arctic, we expect cloud properties to change over time. We, therefore, need sufficiently accurate satellite cloud data records to detect these changes. To this end, we need to understand whether the differences between the satellite data and ground-based data are scale independent or scale dependent. Consequently, the Bland–Altman approach has been used in this study to investigate this issue. If the differences in the cloud properties between the satellite and ground-based data do not significantly increase/decrease with the value of the cloud property itself, they are assumed to be scale independent. The majority of the Cloud_CCI results have no scale biases when compared with the ground-based instruments, except for Summit. In general, the Cloud_CCI underestimates COD by 3 OD units for liquid-water-phase clouds. These results imply that the Cloud_CCI COD values when compared with the ground-based radiometers are expected to have differences ≤3 OD units over NSA and similar regions where liquid-water-phase cloud formation is more active during sunlit months (Gu et al., 2021). The overestimation of LWP at high-latitude sites, such as NSA and Ny-Å, is 2 to 3 times higher than at the low-latitude Arctic site Hyytiälä. At NSA, Ny-Å, and Hyytiälä, the Cloud_CCI CTH retrievals overestimate the single-layer cloud CTH but underestimate the CTH of multi-layer clouds. The comparison over Summit, Greenland, showed a different CTH behavior compared to the other three stations. At Summit, significant scale-dependent biases are observed in single-layer clouds and for all cloud types (single-layer + multi-layer). These issues of high CTH biases over Greenland and the effect of multi-layer clouds need to be taken into account when using the Cloud_CCI CTH data products to evaluate models and for other applications. In general, the AVHRR PM Cloud_CCI cloud data for LWP are within the ground-based instrument uncertainties, and Cloud_CCI CTH agrees within 500 m compared to the ground-based lidar–radar observations. There are some other exceptions, in particular for Summit, as discussed in the “Results and discussion” section. In particular, the Cloud_CCI data products at high altitudes in Greenland and multi-layer clouds are associated with larger errors.

We conclude that further studies are needed comparing ground-based and satellite-borne measurements, not only to extend this study in terms of temporal and spatial coverage but also to investigate further the applicability of passive space-based remote sensing instruments for cloud retrievals in the Arctic.