We present the first extended validation of a new SYNERGY global aerosol product (SY_2_AOD), which is based on synergistic use of data from the Ocean and Land Color Instrument (OLCI) and the Sea and Land Surface Temperature Radiometer (SLSTR) sensors aboard the Copernicus Sentinel-3A (S3A) and Sentinel-3B (S3B) satellites. Validation covers period from 14 January 2020 to 30 September 2021. Several approaches, including statistical analysis, time series analysis, and comparison with similar aerosol products from the other spaceborne sensor, the Moderate Resolution Imaging Spectroradiometer (MODIS), were applied for validation and evaluation of S3A and S3B SY_2 aerosol products, including aerosol optical depth (AOD) provided at different wavelengths, AOD pixel-level uncertainties, fine-mode AOD, and Angström exponent.
Over ocean, the performance of SY_2 AOD (syAOD) retrieved at
550 nm is good: for S3A and S3B, Pearson correlation
coefficients with the Maritime Aerosol Network (MAN) component of the
AErosol RObotic NETwork (AERONET) are 0.88 and 0.85, respectively; 88.6 % and 89.5 % of pixels fit into the MODIS error envelope (EE) of
Over land, correlation coefficients with AERONET AOD (aAOD) are 0.60 and 0.63 for S3A and S3B, respectively; 51.4 % and 57.9 % of pixels fit into MODIS EE. Reduced performance over land is expected since the surface reflectance and angular distribution of scattering are higher and more difficult to predict over land than over ocean. The results are affected by a large number of outliers.
Evaluation of the per-retrieval uncertainty with the
The regional analysis of the Angström exponent, which relates to the
aerosol size distribution, shows spatial correlation with expected sources.
For 40 % of the matchups with AERONET in the Northern Hemisphere (NH) and for 60 % of the matchups in the Southern Hemisphere (SH), which fit into the AE size range of [1, 1.8], an offset between SY_2 AE
(syAE) and AERONET AE (aAE) is within
Good agreement (bias
Differences between the annual and seasonal AOD values from SY_2
and MODIS (mod) Dark Target and Deep Blue products are within 0.02 for the
study area (30
For both S3A and S3B AOD products, validation statistics are often slightly better in the Southern Hemisphere. In general, the performance of S3B is slightly better.
The concern about climate change (e.g. Bergquist and Warshaw, 2019) along with a willingness to reduce its effects (e.g. Leiserowitz et al., 2020; Hoffmann et al., 2022) have been of growing interest during the past decades. Global models introduce different scenarios for climate change (Arbor et al., 2021; Meehl et al., 2007), which are often based on the historical records and trends. Satellite data, including aerosols, provide unique global data on the Earth's surface and atmosphere; they are assimilated into global and regional models (Khaki et al., 2020; Eyre et al., 2022) and used for model evaluation (Gliß et al., 2021).
Product quality depends on instrument specifications and applicability of the retrieval approaches. Despite having an advantage in coverage over ground-based products, satellite products often have lower quality compared with ground-based measurements. However, with the fast development of spaceborne instruments, including improved quality of onboard instruments and increased temporal and spatial coverage (CEOS, 2017; Dubovik et al., 2021), as well as with improved access to satellite products (Borowitz, 2018) following open-access policy (Harris and Bauman, 2015; Olbrich, 2018) and standardisation of satellite data (Loew et al., 2017), the contribution of the spaceborne measurements to climate studies is gradually increasing.
Calibration and validation (cal/val) are essential to characterise the
quality of the performance of a mission
(
Validation is a part of a cal/val activity. In the context of remote sensing, validation refers to the process of quantifying the accuracy of satellite-retrieved products by assessing the uncertainty of the derived products by analytical comparison to reference data, which is presumed to represent the true value of an attribute. Validation shows the maturity of the satellite-derived product and thus provides a conclusion on the mission success. Besides providing information about the product quality, validation may reveal a degradation of the instrument or potential drift (Julien and Sobrino, 2021). Validation results should be used in quality assurance reporting together with product details, calibration characterisation, retrieval algorithm description, and uncertainty characterisation.
Validation is a comparison against in situ measurements, both systematic and from campaigns, and intercomparison against other satellite data sources and/or models. Validation requires reference data with high reliability. Since the performance of a retrieval algorithm may vary in different conditions, validation also requires well-sampled coverage of useful ranges of measured values. Possible uncertainties of the product used as the “truth” must be considered. Since other satellite products and models may have their own biases, the intercomparison against models and other satellite products is called evaluation.
Changes in sensors and algorithms may be revealed if similar validation approaches are employed for different versions of products. Thus, common validation principles and approaches should be followed to allow the intercomparison. General validation is product-specific, while detailed validation is instrument-specific. Validation requires expertise on instrument, processing, and application, as well as a good understanding of limitations; thus, general validation approaches have to be adapted considering specifications of particular products (e.g. temporal, spatial, radiometric resolutions).
An independent verification processing system is important. The purpose of validation is not only to show how good or bad the product is; issues explaining differences between the product and reference data should be identified. Based on validation and evaluation results, recommendations on the product improvements can be provided to the product developers. Recommendations are important as they will help to identify conditions in which an algorithm performance should be improved. Iterations on the product validation results with product developers, such as the round robin approach (Holzer-Popp et al., 2013), are a good example of how communication between the validation team and product developers should be organised to better utilise validation results for improvement of product quality.
In this paper we introduce global validation and evaluation results for the
SYNERGY (SNY) aerosol optical depth (AOD) product, SY_2_AOD (North and Heckel, 2019), for the period from 14 January 2020 to 30 September 2021. The SY_2_AOD product is retrieved from spatially and temporally collocated data
measured with two instruments: the Sea and Land Surface Temperature Radiometer
(SLSTR) and the Ocean and Land Color Instrument (OLCI) aboard Sentinel-3 (S3A
and S3B) satellites. The SYNERGY retrieval algorithm was originally
developed for the retrieval of AOD from the Advanced Along-Track scanning
Radiometer (AATSR) and MEdium-spectral Resolution Imaging Spectrometer
(MERIS) (North et al., 2008) and further developed for the S3 instruments.
The SY_2_AOD product is available from both
S3A and S3B satellites. Extensive and systematic AOD validation against
ground-based measurements and intercomparison with the Moderate Resolution
Imaging Spectroradiometer (MODIS) AOD product were performed in the framework of
the European Space Agency (ESA) Copernicus Space Component Validation
for Land Surface Temperature, Aerosol Optical Depth and Water Vapour
Sentinel-3 Products (LAW,
The paper is structured as follows. The SY_2 retrieval
algorithm and SY_2_AOD product are introduced
in Sect. 2. In Sect. 3 we introduce a validation approach applied in the
current study. An algorithm developed for extracting satellite and
ground-based measurement matchups is explained in Sect. 4. Reference
validation products are introduced in Sect. 5. AOD, AOD uncertainties, fine-mode AOD (FMAOD), fine-mode fraction (FMF), and Angström exponent (AE)
validation results with AERONET are shown in Sect. 6. AOD
OLCI and SLSTR L1b top-of-the-atmosphere (TOA) radiances were utilised in the SYNERGY algorithm for the retrieval of aerosol properties.
The Sentinel-3 OLCI
(
The SLSTR instrument
(
The aim of the SYNERGY aerosol algorithm is to provide global aerosol
optical depth and related aerosol properties for all cloud and ice-free
regions of the Sentinel-3 combined OLCI–SLSTR swaths. The SLSTR
retrieval (ESA climate office, 2022,
The algorithm uses the L1c co-registered OLCI and SLSTR data product as input, projected on the OLCI grid. Co-registration is made based on the common 865 nm radiometric band. Over selected ground-control points, radiometric images of the SLSTR 865 nm band are extracted and compared to the OLCI 865 nm acquisitions. The OLCI image is moved around according to shift vectors and the cross-correlation with the fixed SLSTR window is calculated. The elements of the shift vectors at which a maximum in cross-correlation is reached determine the pixel deregistration between the OLCI and SLSTR reference channel.
Over ocean, AOD is returned using the full swath of the Level 1c (L1c)
product (1400 km), while over land the region covered by both nadir and
oblique view (750 km) is used for the best-quality retrieval, and aerosol
retrieval is also made outside this region where both nadir-only SLSTR
and OLCI are available (
The basis of the algorithm is iterative non-linear optimisation to jointly
retrieve aerosol optical depth at a reference wavelength of 550 nm, referred
to as AOD
A climatology of aerosol composition (Kinne et al., 2013; de Leeuw et al., 2015) is used to provide further information on the fine and coarse components (non-spherical vs. spherical, single-scattering albedo) and a prior estimate of the fine-mode fraction. We fit parameters for both AOD and FMF, which controls the spectral variation of AOD. Although AOD is parameterised by a single nominal wavelength (550 nm), all wavelengths of SLSTR, and additionally the 442.5 nm OLCI channel over land, are used in this fitting. The single-scattering albedo (SSA) is constrained by climatology for the coarse- and fine-mode extremes separately and as a priori information. The retrieval of FMF results in SSA by interpolation between these extremes; however, this should be seen as a potential diagnostic for retrieval performance rather than a user product. Further constraints prevent unfeasible retrieval (e.g. negative AOD or surface reflectance). An estimate of the 1 standard deviation (SD) error in AOD at 550 nm is derived from the second derivative (curvature) of the error surface near the optimal value.
Over ocean, a surface reflectance model gives a reflectance estimate determined from the wind speed and direction and using the models of Cox and Munk (1954) for glint, Monahan and O'Muircheartaigh (1980) and Koepke (1984) for foam fraction and spectral reflectance, and Morel's case I water reflectance model dependent on pigment concentration (Morel, 1988). The ocean inversion uses bands from SLSTR only, using both views to invert if both are available or a single view (either nadir or oblique) if one view is either obscured by cloud, contaminated by glint, or in a swath region where only a single view is present. For land, the reflectance constraint is the result of fitting to separate angular and spectral parameterised models (North, 2002; North et al., 2008; Davies and North, 2015; North and Heckel, 2019). When the oblique SLSTR view is not available, only the spectral constraint is used, allowing AOD estimation over the full L1c swath over both land and ocean.
A final step is used to filter residual cloud contamination or other sources of poor retrieval. This is based on thresholding of local image standard deviation, as discussed in Sogacheva et al. (2017). Over ocean, a final screening is also made on the quality of model fit. Any AOD value outside the AOD valid range of [0, 4] is replaced by a “fill” value of 6.53. A “clean-air” test is performed to recognise cases when an extensive rejection of low AOD values occurs in the case of a clean atmosphere, which often happens over dark surfaces. In the case that this test is positive, which is indicated by quality flags, a value of 0.04 is used.
During post-processing, further aerosol outputs are derived from the
retrieved AOD
Derived aerosol outputs include AOD, AOD uncertainty and single-scattering albedo (each at 440, 550, 670, 865, 1610 nm), aerosol absorption optical depth, fine-mode AOD, dust AOD (each at 550 nm), and the Angström exponent (between 550 and 865 nm). The full list of derived aerosol outputs, which are recorded in gridded NetCDF format at 4.5 km resolution, is shown in Table S1. Additionally for each super-pixel, information is provided giving time and location, solar–view geometry, cloud fraction, AOD retrieval quality flags, and retrieved surface reflectance for each waveband. Quality flags indicate which retrieval method was used, for example nadir-only or dual-view, land–ocean algorithm, and further indicators such as retrieval failure through negative AOD estimation or glint contamination.
The validation approach suggested for the European Space Agency (ESA)
Climate Change Initiative (CCI) AOD product validation (ESA climate office, 2022,
Annual and seasonal validation was performed globally for all data. Furthermore, respective validations were made over selected areas, which represent different surface and aerosol types.
In the NH, the SLSTR oblique scan generally samples backscattered radiance,
which has a weaker aerosol contribution than the corresponding forward-scattering sampled in the SH (e.g.
syAOD
Another specification of the SY_2 AOD product is that the AOD retrieval has been performed with different retrieval approaches, depending on SLSTR and OLCI coverage as well as L1B data availability in different viewing angles (for details, see Sect. 2). The dual-view processor was applied when SLSTR measurements from both views (nadir and oblique) were available. If measurements were available from one view only, the single-view processor was applied to either nadir (over either land or ocean) or oblique view (over ocean or inland waters only). This specification of the product was considered in the current validation exercise.
A matchup is defined as the combination of simultaneous and spatially collocated satellite and ground-based measurements.
Following Ichoku et al. (2002), a macro-pixel of 11
All ground-based measurements were extracted from well-qualified networks introduced in Sect. 5.1 (AERONET), Sect. 5.2 (MAN), and the Supplement (SURFRAD, SKYNET); no additional quality control check has been performed for the reference data. On the contrary, all satellite extractions included all quality flags and contextual parameters present in the Sentinel-3 operational products. Satellite extractions were created automatically for each station, at each overpass, and centred on the station location. They were then associated with relevant ground-based measurements when these data were available and validated.
“Empty” matchups, i.e. when the whole satellite extraction is associated with a fill value for AOD, were not filtered out from the database, except in the case of operational issues with the Sentinel-3 instruments. As these fill values were mainly due to cloud contamination or aerosol retrieval failure, they may provide information about the performance of e.g. cloud screening in the SY_2 algorithm and were therefore relevant to validation objective.
Free access (upon subscription) to this matchup database has been
provided on the ESA LAW web portal (
To explore the performance of different processors, four separate datasets
were created and validated separately. The first dataset (called “all” in
the following) consists of all available data, regardless of which processor
was used. The second dataset (“dual”) contains data retrieved with the dual-view processor. The third (“singleN”) and fourth (“singleO”) datasets are
created using the single-view processors applied to nadir or oblique views,
respectively. The total number of matchups from dual, singleN, and singleO
groups is higher than the total number of all matchups because in the 11
The AERONET is a federation of ground-based remote sensing aerosol networks
(
Ground-based sun photometers directly observe the attenuation of solar
radiation without interference from land surface reflections. They provide
accurate measurements of AOD with uncertainty
For the AOD validation, AERONET version 3 data (Giles et al., 2019) – automated near-real-time quality control algorithm with improved cloud screening for sun photometer aerosol optical depth (AOD) measurements – have been utilised. Version 3 AOD data are computed for three data quality levels: Level 1.0 (unscreened), Level 1.5 (cloud-screened and quality controlled), and Level 2.0 (quality-assured). The Level 2.0 AOD quality-assured dataset is now available within a month after post-field calibration, reducing the lag time from up to several months.
Since AERONET is a network of ground-based sun photometers, and while some of the AERONET stations are in coastal land areas and on islands, the open ocean is poorly covered with AERONET. Thus, another available network (see Sect. 5.2) is used for validation of AOD retrieved over open ocean.
The Maritime Aerosol Network (MAN) component of AERONET provides ship-borne AOD measurements from Microtops II sun photometers (Smirnov et al., 2009). These data provide an alternative to observations from islands and establish validation points for satellite and aerosol transport models. Since 2004, these instruments have been deployed periodically on ships, providing an opportunity for monitoring aerosol properties over the world oceans.
The Microtops II sun photometer is a handheld device specifically designed
to measure columnar optical depth and water vapour content (Morys et al.,
2001). Direct sun measurements are acquired in five spectral channels
within the spectral range 340–1020 nm. The bandwidths of the interference
filters vary from 2–4 nm (UV channels) to 10 nm for visible and
near-infrared channels. The MAN instruments are calibrated against the same
reference instruments as utilised in AERONET. The estimated uncertainty of
the optical depth in each channel does not exceed
Comparison of MAN and AERONET AOD data does not show any particular bias for AERONET and MAN, although a visible cluster of points above the 1 : 1 line was acquired in highly variable dust outbreak conditions west of Africa in the North Atlantic (Smirnov et al., 2011).
The Moderate Resolution Imaging Spectroradiometer (MODIS) was launched aboard
Terra in 1999. It has a wide spectral range from 0.41 to 14.5
In this study, the Level 2 combined Dark Target and Deep Blue (DT&DB) AOD product (MOD04_L2) from MODIS Terra collection C6.1 was utilised, which is characterised by good quality and better coverage than Dark Target or Deep Blue alone (Wei et al., 2019).
The AERONET does not cover the globe evenly. The location of AERONET stations and number of S3A collocations per AERONET station utilised in the validation exercise are shown in Fig. 1. For S3B, the number of matchups is similar (slightly higher).
Location of the AERONET stations and number of matchups with S3A per station (see legend) for the period 14 January 2020 to 30 September 2021.
In the exercise it was found that the validation results for S3A and S3B are, in general, similar (difference between results for S3A and S3B is less than 10 % of S3A AOD). In this paper, validation results for S3A are shown in figures, while validation statistics for both S3A and S3B (shown as S3A/S3B) are summarised in tables and discussed.
AERONET does not provide AOD at 550 nm (this dataset will be referred to in the
following as aAOD
As shown in Fig. 1, AERONET stations are not evenly distributed globally. For the study period, more than 85 % of the matchups were from the NH. Thus, most global results were strongly influenced by the results obtained for the NH. In the case that validation results are similar for the globe and the NH, results for the globe are not visualised. In the case of a significant difference between the results for the globe and the NH, we show figures and discuss results for both. Validation statistics summarised in tables include results for the globe, NH, and SH.
Scatter density plots for S3A SY_2 AOD
Scatter density plots for S3A syAOD
Validation statistics (number of points,
Validation statistics for S3A and S3B products are shown
in Table 1. These include the number of points (
A difference in the algorithm performance in the NH and SH is clear. For
S3A, the fraction of matchups in the EE (70.8 %) and the fraction of
matchups which satisfy GCOS requirements (43.0 %) are considerably
higher in the SH (in the NH, 48.2 % and 20.5 %, respectively), but
In addition to the statistics shown in Table 1, we performed a respective
analysis for limited AOD ranges. For aAOD
Group (dual, singleN, singleO) analysis reveals that most of the low-biased
syAOD outliers were retrieved with the dual processor
(Fig. 2), while most of the high-biased syAOD
outliers were retrieved with the singleN processor. Total bias is smaller
for the dual group globally and in both the NH and SH (Table 1). For aAOD
For S3A, binned in 0.1 aAOD intervals, syAOD offsets (dAOD) for
the globe
Analysis of the binned (based on aAOD, bin size of 0.1) syAOD offsets to
aAOD was carried out. For S3A (Fig. 3), the dual
group shows better performance. In this group, the positive offset at low (
Global difference
For the aAOD binned in 0.1 intervals, the global difference (dAOD) between
syAOD and aAOD represented with the median bias and dAOD standard deviation
is shown in Fig. 4 for all aerosol types including background (aAOD
In the NH, the syAOD offset for the background matchups is
Monthly (January, February, March, etc.), seasonal (DJF, MAM, JJA, SON), and annual (year) variations of the validation results for S3A and S3B syAOD
Validation statistics for syAOD
The correlation coefficient
Bias varies from 0.06 to 0.14 in monthly statistics in the NH. In the SH, bias is lower; it varies from 0.01 to 0.08 in monthly statistics. For S3B, bias is 0.01–0.35 lower than for S3A in all months, except April.
The fraction of matchups in the EE reflects the difference between the NH and SH and between S3A and S3B well. EE is, in general, higher for S3B with the offset up to 15 % in the NH.
As a short summary, syAOD
There are noticeable regional differences in the performance of the retrieval algorithm, which depend on e.g. AOD load and AOD types (composition and optical properties), as well as on the properties of underlying surfaces. Retrieval quality (accuracy, precision, and coverage) varies considerably as a function of these conditions, as well as whether a retrieval is performed over land or over ocean.
Land and ocean regions defined for this study (as in Sogacheva et al., 2020): Europe (Eur), boreal (Bor), northern Asia (AsN), eastern Asia (AsE), western Asia (AsW), Australia (Aus), northern Africa (AfN), southern Africa (AfS), South America (SA), eastern North America (NAE), western North America (NAW), Indonesia (Ind), Atlantic Ocean dust outbreak (AOd), and Atlantic Ocean biomass burning outbreak (AOb). In addition, southeastern China (ChinaSE), which is part of the AsE region marked with a blue frame, is considered separately. Land, ocean, and global AOD was also considered.
Following Sogacheva et al. (2020), we intercompare validation results over 15 regions (as defined in Fig. 6) that seem likely to represent a sufficient variety of aerosol and surface conditions. These and include 11 land regions, two ocean regions, and one heavily mixed region. The land regions represent Europe (denoted by Eur), boreal (Bor), northern, eastern, and western Asia (AsN, AsE, and AsW, respectively), Australia (Aus), northern and southern Africa (AfN and AfS), South America (SA), and eastern and western North America (NAE and NAW). Southeastern China (ChinaSE), which is part of the AsE, is considered separately. The Atlantic Ocean is represented as two ocean regions, one characterised by Saharan dust outflow over the central Atlantic (AOd) and a second that includes burning outflow over the southern Atlantic (AOb). The mixed region over Indonesia (Ind) includes both land and ocean. For exact locations, see Table S2 in the Supplement.
For S3A, syAOD and aAOD scatter density plots for selected regions (as defined in Fig. 6).
High diversity in the validation results was observed between the selected
regions (Fig. 7; Table S2 in the Supplement).
The highest correlation (0.94) was found in AOb region (the number of matchups
is low in this region at 22). For ChinaSE, AsN, AsE, AOd, Aus, and NAE, the
correlation coefficient
Among the land regions, the fraction of the pixels in EE was highest in Aus
(81.6 %) and lowest in Bor and SA (
The fraction of syAOD pixels which satisfy GCOS requirements was low
(
Regional (for Eur, ChinaSE, AfN, AfS, Ind, AOd, SA, NAE)
difference (dAOD
Regional differences between syAOD and aAOD for all aerosol types including
background (aAOD
The syAOD offset analysis was performed for matchups which did not satisfy the
GCOS requirements of
The syAOD relative offset, or dAOD,rel, was defined as in Eq. (1).
In Fig. 9 we show a density scatter plot for the
latitude dependence of the relative offset of the syAOD for all, dual,
singleN, and singleO groups of pixels for S3A. Colour indicates the fraction
of the points with corresponding dAOD,rel from the total number of points
within the 10
For S3A, density scatter plot for the latitude (in degrees) dependence
of the syAOD relative offset for all
In the NH, dAODrel was mostly positive (syAOD was higher than aAOD). In the
SH, dAOD,rel is mostly positive at 30–60
The directional surface reflectance (SR) retrieved with the SYNERGY algorithm is provided in the SY_2_AOD product.
For S3A syAOD matchups with AERONET which do not satisfy GCOS
requirements, scatter density plot for the dependence of the syAOD relative
offset of retrieved surface reflectance for all
In Fig. 10 we show a density scatter plot for the dependence of the relative offset of the AOD on the retrieved SR for the dual, singleN, and singleO groups of matchups. Colour indicates the fraction of the points with corresponding dAOD,rel from the total number of points within the surface reflectance bin.
For all matchups (not shown here), as well as for the dual group (globally,
as well as over the NH and SH), footprints for the dAODrel dependence on the
SR are similar. For SR
In Fig. 11 we show the dependence of the syAOD relative offsets on the OLCI geometry (relative azimuth (Raz), satellite zenith angle (SatZA), and sun (or solar) zenith angle (SunZA) provided in the SY_2_AOD product (North and Heckel, 2019) for the NH and SH. Colour indicates the fraction of the points with a corresponding dAOD,rel interval in the Raz, SatZA, or SunZA bins.
For S3A syAOD matchups with AERONET which do not satisfy GCOS
requirements, the dependence of the AOD relative offsets on relative azimuth
In the NH, positive dAOD,rel increases with Raz increasing from 50 to 80
No significant dependence of dAOD,rel on the SatZA was observed. However, a greater number of negative dAOD,rel values is clearly seen in the SH.
In the NH, dAODrel is slightly positive (0–0.5) in all ranges of SunZA,
except for the most extreme values. For SunZA
Linear fitting for combinations of syAOD
For more details, see Fig. S8 and Table S3, which are both in the Supplement.
Scatter plots for SY_2 AOD
Scatter plots for SY_2 AOD
Validation statistics for all wavelengths are slightly worse for the NH than
global validation statistics (Table S4, Supplement); validation statistics
for the SH are considerably better than for the NH (except for
For the NH (left) and SH (right) for different
wavelengths (top down: 440, 670, 865, 1600 nm), the difference
(dAOD
The syAOD
The concept for validation of the AOD uncertainties applied in the current
study follows the validation strategy suggested by Sayer et al. (2013, 2020)
with consideration of the validation practice further developed in the ESA
Aerosol_cci+ project (Product Validation and Intercomparison Report,
Definitions for uncertainties in the current evaluation of uncertainties are
as follows.
Prognostic (per-retrieval) uncertainties (PUs) for the AOD product are provided at 440, 550, 670, 865, 1600, and 2250 nm wavelengths. Expected discrepancy (ED) is an uncertainty variable which accounts for the PU and the accuracy of the ground-based (AERONET) data (AU), as defined by Sayer et al. (2020) in Eq. (2): AOD error (AODerror) is the difference between a satellite product AOD (syAOD)
and AERONET AOD (aAOD); AOD absolute error (absAODerror) is an absolute
value for AOD error.
Mean bias correction has been performed for the error distributions in some
of the subsequent analysis, since the concept of standard uncertainties
requires bias-free error distributions which can be interpreted as an absence
of remaining systematic and quantifiable biases
(
If wavelength is not specifically mentioned, all variables in Sect. 6.2 refer to the wavelength of 550 nm.
Analysis of the distribution of the uncertainties has been performed for the whole S3A and S3B SY_2_AOD product, as well as for groups of pixels retrieved with different retrieval approaches (dual, singleN, singleO). Results for S3A and S3B are similar; only results for S3A are shown and discussed.
The goodness of the predicted uncertainties was estimated with the
For the whole dataset,
No significant dependence of
Though the number of matchups in the whole dataset is high (which
provides confidence in
For the dataset with the removed outliers,
The influence of
To qualitatively illustrate the accuracy of prognostic uncertainties, we show in Fig. 15 the comparison between the PU, AOD error distribution, and theoretical Gaussian distribution (with a mean of 0 and standard deviation of the syAODerror). PU distribution shows a double peak (the first peak is at ca. 0.02–0.04 for all groups; the second peak is in a range of 0.12–0.18 for different groups). For singleN, two peaks are located close to each other. Mean PU for the dual group is higher; SD is higher for the singleN group. AOD error distributions are Gauss-like with some asymmetry in the positive AOD error direction.
Comparison between PU, AOD error distribution, and theoretical
Gaussian distribution for the whole product
ED is calculated for each pixel by combining PU and AERONET uncertainties, as in Eq. (2).
Histograms of the ED (blue filled bars), AOD errors (red; with
bias correction: green), and ED calculated from uncertainties (purple; scaled
to best fit the mean-bias-corrected error distribution) for all matchups
For a quantitative validation, we follow (with some modifications) a new
approach developed by ESA Aerosol CCI
(
Finally, we calculate an average correction factor for the synthetic distribution (and thus the prognostic uncertainties) in relation to the mean-bias-corrected error distributions as the ratio of the absolute means of both distributions. Correction factors are different for all matchups and for the dual and singleN groups. A small correction is needed for all and singleN (0.80 and 1.1, respectively). For the dual group, the correction is stronger (0.67); ED should be lowered.
However, the correction method applied here does not equally improve ED in
all ranges. The correction factor is biased by the number of pixels with
small (
Sayer et al. (2020) suggested the analysis of the potential of the PU to discriminate between (“good” and “bad”) pixels with likely small or large errors. Instead of PU, we perform analysis of the ED, which, besides PU, includes uncertainties of the ground-based measurements.
Percentile plots of absolute AOD errors at 38 % (black), 68 % (red), and 95 % (blue) as a function of binned expected discrepancy.
To estimate the potential of ED, we plot the absolute errors, which
38 % of all pixels are below, as a function of binned ED
(Fig. 17). We then repeat
this for the fractions 68 % and 95 %. These percentages relate to
0.5
The percentile plots show reasonable agreement (within statistical noise)
with the theoretical lines of 38 % and 68 % for the majority of the
validation points in the lower range of ED (up to 0.05–0.2) for all groups,
with underestimation of the true error at higher values of ED for the 38 % and 68 % lines. For the dual-view case, ED overestimates the true error, while for the single-view case the true error is higher than the ED prediction, especially at higher values of ED (ED
Fine-mode AOD in the SY_2 product (syFMAOD) is provided at
550nm, while AERONET fine-mode AOD (aFMAOD) is provided at 500 nm. As for
aAOD
For S3A and S3B, annual (for the globe, NH, and SH) and seasonal (for the globe) validation statistics for syFMAOD.
Density scatter plots for the relation between syFMAOD and aFMAOD in the NH
and SH are shown in Fig. 18 for S3A; validation statistics are summarised in
Table 2 for both S3A and S3B. The dispersion of
points is higher in the NH. Validation results are considerably better in
the SH:
Density scatter plots for S3A syFMAOD and corresponding aFMAOD
for collocations available over the NH
Looking at the seasonal validation results, for both S3A and S3B, the
correlation coefficient is slightly higher in MAM (0.65 and 0.67 for S3A and S3B,
respectively) and JJA (0.67 and 0.69) and lower (0.56 and 0.59) in DJF
(Table 2; Fig. S10, Supplement). Bias is ca. 0.1–0.12 and slightly higher (0.15 and 0.12) in JJA. The binned mean syFMAOD
values are close to the 1 : 1 line for aFMAOD
Regional (for Eur, ChinaSE, AfN, AfS, Ind, AOd, SA, NAE)
difference (dFMAOD) between syFMAOD and aFMAOD for selected aFMAOD bins:
median bias (circles) and bias standard deviation (error bars) for all AOD
types (purple), aerosol fine-dominated AOD (blue) and coarse-dominated AOD
(green). The fraction (
Among selected regions, offset for all aerosol types is negligible (slightly positive) in Eur, Ind, and NAW (Fig. 19). In ChinaSE and AfN, an offset increases with increasing aFMAOD over 0.5 and becomes more unstable (takes both positive and negative values).
The SY_2 fine-mode fraction (syFMF), which is a fraction of
syFMAOD from the total syAOD, was validated against the AERONET fine-mode
fraction (aFMF). Since syFMAOD is slightly overestimated, we expect that
syFMF is overestimated as well. Density scatter plots for the relation
between syFMF and aFMF in the NH and SH are shown in
Fig. 20 for S3A. In both hemispheres, and thus
globally, syFMF is overestimated in the aFMF range of 0–0.7; a positive offset
of 0.3–0.5 at low (
Density scatter plots for S3A syFMF and corresponding aFM for
collocations available over the NH
Density scatter plot for the difference (dFMF) between syFMF and
aFMF as a function of aAOD
A scatter density plot between dFMF (which is defined as the difference between
syFMF and aFMF) and aAOD is shown in Fig. 21 for
the NH and SH. In general, offset is higher at low AOD and decreases towards
high AOD. The fraction of high (
Regional (for Eur, ChinaSE, AfN, AfS, Ind, AOd, SA, NAE)
difference (dFMF) between syFMF and aFMF for selected aFMF bins: median bias
(circles) and bias standard deviation (error bars) for all AOD types
(purple), as well as aerosol fine-dominated AOD (blue) and coarse-dominated AOD (green). The fraction (
Regional dFMF (Fig. 22) is positive (0.3–0.7) for
low (
The Ångström exponent, AE, is often used as a qualitative indicator
of aerosol particle size. SYNERGY AE (syAE) is calculated in the spectral
interval 550–865 nm, while AERONET AE (aAE) is provided for 500–870 nm. The difference between AE
Scatter plots between syAE
Scatter plots between syAE
For 40 % of the matchups with AERONET in the NH and for 60 % of the
matchups in the SH, which fit into the aAE interval of [1, 1.8], an offset
between syAE and aAE is within
For the whole global product, correlation coefficients between
syAE
Regional scatter density plots between syAE
Regional analysis (Fig. 24, Table S6) reveals
considerable differences in syAE evaluation results for regions with
different surface type and aerosol properties. Footprints for the frequency
of matchups at certain AE ranges (density value on the scatter plot) follow
the “cloudy” shape in regional scatter density plots. The location of the
clouds along the
The syAE is often overestimated in the aAE range [1.3, 1.7], except for AsW,
for which the fraction of good (close to the 1 : 1 line) pixels is as high as the fraction of overestimated syAOD. In AfN, low AE, which is typical for that
region characterised by a high fraction of dust particles, is often highly
overestimated. A dense cloud of good matchups is located near the 1 : 1 line in NAW. However,
Being performed aboard ships, MAN AOD measurements are irregular. S3A and S3B collocations with MAN for the period January 2020–September 2021 are shown in Fig. 25. Altogether, 105 matchups have been found for S3A and 95 matchups for S3B. Note that about half of the collocations are observed near coastal zones. Since the number of validation points is low, we show in Fig. 26 scatter plots and validation statistics for both S3A and S3B.
Collocations of S3A
Scatter plots between S3A and S3B syAOD as well as MAN AOD (mAOD) with validation statistics.
Results for both instruments confirm a good performance of the retrieval algorithm over ocean. For S3A and S3B, correlation coefficients are 0.88 and 0.85, and fractions of pixels in the EE are 88.6 % and 89.5 %. An offset with MAN AOD (mAOD) is slightly higher for S3A (0.02 and 0.01), while rms is slightly higher for S3B (0.06 and 0.1).
One value from each product, S3A and S3B, can be considered a clear outlier: S3A AOD over the Baltic is underestimated, and S3B AOD over the Caribbean Sea is overestimated. The removal of these outliers from the validation exercise improves validation statistics: correlation increases to 0.95 and 0.97, rms decreases to 0.04 and 0.03, and fractions of pixels in the EE increase to 89.4 % and 92.4 % for S3A and S3B, respectively.
The coverage of ground-based reference data is limited. To better evaluate the
spatial distribution of the satellite-retrieved AOD, an intercomparison
with other satellite products is necessary. The satellite product chosen as
a “reference” must fulfil several criteria, e.g.
overpass time as close as possible to Sentinel-3 to avoid possible
different aerosol and cloud conditions; wider swath (for the reference product), which allows considering most
of the pixels from the tested product in the analysis; similar resolution, which allows pixel-to-pixel intercomparison.
Considering these criteria, the MODIS Terra DT&DB AOD product has been
chosen as a reference for evaluation of the SY_2 AOD
The MODIS Terra DT&DB AOD product fulfils two out of the three criteria mentioned
above.
The Sentinel-3 orbit is a near-polar sun-synchronous orbit with a
descending node equatorial crossing at 10:00 mean local solar time. The
MODIS Terra satellite crosses the Equator on descending passes at 10:30–10:45 mean local solar time. The SLSTR dual-view swath centred on the sub-satellite track is 740 km wide, with a single-view swath width of 1470 km. OLCI covers a swath width of 1270 km. MODIS Terra has a viewing swath width of 2330 km.
The third criterion is not fulfilled since MODIS and SY AOD products are
provided at different resolutions. The resolution of the SY_2
product is 4.5
Two different approaches exist for evaluation and intercomparison of satellite monthly AOD. For algorithm performance intercomparison, only the spatio-temporally collocated pixels from the two products were considered (used in monthly aggregates). For climate studies (for e.g. model evaluation, trend analysis) for which existing monthly products are utilised, an intercomparison should be performed for the products built on all points available for each instrument.
For 26 February 2020, all pixels available in S3A
syAOD
Same as Fig. 27 for syFMF; all pixels available in S3A FMF
SY_2 and MODIS Terra AOD products were intercompared over the area shown in Figs. 27 and 28. To evaluate and intercompare AOD products (and thus algorithm performance) in different environments (e.g. surface type, aerosol type, aerosol loading), subregions shown in Fig. 29 (top right) were chosen (see Table S7 for details).
Density scatter plots for MODIS Terra and S3A SY_2_AOD L3 daily collocated products for 2020 for the subregions shown in the top right corner. Statistics are summarised in the Supplement (Table S9).
All pixels available in S3A SY_2_AOD and MODIS Terra L3 daily AOD550 products, collocated products, and differences between collocated products are shown for the whole AOI for 26 February 2020 (Fig. 27). Because of the wider swath, MODIS has larger coverage than S3A. Thus, when collocating two products for closer intercomparison, more pixels from the MODIS product are removed.
For the products containing all original pixels for each instrument, the SY_2 AOD mean over the AOI is higher than MODIS Terra AOD (0.35 and 0.21 for S3A and MODIS, respectively). Mean AOD over land and over ocean are also higher for S3A. For collocated products, mean (over the AOI) AOD for S3A and MODIS as well as AOD over ocean come very close to each other. However, SY_2 FMF (syFMF) over ocean (Fig. 28) is lower than MODIS FMF (modFMF). Also, there are regional differences mainly over a possible dust overflow over the Atlantic. MODIS provides higher AOD over the dust plume. Lower modAOD on the west of the plume may be explained by the offset between MODIS Terra and S3A overpass time. Over land, mean AOD is slightly lower for S3A for collocated pixels, and modFMF over bright surface (Sahara) is missing; over other regions the difference between syFMF and modFMF is lower compared to ocean.
For the chosen day, for S3A, a sharp transition between AOD retrieved over
land and ocean at the west coast of Africa is revealed. This feature is
clearly seen in the S3A and MODIS AOD difference plot. This can be explained
by the land–surface gradient in the syFMF (Fig. 28). A large AOD gradient in S3A data is observed over Nigeria; the
inconsistency with MODIS data reaches above
For the whole year of 2020, S3A SY_2 and MODIS AOD
In the Europe region, which includes parts of eastern and southern Europe and
the Middle East, AOD is low (
In the desert area the disagreement between the two products is most
significant. For MODIS AOD in the range 0–0.8 most of the SY_2 pixels have AOD
In the coast
The footprints for SY_2 and MODIS AOD look similar in the two areas with a seasonal contribution of biomass burning aerosols (Africa,BB and S.America,BB). Agreement between SY_2 and MODIS is good for MODIS AOD below 1.2. Above that threshold, SY_2 AOD is on average lower.
Overall, the majority of data are in the low AOD range, in which agreement is decent (with SY_2 slightly high biased), but at higher AOD there is much more variance (partly due to the scarcity of data) and in general a slight low bias for SY_2.
Seasonal comparison is shown in Fig. S13 in the Supplement. Annual and seasonal statistics for SY_2 and MODIS Terra for all daily pixel AOD intercomparisons are summarised in the Supplement (Table S8).
Two types of monthly datasets have been created from SY_2_AOD and MODIS Terra daily data to study the differences at the monthly, seasonal, and annual (MSA) level.
In the first monthly dataset, all pixels available in the SY_2_AOD and MODIS Terra daily products have been used to build a monthly aggregate for each instrument. Intercomparison of these “all-pixel” monthly aggregates (which are similar to the official monthly products provided for users) is important because it will help in e.g. understanding the difference in climate data records built from the provided monthly AOD products which include all available data.
A second monthly dataset, the “collocated” product, has been aggregated using only collocated daily pixels. Intercomparison of collocated monthly aggregates shows the difference in monthly AOD based on differences in retrieval approaches.
For the year 2020, annual S3A SY_2_AOD
Seasonal (top down: DJF, MAM, JJA, SON) S3A (left panel), MODIS
Terra (middle panel) AOD
Annual AOD from all-pixel and collocated monthly datasets for SY_2_AOD and MODIS Terra, respectively, as well as the corresponding differences are shown in Fig. 30. Seasonal plots for collocated aggregates and the difference between them are shown in Fig. 31. Statistics for difference plots (area, land, and ocean means) have been calculated from pixel-to-pixel difference, but not as the difference between the AOD averaged over AOI, land, and ocean.
Differences between SY_2_AOD and MODIS Terra MSA AOD exist in both the all-pixel and collocated datasets. For both datasets, SY_2 AOD averaged over AOI is higher for the whole area, as well as for land and ocean. The difference is smoother for the all-pixel datasets. Even though difference plots show that regional offset between the two datasets is often within GCOS requirements for AOD quality (0.03) over ocean (SY_2 AOD is in general lower) and the whole AOI, the difference in AOD over land is often higher (up to 0.11 as averaged over AOI in DJF, all-pixel dataset).
Regional differences in seasonal AOD from the collocated dataset are
considerably higher (Fig. 31). For all land
subregions (except for “desert”, JJA), S3A AOD is higher than MODIS AOD.
The offset is highest for the “coast
For the open-ocean regions (“ocean, clean”, and “ocean
We have presented the first validation of a new SYNERGY global aerosol product, derived from the data from the OLCI and SLSTR sensors aboard the Sentinel-3A and Sentinel-3B satellites. Combined, the two satellites provide close to daily global coverage and provide aerosol measurements with a latency of 2–3 d. In this study we have compared the aerosol product with ground-based photometer data from four networks: AERONET, SKYNET, SURFRAD, and MAN, as well as with MODIS combined Dark Target and Deep Blue algorithms. The aim of this study was to provide global characterisation of the current aerosol retrieval and to guide future algorithm development.
Over ocean, the performance of SYNERGY-retrieved AOD is good and consistent
with the reference MAN dataset (rms
Over land, overall performance has a much higher rms error of approximately
0.25 when compared to AERONET. Overall AERONET correlation is
It is clear that retrievals using dual view give higher quality by making use
of more information to allow less reliance on surface spectral assumptions.
Retrieval over land surface in the Northern Hemisphere shows generally
higher retrieval error, including regions of boreal forest where we would
expect higher-quality retrieval due to the low surface signal. In some
cases, this will be due to weak masking of snow and ice cover as well as the
presence of retrievals made at high solar zenith angles (over 70
The retrieval of the Angström exponent, related to aerosol size distribution, shows spatial correlation with expected sources but generally overestimates AE for cases in which AERONET Angström is low, resulting in overall high bias. This is dependent on the retrieval of the fine-mode fraction in the algorithm, which needs to be investigated further and improved. Evaluation of the per-retrieval uncertainty indicated good correlation with measured error distributions, with overprediction of expected error in the dual-view case and underprediction in the single-view case. Evaluation of the uncertainty propagation is difficult in the presence of outliers which do not fit the algorithm assumptions, for which we see a tail of higher errors, for example related to undetected cloud in the input data.
The SY_2_AOD product is available upon subscription at
The supplement related to this article is available online at:
CH, LS, and SD created the original research framework and provided research direction. MD and CH established a database. LS developed a validation strategy, wrote the software, and performed the analysis. PK co-wrote the software. LS, THV, PN, CH, SS, and SD co-wrote the paper.
The contact author has declared that none of the authors has any competing interests.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This research has been performed in the framework of the ESA/Copernicus LAW and OPT-MPC projects supported by the EU Copernicus programme (grant nos. 4000129877/20/I-BG and 4000136252/21/I-Bgi, respectively).
This research has been supported by the EU Copernicus program (project LAW, grant no. 4000129877/20/I-BG; project OPT-MPC, grant no. 4000136252/21/I-Bgi) and ESA.
This paper was edited by Alexander Kokhanovsky and reviewed by Stefan Kinne and two anonymous referees.