Accuracy assessment of MODIS land aerosol optical thickness algorithms using AERONET measurements over North America

The planned simultaneous availability of visible and near-IR observations from the geostationary platforms of Tropospheric Emissions: Monitoring of Pollution (TEMPO) and Geostationary Operational Environmental Satellites (GOES) 16/17 Advanced Base Imager (ABI) will present the opportunity of deriving an accurate aerosol product taking advantage of both ABI's high spatial resolution in the visible range and TEMPO's sensitivity to aerosol absorption in the near-UV range. Because the wavelengths of ABI are similar to those of the Moderate Resolution Imaging Spectroradiometer (MODIS), existing aerosol algorithms of MODIS can be applied to ABI observations. In this work, we evaluate three distinct aerosol algorithms of MODIS deriving aerosol optical thickness (AOT) over land surfaces using visible and near-IR observations. The Dark Target (DT), Deep Blue (DB), and Multiangle Implementation of Atmospheric Correction (MAIAC) algorithms are all applied to the radiance measurements of MODIS on board the Aqua satellite. We have evaluated each algorithm by comparing the satellite-retrieved AOT to space-time collocated ground-based sun photometer measurements of the same parameter at 171 sites of the Aerosol Robotic Network (AERONET) over North America (NA). A spatiotemporal scheme collocating the satellite retrievals with the ground-based measurements was applied consistently to all three retrieval datasets. We find that the statistical performance of all three algorithms is comparable over darker surfaces over eastern NA with the MAIAC algorithm providing relatively better comparison over western NA sites characterized by inhomogeneous elevation and bright surfaces. The higher spatial resolution of the MAIAC product (1 km) allows a substantially larger number of matchups than DB 10 km and DT 10 km (DT 3 km) products by 115 % and 120 % (86 %), respectively, over eastern NA and by 150 % and 220 % (197 %) over western NA. The characterization of the error in AOT for the three aerosol products as a function of bidirectional surface reflectance derived from both MAIAC and an independent MOD09 atmospheric correction shows a systematic positive bias in DT retrievals over brighter surfaces, whereas DB and MAIAC retrievals show no such bias throughout the wide range of surface brightness, with MAIAC offering the lowest spread in errors. The results reported here represent an objective, unbiased evaluation of existing over-land aerosol retrieval algorithms of MODIS. The detailed statistical evaluation of the performance of each of these three algorithms may be used as guidance in the development of inversion schemes to derive aerosol properties from ABI or other MODIS-like sensors.

1 Introduction

The Tropospheric Emissions: Monitoring of Pollution (TEMPO) mission is NASA's first Earth Venture Instrument (Zoogman et al., 2017). It will be hosted on a still undetermined geostationary satellite with an estimated earliest launch in 2020. TEMPO's hyperspectral observations in the 290–490 and 540–740 nm wavelength ranges (0.6 nm spectral resolution) will measure trace gas concentrations (O₃, NO₂, SO₂, CH₂O, and others) and suspended particle matter (PM). Spatial coverage includes most of Canada, the contiguous United States (CONUS), northern Mexico, and part of the Caribbean at an approximate spatial resolution of 2.1×4.7 km². TEMPO partially fulfills the objectives of the Geostationary Coastal and Air Pollution Events (GEO-CAPE) mission recommended by the National Research Council's Earth Science Decadal Survey to measure tropospheric gases, aerosols, and coastal phytoplankton to monitor air and water quality (Fishman et al., 2012).

Accurate characterization of the tropospheric aerosol load is required as input to a particulate matter (PM) computational scheme along with meteorological information such as temperature and pressure profiles, relative humidity, and planetary boundary layer (PBL) height. The simultaneous availability on GEO platforms of TEMPO and GOES 16/17 Advanced Baseline Imager (ABI) observations in the visible and near-IR ranges presents the opportunity of deriving an accurate aerosol product taking advantage of both ABI's high spatial resolution in the visible and near-IR range, and TEMPO's sensitivity to aerosol absorption in the near-UV range. The combination of 500 m to 2 km spatial resolution and multispectral observations in the visible to shortwave-IR ranges makes the ABI an optimum sensor for the derivation of an aerosol optical thickness (AOT) product over land at the GEO-CAPE required accuracy (Fishman et al., 2012) to be used in conjunction with TEMPO observations for air quality and climate applications.

Satellite-based aerosol remote sensing has been an essential tool to monitor the spatial and temporal distributions of aerosols globally. Significant advancements in aerosol retrieval capabilities over both land and ocean have taken place over the last 20 years. The deployment of the Moderate Resolution Imaging Spectroradiometer (MODIS) and the Multi-angle Imaging Spectroradiometer (MISR) on board the Earth Observing System (EOS) Terra (1999) satellite and a second identical MODIS sensor on the Aqua (2002) platform marked the beginning of a new era in space-based aerosol remote sensing. AOT is routinely derived from MODIS observations by three distinct and independent algorithms: the Dark Target algorithm (Remer et al., 2005; Levy et al., 2007, 2013), the Deep Blue algorithm (Hsu et al., 2004, 2013), and the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm (Lyapustin et al., 2011, 2018).

In this paper, we evaluate the accuracy of the available multi-year long records of AOT products derived from the three MODIS algorithms by a direct comparison to ground-based observations from the Aerosol Robotic Network (AERONET) at multiple sites in the North American area or regard for both ABI and TEMPO field of views. A brief description of MODIS aerosol algorithms, their products, and the satellite–ground collocation procedure is given in Sect. 2. The results of the satellite–ground comparison of individual sites, composites of all sites, and error characterization are presented in Sect. 3, followed by concluding remarks given in Sect. 4.

2 Datasets and collocation strategy

2.1 MODIS Dark Target aerosol product

The Dark Target (DT) algorithm of MODIS consists of two separate algorithms, a land component for the retrieval of aerosol properties over vegetated surfaces, and an over-ocean retrieval algorithm. The over-land DT algorithm exploits the top-of-atmosphere (TOA) reflectance measurements in three MODIS bands, i.e., 470, 670, and 2130 nm, to simultaneously derive AOT at all three channels with an underlying assumption that the impact of fine-mode aerosols on the 2130 nm signal is ignorable, and that the 2130 nm channel contains information about coarse-mode aerosol as well as the surface reflectance. The surface characterization is achieved through linear regression of surface reflectance in the 2130 nm and visible channels (470, 670 nm) (Kaufman et al., 1997; Remer et al., 2005) accounting for the viewing geometry and “greenness” of land cover (Levy et al., 2007). DT attempts to perform a retrieval on each 10 km grid box using a limited number of TOA reflectance observations after discarding the 50 % brightest, 20 % darkest, and cloudy pixels out of a total of 400 pixels at 500 m resolution at nadir. The DT over-land algorithm screens cloudy pixels following a series of tests that rely on using absolute magnitude and spatial variability at 470 nm (500 m resolution) and 1380 nm (1 km resolution), the details of which are given in Martins et al. (2002) and Levy et al. (2013). DT is essentially a look-up table search algorithm which combines the precalculated spectral reflectance of the location- and time-dependent aerosol models comprised of dominant fine and coarse modes with a proper weighting to represent the ambient aerosol properties over the target. The weighted-average spectral look-up table (LUT) reflectance values are compared against the TOA spectral measurements of MODIS to find the best match in AOT yielding a least-square difference between simulated and observed reflectances. Each valid retrieval is assigned with a quality assurance confidence (QAC) flag with the best retrievals tagged as QAC =3. Over land, the expected error for AOT (0.55 µm) with QAC =3 is estimated to be ± (0.05+15 %), whereas that over ocean is ± (0.03+5 %) for retrievals with QAC ≥1. A detailed description of the DT Collection 6 algorithm is given in Levy et al. (2013) and also available online at URL https://darktarget.gsfc.nasa.gov/ (last access: 12 July 2019).

In addition to the 10 km AOT product, the MODIS DT algorithm also offers a higher-resolution aerosol product at 3 km spatial scale. While both aerosol products closely resemble each other, the 3 km product differs from the original 10 km product in the manner in which the MODIS pixels are ingested, organized, and selected by the aerosol algorithm (Remer et al., 2013). The expected error associated with the 3 km aerosol retrievals over land globally is found to be 0.01 to 0.02 higher than that of the 10 km product (Remer et al., 2013).

2.2 MODIS Deep Blue aerosol product

The MODIS Deep Blue (DB) aerosol algorithm utilizes the radiance measurements at the blue wavelength (412 nm), where the surface reflectance over land is relatively lower than that at longer visible wavelengths, to retrieve the column AOT over bright surfaces (Hsu et al., 2004) as well as vegetated areas (Hsu et al., 2013). The surface characterization scheme of DB adopts a hybrid approach that applies the dynamical surface reflectance method for built-up urban areas and the precalculated surface reflectance database in conjunction with the normalized vegetation index in arid and semiarid areas (Hsu et al., 2013). The dynamical surface reflectance method allows larger spatial coverage of the DB aerosol product by expanding the retrieval capability from the bright surfaces to all snow-free land surfaces, including vegetated areas. The surface reflectance dataset used in the DB algorithm is created from the full time series and revised during each reprocessing. The surface dataset is essentially based on the minimum reflectivity approach and binned by scattering angle, season, and normalized difference vegetation index (NDVI) with no time dimension except for the seasonal split. Over vegetated surfaces, DB follows the spectral ratio approach similar to that of DT. The hybrid method scales surface reflectance by regional bidirectional reflectance distribution function (BRDF) shape, based on atmospheric correction near AERONET sites. The enhanced second generation of the DB algorithm identifies mineral dust aerosols based on the brightness temperature difference between infrared channels 8.6 and 11 µm as dust often produces stronger absorption at 8.6 than that at 11 µm, providing a robust way to detect strongly absorbing dust such as the silicates (Hsu et al., 2013). Cloudy pixels are screened by examining the spatial variations in TOA reflectance at 412 nm and 1380 nm and brightness temperatures in the 11 and 12 µm bands. DB performs retrievals on cloud-free and snow-free pixels at nominal 1 km ×1 km spatial resolution, and then aggregates afterward to the 10 km ×10 km retrieval box. Unlike the DT algorithm, DB provides prognostic uncertainty defined relative to DB-retrieved AOT rather than to AERONET AOT. The uncertainty estimates for the best quality retrievals (QAC =3) is formalized as ± ([0.086+0.56τ_DB] /[$\mathrm{1}/{\mathit{\mu }}_{\mathrm{0}}+\mathrm{1}/\mathit{\mu }$]), where τ_DB is AOT retrieved by the DB algorithm, and μ₀ and μ are the cosines of solar and view zenith angles for a given retrieval (Sayer et al., 2013). A detailed description of the second-generation, enhanced DB retrieval algorithm is given in Hsu et al. (2013).

2.3 MODIS Multi-Angle Implementation of Atmospheric Correction aerosol product

The Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm retrieves surface bidirectional reflection factor (BRF) and AOT by using the time series of MODIS measurements over both dark vegetated surfaces as well as bright targets (Lyapustin et al., 2011). The surface characterization in MAIAC is carried out by deriving the spectral regression coefficients that relate the surface BRF in the blue (470 nm), green (550 nm), and shortwave infrared (2130 nm) bands of MODIS. MAIAC considers two discrete aerosol models, i.e., background and dust for a given location, similar to the ones adopted in the MODIS Dark Target algorithm (Levy et al., 2007). However, MAIAC prescribes seven different regional aerosol models for different regions of the world and uses either the background model or dust model, if the dust aerosols are detected. For identifying the smoke aerosols generated from biomass burning, MAIAC employs a “smoke test” to discriminate smoke from clouds (Lyapustin et al., 2012). The smoke test relies on a relative increase in aerosol absorption at MODIS wavelength 412 nm compared to 470–670 nm owing to multiple scattering and enhanced absorption by organic carbon released during biomass burning combustion. Each valid 1 km AOT retrieval of MAIAC is accompanied by the associated quality flags which describe the observed conditions. Since its introduction in 2011–2012, the MAIAC algorithm has been continuously updated and evaluated regarding its accuracy and performance. The MAIAC aerosol dataset used in the present study is derived using the latest Collection 6.0 version of the algorithm documented in Lyapustin et al. (2018), for which the AOT accuracy can be evaluated as ± (0.05+15 %) or even better ± (0.05+10 %) as shown in a global validation analysis. For a more detailed description of the MAIAC Collection 6 algorithm, the reader is referred to Lyapustin et al. (2018).

Table 1 MODIS–AERONET aerosol datasets and their characteristics.

2.4 Ground-based AERONET AOT measurements

The Aerosol Robotic Network (AERONET) project is a ground-based federated network of globally distributed Cimel sun photometers designed to measure aerosol optical and microphysical properties (Holben et al., 1998). Started in 1992, AERONET has expanded its network from a few sites in the early years to more than 500 sites across the globe currently. For more than 25 years, the project has provided a long-term, continuous, and readily accessible public domain database of aerosol optical and microphysical properties. AERONET data have been extensively used for aerosol characterization and validation of satellite retrievals. Spectral AOTs from the direct sun measurements are available nominally at 340, 380, 440, 500, 675, 870, and 1020 nm. In the present analysis, we employ the AERONET Version 3, Level 2 (cloud-cleared and quality-assured) (Giles et al., 2019) spectral AOT dataset from a total of 171 sites spanning the United States and Canada. Figure 1 displays the geographical distribution of AERONET sites with the corresponding temporal record (color-coded). Table 1 summarizes the datasets and their characteristics.

Figure 1 (a) Geographical distribution of AERONET sites over North America. Color codes represent the span of AERONET Version 3 Level 2 data calculated from the daily dataset. (b) An illustration of the spatiotemporal schemes for collocating the satellite retrievals with the ground measurements.

2.5 Satellite-ground collocation strategy

The three MODIS aerosol algorithms report AOT at different spatial resolutions. The DT algorithm performs and reports AOT at 10 and 3 km spatial resolution; DB performs retrievals at 1 km but aggregates afterward to the 10 km ×10 km retrieval box, whereas the MAIAC algorithm retrieves and reports AOT at a much higher resolution of 1 km. While all three aerosol products report AOT at their respective nadir spatial resolutions, i.e., 10 and 3 km for DT, 10 km for DB, and 1 km for MAIAC, representing the atmospheric conditions over the respective area intercepted at the ground, the direct measurements of the spectral AOT from the AERONET sun photometer are columnar point measurements. Furthermore, AERONET makes AOT measurements at an interval of 15 min, and the timings of MODIS Aqua overpass may not closely match with those of AERONET measurements. Therefore, collocating both types of measurements requires a spatiotemporal window that can adequately match the spatially averaged satellite AOT retrievals with the temporally averaged ground-based measurements. The spatiotemporal approach developed by Ichoku et al. (2002) has been adopted in several validation studies for validating MODIS aerosol products against ground truth, such as from AERONET. The standard approach suggests comparing spatially averaged satellite retrievals in a $\mathrm{0.5}{}^{\circ }\times \mathrm{0.5}{}^{\circ }$ grid box centered at the ground site with the temporal averaged sun photometer measurements of AOT within a time window of ±30 min of satellite overpass time.

Table 2 Configurations of four spatiotemporal windows for the collocation of MODIS and AERONET AOT datasets. Acronyms are as follows. DT: Dark Target; DB: Deep Blue; MAIAC: Multi-Angle Implementation of Atmospheric Correction.

In this study, we introduce variations in the standard spatiotemporal window by modifying the extent of both spatial and temporal domains to assess the performance of MODIS aerosol products on different space-time scales. Four different spatiotemporal windows were formulated that differ in the size of grid box centered at the AERONET site and corresponding time window around Aqua overpass time for averaging the AERONET AOTs. For the MAIAC and DB products, the minimum number of 1 km satellite observations used by the respective algorithms in the aerosol retrieval is required to be set at 20 % of the maximum possible 1 km pixels contained in the respective grid boxes. Since the DT algorithm discards 50 % of the brightest and 20 % of the darkest pixels out of a total number of available 500 m pixels in each 10 km and 3 km grid box before performing the retrieval, the threshold for the DT algorithm was set to 10 %. The minimum number of AERONET Level 2 AOTs around the satellite overpass time is required to be at least two for all four variants of the collocation scheme. Table 2 lists the configurations of all four spatiotemporal windows designed for the satellite–ground collocation.

The wavelengths of AOT retrievals differ among the three MODIS aerosol algorithms. While the DT algorithm retrieves and reports AOT at 470, 660, and 2130 nm, DB retrievals are available at 412, 470, and 660 nm. MAIAC retrieves AOT at 470 nm and reports it at 550 nm. For a consistent comparison against AERONET, we choose 470 nm as a reference wavelength since all three algorithms actually retrieve AOT at this common wavelength. The AERONET sun photometer, however, measures AOT at nearby wavelengths, i.e., 440, 500, and 670 nm. Using the Ångström exponent calculated from AOTs at these wavelengths, the AERONET AOT was estimated for the 470 nm wavelength following a linear regression on the AOT versus wavelength relation on a log–log space. The MODIS AOT retrievals at 470 nm were then directly compared against the interpolated AOTs of AERONET at the same wavelength. We use the best quality AOT retrievals as identified in their respective quality assurance fields (i.e., QAC =2 and 3 for DT and DB) of all three aerosol products that are claimed to be higher in confidence and free of cloud contamination.

Table 3 Statistical measures of MODIS–AERONET AOT (470 nm) matchups for sites in eastern North America. Numbers in bold indicate relatively best performance in the respective measures of fit between satellite and ground AOTs. Abbreviations are as follows. Long.: longitude; Lat.: latitude; N: number of satellite–ground matchups; R: correlation; RMSE: root-mean-square error between MODIS and AERONET; Bias: mean bias between the two datasets; Slope and Intercept: slope and intercept of the linear regression between MODIS and AERONET AOT matchups.

3 Results

3.1 MODIS versus AERONET AOTs: individual sites

Figure 2 shows scatter plots of MODIS versus AERONET AOT matchups for the selected individual sites located in eastern NA. These sites are characterized by lower surface albedo during the spring and summer seasons due to increased green cover, and are typically influenced by background and urban–industrial aerosols. Different color codes are used to display matchup points derived following the different collocation approaches described in the previous section. Each AOT dataset was collocated to AERONET independently. While the AOT retrievals from all three algorithms are generally well-correlated (R∼0.90) with those of AERONET, MAIAC AOTs are found to be slightly under-estimated, albeit with the lowest RMSE and the largest number of matchups among the three algorithms. The performance of the DB algorithm is found to be intermediate with relatively better statistics of the comparison than those of DT over sites CCNY, Toronto, and Walker_Branch, but inferior performance over sites GSFC and Univ_Of_Huston. Table 3 lists various statistical measures of MODIS–AERONET AOT matchups for a number of sites located in eastern NA.

Figure 2 Scatterplots comparing the aerosol optical thickness (470 nm) retrieved from the three standard aerosol algorithms of MODIS against that of AERONET for selected sites over eastern, central, and southern parts of North America. Statistical measures of the comparison are depicted within each plot with different color codes denoting matchups obtained following the four spatiotemporal schemes, i.e., black, blue, green, and red for 5, 10, 20, and 40 km grid boxes.

Figure 3 shows similar MODIS versus AERONET comparison, but for a subset of sites over the western NA characterized by bright surfaces and inhomogeneous surface elevation. The retrieved AOT by the three MODIS algorithms differs markedly over these sites. The DT algorithm, which is designed to produce accurate aerosol retrievals over dark surfaces, significantly overestimates AOTs, particularly at a smaller spatial scale of the collocation domain. Noticeably, spatial averaging of DT AOTs over a larger spatial scale (40×40 km²) at the Fresno site provides significantly improved agreement with AERONET AOTs as reflected by the different measures of statistics included in the scatter plot. DB and AERONET AOT matchups over these sites are found to be less correlated but with reduced RMSE. Over the Railroad_Valley site, most AOT matchups from DB under all four collocation approaches remained in the range 0.0–0.2, whereas AERONET AOTs varied in the range 0.0–0.4. The MAIAC–AERONET comparison over these sites shows relatively better statistics than that of DT and DB comparisons with a significantly larger number of matchups, higher correlation coefficient, and lower RMSE values. Various statistical measures of MODIS versus AERONET AOT matchups for selected western NA sites are listed in Table 3.

Figure 3 Same as in Fig. 3 but for AERONET sites located in the western North American region.

3.2 MODIS versus AERONET AOTs: composites for eastern and western North America

This section describes the MODIS–AERONET comparison results obtained by accumulating matchups derived separately for all eastern and western NA sites. Figure 4a shows the composite comparison of MODIS AOTs to those of AERONET for all eastern NA sites combined. The comparison includes matchups obtained following the collocation scheme that averages satellite data in 40×40 km² spatial domain and AERONET data within ±30 min of Aqua overpass time. Satellite–ground matchup points are color-coded according to the density of data for each AOT bin of size 0.01 as depicted in the color bar. One of the striking features of the comparison is that the total number of MAIAC AOT data points collocated with AERONET is significantly larger than that obtained from DB and DT (10 and 3 km) comparisons. Quantitatively, MAIAC provides ∼115 % and ∼120 % (86 %) more matchups than DB 10 km and DT 10 km (3 km aerosol product) products, respectively. In addition to the higher frequency of AOT retrievals, MAIAC AOTs are found to compare better with those of AERONET with an overall lower RMSE (0.056) and a correlation of 0.91. Conversely, the performance of the DT 10 km algorithm is relatively inferior in terms of matchup frequency, larger RMSE, and bias with the slope (1.23) of the satellite–ground relationship higher than unity. DB and MAIAC comparisons to AERONET provide slopes (0.80 and 0.86) less than 1.0 mainly due to underestimation (overestimation) of retrievals at higher (lower) AOTs, but with an overall improvement in the other statistical measures. Noticeably, the DT 3 km product owing to its higher spatial resolution offered more matchups accompanied by similar correlation (∼0.93), slope (1.20), and marginally improved RMSE (∼0.08) compared to those of the 10 km product.

Figure 4 Scatterplots comparing MODIS–AERONET AOT matchups for all sites combined located in eastern (a) and western (b) North America. MODIS–AERONET matchups derived independently without the requirement of having simultaneous measurements. The color codes denote the number density of matchups for each bin of AOT.

For the combined western NA site comparison, MAIAC again provides a significantly larger number of matchup points, quantitatively ∼150 %, ∼220 %, and ∼197 % compared to DB 10 km, DT 10 km, and DT 3 km products, respectively, with the relatively lowest RMSE (0.062) and the highest correlation (0.83). However, the slope of the satellite–ground AOT relationship is found to be the lowest (0.705) for MAIAC results compared to those obtained from DB (0.86), DT 10 km (1.14), and DT 3 km (1.05) datasets. The intercepts of the relationships are found to be comparable for DT and DB (∼0.02), but higher (0.043) for MAIAC.

Table 4 Abbreviations are as follows. Long.: longitude; Lat.: latitude; N: number of satellite–ground matchups; R: correlation; RMSE: root-mean-square error between MODIS and AERONET; Bias: mean bias between the two datasets; Slope and Intercept: slope and intercept of the linear regression between MODIS and AERONET AOT matchups.

The results presented so far considered satellite–ground matchups obtained independently for each MODIS aerosol product. Such comparison allows evaluation of both the relative accuracy of different products as well as the frequency of the retrievals, whereas the comparison imposed by the requirement of having AOT retrievals from all three algorithms simultaneously would provide only the relative accuracy assessment. Such comparison is shown in Fig. 5 for eastern (top) and western (bottom) NA sites. Note that the number of matchups is identical for all three algorithms and is drastically lower than the collocation points obtained when matched with AERONET independently. Given the simultaneous measurements of AOT and near-equal sampling among the three algorithms, MAIAC provides the highest correlation (0.93 and 0.88) and lowest RMSE (0.056 and 0.053) over eastern and western NA sites, respectively. The slope of the satellite–ground relationship, however, was farthest from unity for MAIAC compared to those of DT and DB results.

Figure 5 Scatterplots comparing MODIS–AERONET AOT matchups obtained over all sites located in eastern (a) and western (b) North America. Only those satellite–ground matchups were included for which AOT retrievals and measurements from all four methods are available simultaneously. The color codes denote the number density of matchups for each bin of AOT.

3.3 Impact of surface reflectance on AOT retrievals

The surface characterization is a crucial step for delineating surface contribution from the TOA reflectance measurements to separate atmospheric signal for the aerosol retrieval. Earlier studies suggest that an absolute uncertainty of 0.01 in the estimation of surface reflectance in the visible channels can produce an error of up to 0.1, i.e., approximately 10 times, in the AOT retrieval from satellites (Kaufman et al., 1997; Jethva et al., 2010). The three independent MODIS aerosol algorithms under consideration here employ different approaches to characterize the surface reflectance as briefly described in the data section. The DT algorithm estimates surface reflectance in the visible channels (470 and 660 nm) through a quasi-static regression between the reflectance at 2130 nm and those of visible channels by accounting for the dependence of these relationships on scattering angle and NDVI. The surface characterization in the DB algorithm is achieved through a hybrid scheme that applies the dynamical surface reflectance method for built-up urban areas and the precalculated surface reflectance database in arid and semiarid areas. The MAIAC algorithm, however, derives the spectral regression coefficients dynamically that relate the surface reflectance in the 470, 550, and 2130 nm bands of MODIS.

Figure 6 Difference in AOT (470 nm) between MODIS and AERONET as a function of coincident bidirectional reflectance retrievals (470 nm) from the MODIS MOD09 product for sites in eastern (a) and western (b) North America. Data are represented as a box-and-whisker plot with the thick horizontal line as the median, black dot as mean, shaded boxes covering the 75th and 25th percentiles, and vertical lines as 1.5 times the interquartile range (25–75th percentiles). The number of matchups for each bin is shown at the top of the plot.

In this section, we explore the relationship between the surface reflectance either assumed (DT and DB) or retrieved (MAIAC) and its impact on the accuracy of AOT retrieved from three algorithms. For this purpose, we consider two datasets: (1) the MODIS MYD09 daily L3 Global 0.05Deg CMG atmospheric correction product (Vermote, 2015) and (2) MAIAC BRF retrievals. Both atmospheric correction algorithms differ in their approaches to estimate the surface reflectance by removing scattering and absorption from TOA measurements. Both products dynamically capture the temporal variations in surface properties and provide surface characterization over a wide range of surface conditions, including darker as well as brighter surfaces. The MAIAC BRF product at the time of conducting the present work has not been evaluated over the North American region. However, some recent studies have reported a significant increase in the accuracy of MAIAC surface reflectance compared to MODIS standard products MOD09 and MOD035 over the tropical Amazon (Hilker et al., 2012, 2014, 2015; Maeda et al., 2016). Furthermore, a study by Chen et al. (2017) found an improvement in leaf area index (LAI) retrievals with the MODIS LAI–FPAR algorithms when using MAIAC instead of the standard MODIS MOD09 input. Note that the sole purpose of using the MAIAC surface retrieval dataset here is to evaluate relative differences between satellite retrievals and ground measurements of AOT at varying surface brightness, which in no way constitutes a validation exercise of MAIAC surface retrievals over the study region nor does it act as a bias towards a particular algorithm.

Figure 6 shows box-and-whisker plots of differences in the AOT (470 nm) between the collocated MODIS retrievals and AERONET measurements as a function of coincident MYD09 BRF for eastern NA sites (panel a) and western NA sites (panel b). The collocated dataset of MODIS and AERONET within a 40 km diameter centered at the AERONET site and ±30 min of MODIS overpass was used in these calculations. The total number of samples obtained in each bin of surface BRF is depicted at the top of each subplot. For the eastern NA sites, the mean and mode of error in DT and DB retrievals show negligible dependence on surface BRF with most matchups remaining close to the no-error limit but with an increased spread in data at surface BRF > 0.06. The error in MAIAC AOT retrievals, however, is found to be very small with the mean and mode for each bin close to no error throughout the entire range of MYD09 BRF retrieved over eastern NA. Also, the spread of error (10th to 90th percentile group) in the MAIAC–AERONET matchups is noted to be smaller with an error limit mostly confined to within ±0.1.

For the sites located in western NA, the error in DT-retrieved AOT (both 10 and 3 km) exhibits a systematic behavior showing significant growth of error accompanied by the larger spread in the data population at relatively higher surface BRF (0.05–0.1). Also, note that no sufficient matchups are found between DT and AERONET for conditions when MYD09 retrieved much higher values of surface BRF. Similar results are obtained when MAIAC BRF is used in the analysis shown in Appendix Fig. A1. Both MYD09 and MAIAC BRF datasets, though derived differently, show consistent behavior of errors in AOT as a function of surface reflectance over eastern and western NA regions. Superczynski et al. (2017) further support our findings using the AOT validation results of the Suomi NPP Visible Infrared Imaging Radiometer Suite (VIIRS) aerosol algorithm essentially based on the DT approach, where VIIRS-derived AOTs are found to be biased significantly higher with respect to AERONET measurements over North America at larger values of coincident MAIAC-retrieved surface reflectance. The poor performance of the DT algorithm over brighter surfaces has been a known problem (Levy et al., 2010), although it was expected that the DT collection 006 algorithm would yield a lower bias over bright surfaces (Levy et al., 2013). The DT algorithm was primarily designed and developed for the aerosol retrieval over darker vegetated surfaces, as the name suggests, and follows the principle that aerosols brighten the scene, which over the brighter surfaces breaks down. Moreover, aerosol loading over western NA is relatively low, resulting in an inferior signal from aerosols compared to that from a brighter background.

4 Concluding remarks

In this paper, we have performed the accuracy assessment of three MODIS Aqua products of aerosol optical thickness derived from three independent algorithms using ground-based AERONET measurements over the North American region. This is, to our knowledge, the first attempt to simultaneously evaluate the relative performance of the three MODIS aerosol products, i.e., DT, DB, and MAIAC, over the region, which is in the field of view of the currently operational GOES geostationary platform and future TEMPO mission. A spatiotemporal collocation scheme of satellite retrievals with ground measurements was applied identically to all three satellite-based products, except for the relaxed required minimum number of retrievals for the DT algorithm which discards many sub-kilometer pixels prior to performing the aerosol inversion. The comparison was carried out over a number of AERONET sites situated mostly in the United States, and a few in Canada for the period 2002–2016, and under two sets of configurations, (1) independent comparison against AERONET and (2) when retrievals from all three algorithms are available simultaneously.

We find that the performance of all three aerosol algorithms, when assessed independently without having the requirement of simultaneous retrievals from all three algorithms, is comparable over darker surfaces of eastern NA with the MAIAC algorithm providing marginally better results with the lowest RMSE (0.056) and comparable correlation (∼0.90). Conversely, the DT algorithm yields a larger RMSE (0.095), but offers a better correlation of 0.933; the DB algorithm provided the worst correlation (0.756) with an intermediate RMSE of 0.069. One of the most striking differences noted in this comparison is the number of retrievals, with the MAIAC algorithm yielding significantly more matchups with AERONET – more than double than those of the DB and DT algorithms.

Over the western NA, where the surface is characterized by steep changes in topography and brighter surface background, the AOT retrievals from the DT algorithm are found to be overestimated compared to those from AERONET with poorer RMSE, correlation, and bias of ∼0.12, 0.82, and 0.037, respectively. In comparison, DB and MAIAC both show a relatively robust match with AERONET resulting in an RMSE of ∼0.06 and correlation of 0.72–0.83. Noticeably, the MAIAC dataset provides the maximum number of matchups (N=27 653) compared to those of DB (N=11 026) and DT (N=8623 for 10 km and N=9299 for 3 km) – a factor of 2.51 and 3.21 (2.97) higher matchup frequency than that for DB and DT, respectively.

The error in AOT characterized as a function of MYD09 and MAIAC bidirectional surface reflectance products reflects the ability of DB and MAIAC algorithms to retrieve AOT with practically no bias over a wide range of surface conditions, whereas DT-retrieved AOTs are found to be systematically overestimated at higher values of surface reflectance (> 0.05). The results reported here represent an objective, unbiased evaluation of the DT, DB, and MAIAC land AOT retrieval algorithms currently applied to MODIS observations. The detailed statistical assessment of each of these three algorithms against AERONET may be used as guidance in the development of inversion schemes to derive aerosol properties from ABI or other MODIS-like sensors. An accurate AOT product from GOES-ABI measurements would fulfill the GEO-CAPE-stated need of an aerosol product that can be used for both climate and air quality applications.

Appendix A

Figure A1 Difference in AOT (470 nm) between MODIS and AERONET as a function of coincident bidirectional reflectance retrievals (470 nm) from the MODIS MAIAC product for eastern NA (a) and western NA (b). Data are represented as a box-and-whisker plot with the thick horizontal line as the median, black dot as mean, shaded boxes covering the 75th and 25th percentiles, and vertical lines as 1.5 times the interquartile range (25–75th percentiles). The number of matchups for each bin is shown at the top of the plot.