Use of machine learning and principal component analysis to retrieve nitrogen dioxide (NO2) with hyperspectral imagers and reduce noise in spectral fitting
Nitrogen dioxide (NO2) is an important trace-gas pollutant and climate agent whose presence also leads to spectral interference in ocean color retrievals. NO2 column densities have been retrieved with satellite UV–Vis spectrometers such as the Ozone Monitoring Instrument (OMI) and the Tropospheric Monitoring Instrument (TROPOMI) that typically have spectral resolutions of the order of 0.5 nm or better and spatial footprints as small as 3.6 km × 5.6 km. These NO2 observations are used to estimate emissions, monitor pollution trends, and study effects on human health. Here, we investigate whether it is possible to retrieve NO2 amounts with lower-spectral-resolution hyperspectral imagers such as the Ocean Color Instrument (OCI) that will fly on the Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) satellite set for launch in early 2024. OCI will have a spectral resolution of 5 nm and a spatial resolution of ∼ 1 km with global coverage in 1–2 d. At this spectral resolution, small-scale spectral structure from NO2 absorption is still present. We use real spectra from the OMI to simulate OCI spectra that are in turn used to estimate NO2 slant column densities (SCDs) with an artificial neural network (NN) trained on target OMI retrievals. While we obtain good results with no noise added to the OCI simulated spectra, we find that the expected instrumental noise substantially degrades the OCI NO2 retrievals. Nevertheless, the NO2 information from OCI may be of value for ocean color retrievals. OCI retrievals can also be temporally averaged over timescales of the order of months to reduce noise and provide higher-spatial-resolution maps that may be useful for downscaling lower-spatial-resolution data provided by instruments such as OMI and TROPOMI; this downscaling could potentially enable higher-resolution emissions estimates and be useful for other applications. In addition, we show that NNs that use coefficients of leading modes of a principal component analysis of radiance spectra as inputs appear to enable noise reduction in NO2 retrievals. Once trained, NNs can also substantially speed up NO2 spectral fitting algorithms as applied to OMI, TROPOMI, and similar instruments that are flying or will soon fly in geostationary orbit.
Nitrogen dioxide (NO2) is an important trace gas for both air quality and climate. It is identified as a criteria pollutant by the United States (US) Environmental Protection Agency (EPA). As a climate agent, it is a precursor for tropospheric ozone, a potent greenhouse gas in the upper troposphere. NO2 also contributes to the formation of aerosols that can cool the planet by reflecting incoming solar radiation back to space (Shindell et al., 2009). Over non-polluted regions, most of the atmospheric column of NO2 resides in the stratosphere, where it participates in photochemical reactions that can affect the ozone layer (see, e.g., van Geffen et al., 2020, and references therein).
Much effort has been expended to develop sophisticated physically based retrieval algorithms for spectrometers that measure scattered solar radiation at ultraviolet (UV) and blue wavelengths at the ground (e.g., Noxon, 1975; Platt and Perner, 1983; Platt, 1994; Platt and Stutz, 2008) as well as from satellite platforms (e.g., Burrows et al., 1999; Richter and Burrows, 2002; Bucsela et al., 2006; Boersma et al., 2007, 2011; Valks et al., 2011; Bucsela et al., 2013; Yang et al., 2014; Marchenko et al., 2015; Boersma et al., 2018; van Geffen et al., 2020; Lamsal et al., 2021). Retrievals from satellite-based instruments such as the Global Ozone Monitoring Experiment (GOME) (Burrows et al., 1999), SCIAMACHY (Bovensmann et al., 1999), the Ozone Monitoring Experiment (OMI) (Levelt et al., 2006), GOME-2 (Munro et al., 2016), the Ozone Mapping and Profiler Suite Nadir Mapper (OMPS-NM) (Bak et al., 2017), and the TROPOspheric Monitoring Experiment (TROPOMI) (Veefkind et al., 2012) have been used in numerous studies related to top-down emissions estimates, air quality monitoring and forecasting, pollution events, trends, and related health studies (see, e.g., Bovensmann et al., 2011; Lamsal et al., 2015; Krotkov et al., 2016; Duncan et al., 2016; Levelt et al., 2018; Goldberg et al., 2021; Kerr et al., 2021; Cooper et al., 2022, and references therein).
NO2 absorption impacts satellite radiance measurements that are used in ocean color algorithms. The affected spectral ranges include those used for retrievals of colored dissolved organic matter (CDOM) and chlorophyll a (e.g., Mannino et al., 2008; Le and Hu, 2013). In particular, large variations in NO2 total columns near polluted coastlines can affect ocean color measurements from sensors in both low-Earth orbit (LEO) and geostationary Earth orbit (GEO) (Ahmad et al., 2007; Tzortziou et al., 2014). For example, under high NO2 loading (∼ 1 × 1016 molec. cm−2), if not accounted for, errors in water-leaving radiance could reach 10 %–20 % (Ahmad et al., 2007) and produce spectral structure that can interfere with ocean color retrievals.
Two planned hyperspectral imagers, the Ocean Color Instrument (OCI) on the Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission in LEO and the GEO Geosynchronous Littoral Imaging and Monitoring Radiometer (GLIMR) were designed for ocean color measurements. The PACE OCI is scheduled to launch in the early 2024 time frame (Werdell et al., 2019), and GLIMR will make diurnal measurements over the Gulf of Mexico and surrounding coastlines later this decade (NASA, 2019). There are several options for atmospheric NO2 correction algorithms for ocean property retrievals: (1) use of a satellite-based NO2 climatology; (2) use of co-located satellite data from atmospheric instruments, such as TROPOMI; (3) use of simulated NO2 data from a chemistry transport model. However, all of these have shortcomings. For example, NO2 derived from atmospheric instruments may not be available at the spatial or temporal scales of ocean color instruments. While a climatology or model simulations can capture the basic features of NO2, including high values around coastal cities, they may miss important details of pollution plumes that can extend over the ocean. It should also be noted that while TROPOMI and PACE will be in similar orbits (the respective TROPOMI and PACE Equator crossing times are 13:30 and 13:00 LST, local solar time), TROPOMI will be nearing its end of primary mission by the time PACE launches; its follow-on mission, Sentinel-5, will be in a morning orbit (09:30 LST Equator crossing time) that will result in more temporal mismatch with PACE. Therefore, it has been a stated and challenging goal of these ocean color missions to quantify NO2 spatiotemporal variations with their lower-spectral-resolution measurements. NO2 retrievals have been demonstrated with hyperspectral sensors on aircraft (Tack et al., 2017; Kuhlmann et al., 2022) and the Russian Resurs-P satellite (Postylyakov et al., 2017, 2019; Zakharova et al., 2021); these sensors have somewhat higher spectral resolution than PACE and GLIMR.
This paper attempts to answer two questions: (1) Can NO2 slant columns be accurately estimated with planned ocean sensors such as PACE OCI and GLIMR using a machine learning algorithm?; (2) Can the machine learning techniques developed for retrieving NO2 from the ocean color instruments be harnessed to improve existing algorithms applied to atmospheric instruments, both in terms of quality and efficiency? If the answer to question 1 is affirmative and retrievals from PACE OCI and GLIMR are successful, they could provide improved spatial resolution of NO2 total column amounts compared with existing instruments, a benefit for land and ocean retrievals as well as for atmospheric science. Regarding question 2, state-of-the-art spectral fitting algorithms can be computationally burdensome. Algorithm efficiency is particularly important for current and future atmospheric composition instruments, including those in geostationary orbit, that have very large data volumes. Machine learning has been shown to be an efficient means of estimating NO2 vertical columns from satellite spectra (Li et al., 2022) as well as for other applications in remote sensing and geoscience (Maxwell et al., 2018; Lary et al., 2016). In addition, machine learning combined with principal component analysis may be able to reduce noise in the spectral fitting compared with the more traditional approaches.
2.1 OMI, PACE, and GLIMR satellite instrument characteristics
Table 1 gives a summary of the approximate relevant characteristics of the satellite instruments used in this study and other relevant sensors used for NO2 retrievals. OMI is a push-broom spectrometer that measures backscattered sunlight and solar irradiance (Levelt et al., 2006, 2018). There are three separate detectors on OMI. We ran experiments with Level 1B (L1B) Global Geolocated Earthshine Radiances from the OMI collection 3 for the visible (Vis) (Dobber, 2007b) and UV-2 detectors (Dobber, 2007a) that cover wavelengths from 349 to 504 and from 307 to 383 nm, respectively. OMI employs a two-dimensional (2D) charged-coupled device (CCD) that provides spectral information in one dimension and spatial information in the other. This results in 60 rows of spectra in the cross-track direction. Spacecraft motion provides observations along the satellite swath. Therefore, each cross-track row of OMI can be considered as a distinct instrument with its own characteristics (e.g., wavelengths, response functions, and calibration) and biases. The wavelengths vary slightly across the swath resulting in a so-called spectral smile. We use data from early in the mission (2005) for this study. Later in the mission, some of the rows were affected by an anomaly presumably outside the instrument that caused blockage and scattering of light into some of OMI's rows (Levelt et al., 2018) resulting in a decrease in spatial coverage. OMI's spatial resolution is approximately 13 km in the along-track direction by 24 km in the cross-track direction at nadir with larger pixels towards the swath edges. The total swath width is ∼ 2600 km. The TROPOMI instrument has similar spectral characteristics for NO2 retrievals but with higher spatial resolution. PACE OCI will cover wavelengths from UV (∼ 340 nm) through the shortwave infrared wavelengths. It will provide daily global coverage from LEO at a spatial resolution of approximately 1 km (Werdell et al., 2019). GLIMR will have a spatial resolution of about 300 m.
Figure 1a shows the specified signal-to-noise ratio (SNR) for GLIMR provided by the instrument team (Antonio Mannino, personal communication, 2022); we assume these values are independent of radiance value. Figure 1b shows an SNR model for PACE OCI based on measurements, where the SNR varies with radiance, also provided by the instrument team (Brain Cairns, Bryan Franz, Gerhard Meister, and Shihyan Lee, personal communication, 2022). Typical radiance distributions for different wavelengths are shown in Appendix A. Note that, for the determination of NO2 tropospheric vertical columns, cloudy observations are used to help estimate stratospheric column amounts, so that the full range of radiance values is needed, not just clear-sky observations (Bucsela et al., 2013).
2.2 NO2 absorption cross sections and differential optical absorption spectroscopy (DOAS) retrievals
NO2 absorption covers a broad range of wavelengths from the UV to the near-infrared (NIR) with a peak near 400 nm. Figure 2 shows NO2 absorption cross sections from the UV through to the red wavelengths (Vandaele et al., 1998), where the blue and green curves have been convolved and resampled to approximate effective cross sections for OMI and OMPS, respectively. The red and black curves show the NO2 cross sections convolved with boxcar functions of widths 5 and 10 nm and two samples per box, similar to the expected spectral resolution and sampling of OCI and GLIMR, respectively. Particularly at the OCI spectral resolution, there is still marked high-frequency structure throughout the visible wavelength range.
Most NO2 spectral fitting algorithms are based on the differential optical absorption spectroscopy (DOAS) methodology (e.g., Platt and Stutz, 2008). In a DOAS-type spectral fit, the retrieved quantity is the slant column density (SCD) of a weakly absorbing trace gas, defined as the integrated number of molecules per unit area to produce an observed amount of absorption at a particular wavelength along the total atmospheric photon path. DOAS generally works by fitting appropriately convolved absorption cross sections (effective cross sections) to the logarithm of Sun-normalized radiance spectra for a given fitting window. A DOAS slant column retrieval for NO2 involves fitting the high-frequency structure in the NO2 absorption cross sections generally in the range of 400–497 nm (see Fig. 2) while accounting for the low-frequency envelope of NO2 absorption, e.g., using a polynomial function (e.g., Richter and Burrows, 2002; Lerot et al., 2010; Richter et al., 2011; Marchenko et al., 2015; van Geffen et al., 2015, 2020). At the GLIMR spectral resolution, a retrieval would likely need to make use of the broad NO2 absorption feature peaking at around 400 nm rather than the finer spectral features used in DOAS retrievals. While this type of approach has been achieved for ozone (Fleig et al., 1986) whose stratospheric column amount is typically quite large, it has not be demonstrated for gases with weaker absorption (owing in part to lower column amounts) such as NO2.
Within any type of spectral fit, other known absorbers or pseudo-absorbers, such as rotational-Raman scattering (also known as the Ring effect), should be accounted for. In typical NO2 fitting windows, the interfering absorbers include ozone (O3), water vapor (H2O), the oxygen dimer (O2−O2), and glyoxal (CHO−CHO). The spectral signature of the Earth's surface must also be accounted for along with other instrumental effects such as spectral alignment of the radiance spectra and proper characterization of the instrument response function.
A secondary retrieval step involves estimation of the vertical column density (VCD) of NO2 using the SCD, Sun–satellite geometry, and information about clouds, aerosols, the Earth's surface, and the NO2 profile shape. The SCD and VCD are related through the concept of an air mass factor (AMF), i.e., VCD = SCD AMF. Both the SCD and AMF are formally wavelength dependent, so that typical DOAS retrievals using a range of wavelengths to fit a single value SCD or compute an AMF refer to an average over the fitting window. Richter et al. (2014) have accounted for this wavelength dependence, but this type of approach is typically not used in operational algorithms. A direct VCD spectral fitting algorithm was also applied to UV wavelengths to retrieve NO2 from the OMPS-NM (Yang et al., 2014).
2.3 Data flow and retrieval
Figure 3 shows a flow diagram of the data processing that we use here to train and evaluate results from a neural network (NN) that predicts NO2 SCDs using simulated data from imagers, such as OCI and GLIMR, with lower spectral resolution compared with spectrometers, such as OMI and TROPOMI, designed for atmospheric measurements. We simulate OCI and GLIMR observations by reducing noise and then spectrally averaging and resampling OMI radiances and adding noise according to instrument specifications and measurements (see Fig. 1). A machine learning approach is then employed to estimate the OMI-derived NO2 slant columns with simulated data from the lower-spectral-resolution instruments. As explained in more detail below, we focus on NO2 SCD retrievals. The conversion of SCD to total or tropospheric VCD can be accomplished in a straightforward and computationally efficient manner as in current algorithms and is not addressed further here. Information about the surface, aerosol, and clouds, such as cloud/aerosol radiance fraction and effective pressure (e.g., from the O2 A band), needed for the calculation of the AMF, will be available from OCI itself (Werdell et al., 2019). Details of the input features, target outputs, and architecture of the machine learning algorithm are provided in the following subsections.
2.3.1 OMI NO2 data
We use the destriped SCDs (parameter name SlantColumnAmountNO2Destriped) from the collection 3 version 4.0 OMNO2 NO2 product (Lamsal et al., 2021; Krotkov et al., 2019) as the NN training target. The NO2 spectral fitting algorithm is based on Marchenko et al. (2015), who use an iterative approach in which the 402–465 nm range is broken up into seven smaller overlapping micro-windows. This method leads to flexible determination of wavelength-dependent shifts between radiance and irradiance spectra as well as the rotational Raman scattering or so-called Ring spectrum. The overall OMI standard NO2 product has undergone substantial changes over the years after being evaluated in numerous studies with respect to other model-, ground-, air-, and satellite-based data sets (e.g., Lamsal et al., 2014; Choi et al., 2020, and references therein).
2.3.2 Simulated OCI and GLIMR data
We start with the OMI Level 1B (L1B) radiances. Unlike in standard OMI retrieval algorithms, we do not perform any normalization with respect to either an observed or reconstructed (laboratory) solar reference spectrum. While this normalization could be done and there may be advantages, such as for processing long time series in which instrument degradation occurs, here we elected to keep the approach as simple as possible.
The next step is to optionally reduce noise in the OMI data using a principal component analysis (PCA) approach, where spectra are reconstructed using coefficients of leading principal components (PCs) constructed from a large sample of data. Here, as in standard DOAS fits, we use the natural logarithm of the radiances, although without normalization, as discussed above. The goal of the noise reduction is to provide a relatively clean, although not necessarily perfect, set of spectra to simulate data for different instrument configurations. Our aim is to produce a realistic simulation of satellite observations. While we could have used simulated spectra instead of OMI data, it would be difficult to capture all of the complexities present in real satellite-observed spectra, including instrumental artifacts, and other complex interactions involving scattering and absorption in the atmosphere and at the surface.
The data sample used in the initial PCA starts with all observations from 1 d in every month (the 15th day) in 2005 supplemented with additional days in winter when the NO2 lifetime is long and pollution can build up in the boundary layer to give very high SCDs. The added high-pollution days are 29 January through 4 February of 2005. The same data sample is used for training and evaluating the NN models, as described below. Use of a large sample spanning different seasons ensures that we cover a wide range of Sun–satellite geometries as well as pollution, cloud, and surface conditions in the training data.
We found, by trial and error, that use of 60 PCs sufficiently reconstructs the OMI spectra in the range of 349–503 nm (742 spectral samples) for the purposes of NO2 spectral fitting, while removing spurious features that can occur for example within the South Atlantic Anomaly region, as described by Gorkavyi et al. (2021). In other words, beyond this number, there is not significant correlation between PCs and NO2 absorption cross sections. The cleaned OMI L1B data are then averaged over different spectral band passes to simulate data from lower-spectral-resolution instruments. Here, we used a simple boxcar function with widths of 5 and 10 nm and resampled at 2.5 and 5 nm to simulate OCI- and GLIMR-like instruments, respectively. This produces 59 samples for the OCI-like instrument for a spectral range of 355–502.5 nm. Finally, noise following a Gaussian distribution is optionally added to the simulated spectra at the same wavelength grid for all rows according to instrument specifications and measurements. Here, we assume that errors are not correlated with wavelength, as information on correlated errors was not provided. Correlated errors could possibly degrade the performance of the retrieval if the NN is not able to effectively account for them.
2.3.3 Machine learning architecture
The machine learning algorithm was constructed within the Interactive Data Language (IDL) software package. It consists of a three-layer feed-forward artificial NN with two hidden layers and 1.3N nodes in each layer, where N is the number of inputs (Schmidhuber, 2015). Details regarding the inputs are given below. The output layer has OMI NO2 SCD as a single node. We use a soft-sign activation function for the first layer, a logistic (sigmoid) for the second layer, and a bent identity for the third layer. An adaptive moment estimation optimizer minimizes the error function with a learning rate of 0.1. We scale all inputs and outputs such that means are zero and standard deviations are equal to unity.
2.3.4 NN inputs
We perform a PCA (or eigen-decomposition) of the simulated spectra using the same data sample as described above for the noise reduction. The coefficients of a leading number of PCs are used as inputs to the NN. We find that the NN training converges faster when coefficients of the PCs are used rather than the measured radiances themselves, even when coefficients of all modes are used as inputs. The PCA concentrates on the spectral features corresponding to information about the atmosphere and surface in the leading modes while projecting the random instrument noise onto the trailing modes. This may make it easier for the NN to reject those coefficients with little information content pertaining to the target. We found, by trial and error, that maintaining a number of PCs equal to half the number of spectral elements was sufficient to capture the spectral information associated with NO2 while providing some noise reduction. As the trailing PCs typically express random spectral noise, eliminating these modes can lead to noise reduction.
We then perform quality control on the spectra. We remove all data with slant columns less than zero or greater than 1 × 1017 molec. cm−2. We also remove any pixels where the solar zenith angle (SZA) is greater than 75∘. Finally, we check that the quality flag on the OMI SCD data indicates a good fit.
The NN training is performed separately for each OMI CCD detector row, because each row has unique spectral characteristics. While it is possible to perform NN training on all rows at once, as the data are spectrally averaged to a uniform wavelength grid as in Fasnacht et al. (2022), we find that slightly better performance is achieved by training on each row individually.
The inputs to the NN are then the coefficients of the leading PCs and other optional parameters that may aid the NN in trying to match the target OMI NO2 SCD output variable. An important consideration for the selection of input parameters is that we are training a NN to estimate SCDs produced by a DOAS-like algorithm that used a more narrow fitting window weighted towards the blue spectral region, as noted in Fig. 2. SCDs, because they depend upon the atmospheric photon path, are by definition wavelength dependent. UV wavelengths have less sensitivity to lower-tropospheric NO2 than blue wavelengths owing to the effects of Rayleigh scattering that increase towards the UV and generally reduce the amount of light reflected from the surface (Richter et al., 2014).
We tested several parameters that can help to determine how much of the OMI-derived SCD originates from the lower troposphere, where UV wavelengths have less sensitivity. These include the cosine of the solar and view zenith angles, the cosine of the scattering angle, the stratospheric NO2 column from the Global Modeling Initiative (GMI) chemical transport model, the geometry-dependent Lambertian-equivalent reflectivity (GLER) of the surface (Vasilkov et al., 2017, 2018; Qin et al., 2019; Fasnacht et al., 2019), the surface pressure, and the effective scene pressure that is related to both the cloud pressure and optical thickness. Over ocean, the GLER accounts for the anisotropy of solar reflection from the ocean surface and backscatter from the bulk of ocean water; over land, it accounts for the bi-directionality of scattered sunlight from shadowing in vegetation. We also tried to use the stratospheric column provided by the Global Modeling Initiative (GMI) model multiplied by the stratospheric air mass factor, i.e., the expected stratospheric SCD based on the model estimate.
The parameters that were ultimately selected as input features are shown in Fig. 3. We find that the combination of stratospheric column NO2 and the cosines of solar zenith and scattering angles slightly improve the estimates of the target OMI NO2 SCDs, as discussed in more detail below. The cosine of the view zenith angle is nearly constant for a given row and does not provide significant improvement. The other inputs tested similarly did not substantially improve the fitting. The spectra themselves contain information about these variables, although the training of the NN may require more iterations if these variables are not included as predictors. For example, information about the cloud optical thickness and underlying surface is present in the radiances and can be disentangled using machine learning (Joiner et al., 2022; Fasnacht et al., 2022). Information about cloud pressure is contained within the oxygen dimer absorption band near 477 nm (Acarreta et al., 2004) as well as from the infilling of solar Fraunhofer lines by rotational-Raman scattering (e.g., Joiner and Bhartia, 1995). We take a minimalist approach here with respect to the predictors and only include stratospheric column amount and the cosines of solar zenith and scattering angles in addition to coefficients of leading PCs as features in the results shown below.
2.3.5 NN outputs
We also tested a variety of different target outputs. We tested both NO2 SCD and the natural logarithm of the NO2 SCD as target outputs. We find that slightly better results are obtained using the natural logarithm of the NO2 SCD, likely because SCDs are more normally distributed in log space, which is desirable for NN training. Box–Cox transformations (Box and Cox, 1964) may produce slightly better results again, but this will depend upon the sample used. As the distributions of NO2 have undergone changes with time, particularly in polluted regions, results obtained by training on a single distribution may not be optimal for a given time period.
We also tried training directly on the total VCD. More iterations for training may be needed owing to complex relationships with atmospheric constituents and the Earth's surface that impact the photon pathlength and that are needed to estimate the VCD, including dependencies on clouds, aerosol, and the surface bi-directionality. Without detailed knowledge of the NO2 profile (only estimates of the stratospheric and tropospheric columns from the GMI were used), we did not obtain a satisfactory result; thus, for simplicity, we focus on SCD exclusively as the target parameter below. The conversion from SCD to VCD can be achieved efficiently using existing algorithms.
2.3.6 NN training and evaluation
For the training, we used every third data point from the sample described above (i.e., using data from at least 1 d in every month), providing more than 100 000 samples for each row. We then compare statistical results including variance explained (r2); bias, defined as the mean of SCDtrue − SCDest, where SCDtrue and SCDest are the true (from OMNO2) and estimated NO2 SCDs, respectively; and root-mean-square difference based on independent samples (i.e., not used in the training set). We did not find evidence of overtraining. We also use a completely independent day (28 January 2005) for visual evaluation below.
2.4 Addressing instrumental noise in the training process
There are several possible ways of dealing with the effects of instrumental noise in the training and application of NNs. One method is to train a NN using noiseless data and then apply the trained network to noisy data. Another method is to train a NN using noisy data. The latter approach is likely to provide the best result, as the NN learns how to properly weight the different wavelengths based on a large sample that provides information about the wavelength dependence of the SNR. The former approach may work well if the SNR is relatively constant with wavelength. Another advantage to the former approach is that only one trained network is needed for either simulations or application to real data where the inputs could have a variety of different SNRs.
To mitigate the impact of instrumental random noise, one may either spatially average the noisy SCD retrievals or spectrally average radiance observations from adjacent fields of view (FoV) and use the coefficients computed with averaged spectra as inputs to the trained NN. If there are spatially dependent biases in the SCD results obtained with noisy data, these biases will not be eliminated by averaging together noisy retrievals. Therefore, it may be more advantageous to average spectra together and present the averaged data as inputs to a NN. The disadvantage of this approach, as described above, is that for optimal results, one may need to train a separate NN for each SNR scenario. We tested all of these approaches and found that the best results were obtained by averaging spectra together and training separate NNs for each SNR scenario. In practice, spatial averaging of pixels would be employed to increase the SNR, thereby degrading the spatial resolution of the resulting retrievals. This is the approach taken by studies such as Tack et al. (2017) with the Airborne Prism EXperiment (APEX) hyperspectral sensor, where spectra were averaged over an array of 20 × 20 pixels (a ground sample distance of 60 × 80 m2) to increase the SNR to 2500. This SNR value gave SCD uncertainties of 3.4–4.4 × 1015 molec. cm−2 with their limited fitting window of 470–510 nm and full width at half maximum (FWHM) values of 2.4–3.4 nm at the center wavelength of 490 nm. This relatively small fitting window was used to estimate NO2 SCDs owing to interference from instrumental artifacts and/or other atmospheric spectral contaminants at other wavelengths where there is strong NO2 absorption. Postylyakov et al. (2017) similarly averaged spectra from a hyperspectral sensor on the Resurs-P 2 satellite to a spatial resolution of 2.4 km to provide precision of about 2.5 × 1015 molec. cm−2 for SCD. They used a fitting window covering 420–490 nm for their retrieval with a spectral resolution that varies from 2.5 to 4 nm over this range.
3.1 Results with simulated PACE OCI spectra
We tested approaches initially using spectra from the OMI UV-2 and Vis channels with a range of 325–502.5 nm. We concatenated spectra from the two channels using UV-2 for wavelengths <355 nm and Vis for the remaining range. There was only a slight discontinuity in the joined spectra at the overlap wavelength. However, we found that use of the UV-2 wavelengths did not significantly improve results compared with those using the Vis channel alone. Therefore, all results shown below are obtained using a range or subset of the range from 355 to 502.5 nm obtained with the OMI Vis channel only. We note that there is a slight spatial mismatch between the OMI UV-2 and Vis channels that may have contributed to the overall lack of improvement using UV-2 wavelengths. In addition, there is limited high-frequency spectral structure in the convolved NO2 absorption cross sections in the UV-2 range (see Fig. 2). When PACE OCI data become available, we encourage testing again using all available wavelengths including those with wavelengths >502.5 nm that are not available from OMI.
∗ Data from 15 June 2005 withheld from training and used as evaluation (16 286 samples).
Table 2 shows the results of testing with different wavelength ranges and inputs on a single OMI row (row 1, zero based) and with and without noise using the SNR models (Fig. 1). The use of the SNR models assuming no spatial binning results in significant degradation for both OCI and GLIMR. For OCI, we find that the use of more restricted wavelength ranges results in little or no significant degradation in the results compared with the full range of 355–500 nm. Note that, in all cases, the training target is SCD from the OMNO2 algorithm that corresponds to the OMNO2 fitting window, and all statistics are computed with respect to the OMNO2 SCDs. Our results are consistent with the full-spectral-resolution results of Li et al. (2022), who found an optimal retrieval window of 390–495 nm for estimating NO2 vertical columns from TROPOMI radiances. Our results indicate that most of the information for NO2 at the OCI spectral resolution is provided by the high-frequency structure of the radiances produced by NO2 absorption within the fitting window currently used in the OMNO2 product. Little additional information is provided by UV wavelengths that define the more broad NO2 absorption feature. In addition, we show that removing the geometrical information (cosines of the solar zenith and relative azimuth angles) results in only a very small degradation. The use of an estimate of the stratospheric column NO2 does appear to aid the estimation of the NO2 SCD.
Most results in Table 2 are shown for 28 January 2005, a day not used in the training. For comparison, we also show results for a model with all predictors where we withheld data from 15 June 2005 from the training and instead used it for evaluation. On this day, the correlation is significantly lower compared with 28 January 2005 and the root-mean-square difference (RMSD) is slightly higher. In the Northern Hemisphere, there are generally high anthropogenic NOx emissions in populated regions. These emissions lead to higher NO2 column amounts in the winter when lifetimes are generally longer. The solar zenith angles are also higher in winter than in summer. These factors lead to higher SCDs in winter in the Northern Hemisphere populated regions than in summer. The higher NO2 SCDs and variability in the Northern Hemisphere winter result in higher sensitivities and improved global statistics.
The results for GLIMR, shown in Table 2, were not satisfactory even without adding noise. GLIMR results were substantially worse with added noise, even after increasing the nominal SNR by factors of 3 and 6 that would be equivalent to averaging the spectra of 9 and 36 pixels together, respectively. One reason for the poor performance is the lack of higher-frequency spectral structure of the NO2 effective cross sections at the GLIMR resolution, particularly at blue wavelengths (> 400 nm). Another factor is that there are far fewer available spectral samples for GLIMR; therefore, the impact of instrumental noise is substantial. No additional results will be shown for GLIMR.
Table 3 shows the results of the trained NN applied to data from all rows on 28 January 2005. Here, we report statistics for SCDs normalized by the stratospheric AMF (essentially assuming that all NO2 is in the stratosphere) to give a rough estimate of the VCD. Simulations without noise produced quite reasonable results, capturing about 94 % of the variability with little overall bias. Results degrade noticeably when the SNR from Fig. 1b is applied to the simulated spectra; the variability captured drops to about 91 %, and the RMSD increases from about 0.30 × 1015 to 0.36 × 1015 molec. cm−2. More than half of the degradation that results from adding noise is recovered when the SNR is increased by a factor of 4. Such an increase in the SNR could be achieved by averaging together spectra from a 4×4 array of OCI pixels to give an area of approximately 16 km2, which is still an improvement over TROPOMI or averaging over 16 d of good observations. We computed statistics for a sample of data with less pollution (NO2 < 4 × 1015 molec. cm−2). Here, we see a significant decrease in correlation, likely due to lower variability within the sample and decreased sensitivity, consistent with results shown for a summer month in Table 2. The RMSD was also a bit smaller for this cleaner sample.
We also looked at how the performance varies across the OMI swath. We found better performance at the OMI swath edges where the SCD values are largest, owing to larger view angles, leading to deeper absorption features (last two lines of Table 3; swath edges are the rows with both the highest and lowest values). Even when normalized by the stratospheric AMF, the performance enhancement at the swath edges remains.
Figure 4 shows normalized SCD results obtained with data from 28 January 2005 using the PACE OCI SNR × 2 scenario. The bulk of the normalized SCDs are in the approximate range of 1 × 1015–4 × 1015 molec. cm−2. Figure 4b shows that there is little evidence of striping (systematic row-dependent errors) in the NN results. Note that there was no attempt to destripe the SCDs on this day of independent data as is done on a daily basis in the OMNO2 data used to train the NN. Striping can occur on an orbital or daily basis which is why OMNO2 initial SCDs undergo a destriping process. Figure 4c shows that there are some systematic differences between OMNO2 and the NN normalized SCDs as a function of latitude. For example, the NN-derived SCDs are lower than OMNO2 at low southern latitudes (over Antarctica) and also at middle northern latitudes that tend to occur for higher values of SCD (see Fig. 4a, d), likely over polluted regions. The NN values are somewhat higher near the Equator. These systematic differences are not understood and may result from errors in either OMNO2 or the NN.
Figure 5 shows an example of how the NO2 SCD errors may be spatially correlated. Because we use OMI data as the “truth” or target for training, simulated PACE OCI results are shown at the OMI spatial resolution rather than at the OCI resolution. The reader must imagine that real OCI results can be obtained at a spatial resolution as high as 1 km × 1 km, rather than at the OMI resolution (12 km × 24 km at nadir) shown here. Judd et al. (2020) provided examples of fine-resolution NO2 over an urban area as measured from an aircraft instrument.
Here, we display retrieved SCDs based on simulations at various SNR levels on 28 January 2005, a day with high pollution over the northeastern and midwestern portions of the United States. Figure 5a shows the original OMNO2 SCDs normalized by the stratospheric AMFs to provide values similar to the total VCDs. These are considered as the true values used for comparisons with those from simulated OCI retrievals. Figure 5b shows the estimated fraction of radiance coming from cloudy portions of a pixel (cloud radiance fraction). In areas that have cloud radiance fractions approaching unity, most of the observed NO2 SCD would result from NO2 in the stratosphere or upper troposphere owing to clouds that screen the polluted atmosphere below them from satellite view. Note that pixels with snow cover that is not reported in the input snow/ice data set will be reported as cloud cover.
Results obtained using the simulated PACE OCI spectra with no noise added (Fig. 5c, d) show relatively small regional differences, with the largest differences occurring over highly polluted areas with some spatial dependence. Differences over polluted regions may occur because wavelengths in the UV, which the OCI retrieval uses, have less sensitivity to NO2 in the boundary layer where NO2 can accumulate under high-pollution conditions. It is also possible that there are imperfections in the OMNO2 data. The “no noise” case represents the best result (upper limit) that can be expected based on the OCI sampling and spectral resolution for this particular training scenario. We tried alternate training scenarios such as training and applying NNs separately over land and water, but this failed to remove all of the differences.
The effects of the expected PACE OCI instrument noise, shown in Fig. 5e and f, degrade the results with noticeably larger differences in NO2 SCDs over the relatively clean oceanic regions and spatially dependent differences over polluted areas. The effect of adding random noise to the spectra causes the NN to draw less closely to the input data, and the ultimate effect may be to produce systematic or spatially dependent errors as well as random errors. Figure 6a–d shows results of simulations using the OCI SNR multiplied by 2 or 4. The effect of increasing the SNR reduces the spatially dependent differences in polluted areas.
Figure 6e shows fitting uncertainties as given in the OMNO2 data set. The SCD uncertainties generally correspond to values of ∼ 0.12 to 0.6 molec. cm−2 in the normalized SCD. For the case of no noise added to the simulated OCI spectra in Fig. 5c and d, most differences fall within the range of fitting uncertainties. As noise is added to the simulated OCI spectra, the differences start to fall more outside the OMNO2 fitting uncertainties, particularly in the polluted areas. Figure 6f shows a histogram of normalized SCD for low values typical of cleaner regions (0 × 1015–4 × 1015 molec. cm−2) for OMNO2 and for the case of OCI spectra with noise added according to 2 × the SNR model. Here, we see a more peaked distribution of values for the NN estimates. This may indicate that the NN with inputs of leading PCA coefficients may be reducing the effects of random instrument noise. We further investigate potential noise reduction for a cleaner region in Sect. 3.3.
3.2 Practical implementation issues
We next address how our approach can be practically implemented with a high-spatial-resolution, low-spectral-resolution hyperspectral sensor such as PACE OCI and an existing moderate-spectral-resolution spectrometer such as TROPOMI. The desired retrieved quantity for atmospheric correction in ocean color algorithms is not the NO2 SCD for a particular fitting window but rather the VCD such that the appropriate absorption can then be accurately computed at any wavelength for atmospheric correction (Ahmad et al., 2007). Once a NN is trained to produce NO2 SCD from input spectra, the SCDs may be converted to VCD using a computed AMF that will be a function of the Sun–satellite geometry, surface and cloud conditions, and NO2 profile shape, as described above and shown in Figs. 7–8. This is typically accomplished with lookup tables and model-generated NO2 profiles. With the NO2 VCD, the spectral transmittance due to NO2 can then be computed with a radiative transfer model that will be a function of wavelength, the surface albedo, and other absorbers and scatters in the atmosphere. This last step may be performed with either a lookup table or machine learning.
To use NO2 SCDs from TROPOMI or a similar spectrometer for the training of a NN with co-located spectra from a higher-spatial-resolution hyperspectral instrument, such as OCI, as inputs for the purpose of estimating SCDs from the imager, it is necessary to use radiative transfer to transform the SCDs from the spectrometer to the appropriate Sun–satellite geometry of the imager, as shown in Fig. 7; the overlap between these two instruments on different satellites will not typically occur for the same Sun–satellite geometries. One approach to prepare a training data set would be to use the total VCD retrievals from the spectrometer (e.g., TROPOMI) to derive an estimated SCD for the imager (e.g., OCI) at its geometry. One can simply apply computed AMFs at the appropriate geometries. This can be written as follows:
where AMFimager is the AMF computed using the same inputs as for the spectrometer (e.g., cloud radiance fraction and NO2 profile shape) but using the appropriate Sun–satellite geometry, including its impact on surface bidirectional reflectance (currently accomplished using the GLER framework), for the imager observation. This kind of approach could be made to work relatively quickly with mature and well-validated VCDs from a spectrometer without the need to understand or make adjustments for instrumental artifacts in the imager.
The imager spatial resolution will be higher than that of the spectrometer. One way to prepare a co-located training set would involve averaging the spectra from the imager to match the footprints of the spectrometer. This averaging will effectively change the SNR of the imager data compared with that at its native spatial resolution. The resulting training will, therefore, not be optimized for application at the native resolution. If the trained network is then applied at the native resolution of the imager, the results will have to be carefully validated and checked. Another possible approach would be to spatially interpolate the spectrometer data to the locations of the imager or to perhaps use a high-resolution model to downscale the spectrometer data to the resolution of the imager. Addressing these details is beyond the scope of the present work and will be dealt with in future studies.
One advantage of using co-located data from a spectrometer is that a fitting algorithm would not have to be developed and tested for the imager. We have found that developing and validating such algorithm can require significant human resources. However, our results suggest that it is possible to develop a fitting algorithm for the imager by exploiting the high-frequency structure of NO2 absorption. Such an approach, not requiring co-located data from another instrument, can be considered as an alternative approach. Spectral fitting algorithms can be computationally intensive, and it may still be desirable to use machine learning to speed up the processing of dense imager data, as shown in Fig. 8. For example, Li et al. (2022) found that a NN implementation for NO2 vertical columns using TROPOMI spectra was about 12 times faster than a full implementation using a priori profiles from a high-spatial-resolution chemistry transport model.
Another consideration is how often a NN would need to be retrained. If the instrument were spectrally stable, retraining might not be necessary or might be infrequent. However, destriping may still be necessary to correct for transient spectral artifacts. Retraining should be done whenever there is a substantial change in the instrument spectral characteristics. As the OMNO2 algorithm uses monthly averaged solar irradiances, it may be more optimal to similarly normalize with respect to the same set of solar irradiances before training than to use only the radiances (as we have done here), as the solar data may help to account for instrumental changes.
3.3 Use of PCA and a NN for reducing noise in NO2 slant column retrievals
We next explore the use of PCA in conjunction with a NN for noise reduction in SCD retrievals. In addition, a NN implementation may have the benefit of significantly reducing computation time of the spectral fitting algorithm. Machine learning that uses leading PCA coefficients as inputs may be adept at filtering out spectral interferences as well as random instrument noise.
For the following experiments with real OMI data, our assumption is that, over a generally clean environment (Pacific Ocean), variability in the NO2 SCD, due to clouds for example, is relatively small. In this region, the majority of the NO2 column resides in the stratosphere and upper troposphere, where UV wavelengths have good sensitivity. We use the same training days as above and evaluate using data from an independent day (28 January 2005).
Table 4 shows the standard deviation of normalized SCDs computed over the region bounded by 10 ∘S to 20 ∘N latitude and 110 to 173 ∘E longitude for OMNO2 and three separately trained NNs. Here, we attempt to disentangle the impact of three different factors on the retrieval noise: (1) use of a NN with leading PCA coefficients as inputs and a spectral fitting window mirroring that of OMNO2; (2) a NN trained similar to (1) but with an extended spectral range; and (3) a NN similar to (2) but spectrally averaged to a coarser resolution. The results of using the NN with a PCA approach with the OMNO2 spectral fitting window (402–465 nm) and at the OMI spectral resolution decrease the variability in the region by about 30 %. Nearly identical reductions are obtained using the NN–PCA setup with either an increased spectral fitting window (355–500 nm) or a reduced spectral resolution (using 5 nm compared with the OMI native resolution of 0.63 nm). The results of these experiments suggest that it is the NN setup with leading coefficients of PCs that is leading to the noise reduction. The NN approach combined with PCA appears to be effective at isolating information about NO2 in the spectra while rejecting interfering spectral features and random noise.
Figure 9a and b show the noise reduction visually using NN results obtained with the OMNO2 spectral fitting window (402–465 nm) for a day not used in the training. There appears to be a noticeable reduction in random noise over this clean region. The noise reduction is particularly apparent over highly cloudy regions (see Fig. 9c) where the atmosphere below the clouds is shielded from satellite view and the majority of the observed SCD is in the stratosphere. The difference map in Fig. 9d shows mostly random patterns with similar magnitudes to the OMNO2 fitting uncertainties the OMNO2 fitting uncertainties shown in Fig. 9e. Figure 9f shows a histogram of the retrieved normalized SCDs from the target OMNO2 and the NN. OMNO2 has more pixels in the tails of the distribution, particularly at the low end. Negative SCDs are reported in the OMNO2 data set. These are retained for statistical purposes, although they are physically unrealistic. The trained NN substantially reduces these low values and also reduces the number of pixels at the high SCD tail of the distribution. Similar results are obtained with the other setups described above (lower spectral resolution and a wider spectral fitting window).
We have simulated data from the hyperspectral imagers PACE OCI and GLIMR using OMI to demonstrate that it is possible to retrieve the NO2 SCD with reasonable accuracy and precision using lower-spectral-resolution data from the PACE OCI. Instrumental noise significantly impacts the results as do the spectral resolution and sampling. Better results are obtained in cases of NO2 pollution contained in the boundary layer when the spectral resolution is high enough (of the order of 5 nm or better) to capture the higher-frequency spectral structures in the blue part of the NO2 absorption spectrum. The longest OMI wavelength is at about 500 nm; OCI spectral coverage will continue to longer wavelengths in the green and yellow parts of the spectrum where NO2 has additional absorption features. The added spectral range may improve results over those shown here, although the magnitude of improvement is not expected to be dramatic. Averaging spectra from several adjacent OCI pixels together will improve the performance of NO2 retrievals from PACE OCI that may be used for atmospheric correction in ocean color retrievals.
There are other potential applications for PACE OCI NO2 retrievals. Currently, NO2 tropospheric column density retrievals from instruments such as OMI and TROPOMI are averaged over various time periods to reduce the impacts of retrieval noise and meteorology (e.g., Lamsal et al., 2015; Duncan et al., 2016). Similar averaging of NO2 data from imagers over time, e.g., of the order of a month or more, may produce good-quality data at higher spatial resolution than is available from TROPOMI. This higher-spatial-resolution data could then be used to downscale TROPOMI and historical OMI retrievals or could be used for emissions estimates based on averaged maps – for example, using recently developed methods for high-resolution data (Liu et al., 2022). The use of high-resolution averages is also useful for studies involving health impacts, including investigations involving environmental inequities (e.g., Goldberg et al., 2021; Kerr et al., 2021; Cooper et al., 2022).
We also show that our machine learning with PCA approach for OCI can be used to reduce noise in retrieved NO2 SCDs (at the least in unpolluted situations) for spectrometers such as OMI and TROPOMI. An additional advantage of using machine learning for noise reduction in spectral fitting is that once trained, an applied NN is an extremely efficient algorithm. The current OMNO2 spectral fit is the most computationally intensive portion of the OMNO2 NO2 tropospheric column retrieval algorithm. This may be an important consideration with the new generation of sensors in geostationary orbit with very large data volumes (Zoogman et al., 2017). These include the Korean Geostationary Environment Monitoring Spectrometer (GEMS), NASA Tropospheric emissions: Monitoring of pollution (TEMPO), and Copernicus Sentinel-4. Training could be applied intermittently throughout the data record to ensure that time-varying instrumental artifacts are accounted for. We stress that a high-quality physically based fitting algorithm is a necessary part of any machine learning approach, as it produces the retrievals needed as the training target. Our machine learning approach is not meant to replace these algorithms but rather to enhance and speed them up.
Figure A1 shows typical radiance distributions for OMI row 1 taken over the same orbits as used for the NN training as described above. The radiance distribution depends upon how the incoming solar irradiance is modified by gaseous and particle scattering and absorption in the atmosphere as well as surface reflectance properties.
JJ was responsible for the design of the methodology, investigation, including software, visualization, calculations, and formal analysis, and wrote the first draft of the article. JJ, AV, and NK provided overall supervision of the project. NK and JJ contributed to funding acquisition. LL, NK, ZF, CL, and SM provided guidance on the use of OMI products. All authors contributed to article revision as well as reading and approving the submitted version.
The contact author has declared that none of the authors has any competing interests.
Publisher’s note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The authors thank the international OMI team that produced and distributed the OMI data sets used here. They also thank the PACE OCI and GLIMR teams, particularly Antonio Mannino, Bryan Franz, Shihyan Lee, Amir Ibrahim, Andrew Sayer, Brian Cairns, and Gerhard Meister who provided the signal-to-noise (SNR) estimates used here and assistance in simulating PACE OCI and GLIMR data. Finally, the lead author thanks Arlindo da Silva for enlightening conversations. This work was supported by NASA through the PACE and Aura science team programs.
This research has been supported by the National Aeronautics and Space Administration under the PACE science team program (NNH19ZDA001N-PACESAT).
This paper was edited by Michel Van Roozendael and reviewed by two anonymous referees.
Acarreta, J. R., De Haan, J. F., and Stammes, P.: Cloud pressure retrieval using the O2-O2 absorption band at 477 nm, J. Geophys. Res.-Atmos., 109, D05204, https://doi.org/10.1029/2003JD003915, 2004. a
Ahmad, Z., McClain, C. R., Herman, J. R., Franz, B. A., Kwiatkowska, E. J., Robinson, W. D., Bucsela, E. J., and Tzortziou, M.: Atmospheric correction for NO2 absorption in retrieving water-leaving reflectances from the SeaWiFS and MODIS measurements, Appl. Optics, 46, 6504–6512, https://doi.org/10.1364/AO.46.006504, 2007. a, b, c
Bak, J., Liu, X., Kim, J.-H., Haffner, D. P., Chance, K., Yang, K., and Sun, K.: Characterization and correction of OMPS nadir mapper measurements for ozone profile retrievals, Atmos. Meas. Tech., 10, 4373–4388, https://doi.org/10.5194/amt-10-4373-2017, 2017. a
Boersma, K. F., Eskes, H. J., Veefkind, J. P., Brinksma, E. J., van der A, R. J., Sneep, M., van den Oord, G. H. J., Levelt, P. F., Stammes, P., Gleason, J. F., and Bucsela, E. J.: Near-real time retrieval of tropospheric NO2 from OMI, Atmos. Chem. Phys., 7, 2103–2118, https://doi.org/10.5194/acp-7-2103-2007, 2007. a
Boersma, K. F., Eskes, H. J., Dirksen, R. J., van der A, R. J., Veefkind, J. P., Stammes, P., Huijnen, V., Kleipool, Q. L., Sneep, M., Claas, J., Leitão, J., Richter, A., Zhou, Y., and Brunner, D.: An improved tropospheric NO2 column retrieval algorithm for the Ozone Monitoring Instrument, Atmos. Meas. Tech., 4, 1905–1928, https://doi.org/10.5194/amt-4-1905-2011, 2011. a
Boersma, K. F., Eskes, H. J., Richter, A., De Smedt, I., Lorente, A., Beirle, S., van Geffen, J. H. G. M., Zara, M., Peters, E., Van Roozendael, M., Wagner, T., Maasakkers, J. D., van der A, R. J., Nightingale, J., De Rudder, A., Irie, H., Pinardi, G., Lambert, J.-C., and Compernolle, S. C.: Improving algorithms and uncertainty estimates for satellite NO2 retrievals: results from the quality assurance for the essential climate variables (QA4ECV) project, Atmos. Meas. Tech., 11, 6651–6678, https://doi.org/10.5194/amt-11-6651-2018, 2018. a
Bovensmann, H., Burrows, J. P., Buchwitz, M., Frerick, J., Noël, S., Rozanov, V. V., Chance, K. V., and Goede, A. P. H.: SCIAMACHY: Mission objectives and measurement modes, J. Atmos. Sci., 56, 127–150, https://doi.org/10.1175/1520-0469(1999)056<0127:SMOAMM>2.0.CO;2, 1999. a
Bovensmann, H., Aben, I., Van Roozendael, M., Kühl, S., Gottwald, M., von Savigny, C., Buchwitz, M., Richter, A., Frankenberg, C., Stammes, P., de Graaf, M., Wittrock, F., Sinnhuber, M., Sinnhuber, B. M., Schönhardt, A., Beirle, S., Gloudemans, A., Schrijver, H., Bracher, A., Rozanov, A. V., Weber, M., and Burrows, J. P.: SCIAMACHY's view of the changing Earth's environment, in: SCIAMACHY – Exploring the changing Earth's atmosphere, edited by: Gottwald, M. and Bovensmann, H., Springer Netherlands, Dordrecht, 175–216, https://doi.org/10.1007/978-90-481-9896-2_10, 2011. a
Bucsela, E., Celarier, E., Wenig, M., Gleason, J., Veefkind, J., Boersma, K., and Brinksma, E.: Algorithm for NO2 vertical column retrieval from the Ozone Monitoring Instrument, IEEE T. Geosci. Remote, 44, 1245–1258, https://doi.org/10.1109/TGRS.2005.863715, 2006. a
Bucsela, E. J., Krotkov, N. A., Celarier, E. A., Lamsal, L. N., Swartz, W. H., Bhartia, P. K., Boersma, K. F., Veefkind, J. P., Gleason, J. F., and Pickering, K. E.: A new stratospheric and tropospheric NO2 retrieval algorithm for nadir-viewing satellite instruments: applications to OMI, Atmos. Meas. Tech., 6, 2607–2626, https://doi.org/10.5194/amt-6-2607-2013, 2013. a, b
Burrows, J. P., Weber, M., Buchwitz, M., Rozanov, V., Ladstätter-Weißenmayer, A., Richter, A., DeBeek, R., Hoogen, R., Bramstedt, K., Eichmann, K.-U., Eisinger, M., and Perner, D.: The Global Ozone Monitoring Experiment (GOME): Mission concept and first scientific results, J. Atmos. Sci., 56, 151–175, https://doi.org/10.1175/1520-0469(1999)056<0151:TGOMEG>2.0.CO;2, 1999. a, b
Choi, S., Lamsal, L. N., Follette-Cook, M., Joiner, J., Krotkov, N. A., Swartz, W. H., Pickering, K. E., Loughner, C. P., Appel, W., Pfister, G., Saide, P. E., Cohen, R. C., Weinheimer, A. J., and Herman, J. R.: Assessment of NO2 observations during DISCOVER-AQ and KORUS-AQ field campaigns, Atmos. Meas. Tech., 13, 2523–2546, https://doi.org/10.5194/amt-13-2523-2020, 2020. a
Cooper, M. J., Martin, R. V., Hammer, M. S., Levelt, P. F., Veefkind, P., Lamsal, L. N., Krotkov, N. A., Brook, J. R., and McLinden, C. A.: Global fine-scale changes in ambient NO2 during COVID-19 lockdowns, Nature, 601, 380–387, https://doi.org/10.1038/s41586-021-04229-0, 2022. a, b
Dobber, M.: OMI/Aura Level 1B UV Global Geolocated Earthshine Radiances 1-orbit L2 Swath 13x24 km V003, Goddard Earth Sciences Data and Information Services Center (GES DISC) [data set], https://doi.org/10.5067/AURA/OMI/DATA1002, 2007a. a, b
Dobber, M.: OMI/Aura Level 1B VIS Global Geolocated Earthshine Radiances 1-orbit L2 Swath 13x24 km V003, Goddard Earth Sciences Data and Information Services Center (GES DISC) [data set], https://doi.org/10.5067/AURA/OMI/DATA1004, 2007b. a, b
Duncan, B. N., Lamsal, L. N., Thompson, A. M., Yoshida, Y., Lu, Z., Streets, D. G., Hurwitz, M. M., and Pickering, K. E.: A space-based, high-resolution view of notable changes in urban NOx pollution around the world (2005–2014), J. Geophys. Res.-Atmos., 121, 976–996, https://doi.org/10.1002/2015JD024121, 2016. a, b
Fasnacht, Z., Vasilkov, A., Haffner, D., Qin, W., Joiner, J., Krotkov, N., Sayer, A. M., and Spurr, R.: A geometry-dependent surface Lambertian-equivalent reflectivity product for UV–Vis retrievals – Part 2: Evaluation over open ocean, Atmos. Meas. Tech., 12, 6749–6769, https://doi.org/10.5194/amt-12-6749-2019, 2019. a
Fasnacht, Z., Joiner, J., Haffner, D., Qin, W., Vasilkov, A., Castellanos, P., and Krotkov, N.: Using machine learning for timely estimates of ocean color information from hyperspectral satellite measurements in the presence of clouds, aerosols, and sunglint, Frontier in Remote Sensing, vol. 3, https://doi.org/10.3389/frsen.2022.846174, 2022. a, b
Fleig, A. J., Bhartia, P. K., Wellemeyer, C. G., and Silberstein, D. S.: Seven years of total ozone from the TOMS instrument-A report on data quality, Geophys. Res. Lett., 13, 1355–1358, https://doi.org/10.1029/GL013i012p01355, 1986. a
Goldberg, D. L., Anenberg, S. C., Kerr, G. H., Mohegh, A., Lu, Z., and Streets, D. G.: TROPOMI NO2 in the United States: A detailed look at the annual averages, weekly cycles, effects of temperature, and correlation with surface NO2 concentrations, Earth's Future, 9, e2020EF001665, https://doi.org/10.1029/2020EF001665, 2021. a, b
Gorkavyi, N., Fasnacht, Z., Haffner, D., Marchenko, S., Joiner, J., and Vasilkov, A.: Detection of anomalies in the UV–vis reflectances from the Ozone Monitoring Instrument, Atmos. Meas. Tech., 14, 961–974, https://doi.org/10.5194/amt-14-961-2021, 2021. a
Joiner, J. and Bhartia, P. K.: The determination of cloud pressures from rotational Raman scattering in satellite backscatte ultraviolet measurements, J. Geophys. Res.-Atmos., 100, 23019–23026, https://doi.org/10.1029/95JD02675, 1995. a
Joiner, J., Fasnacht, Z., Qin, W., Yoshida, Y., Vasilkov, A. P., Li, C., Lamsal, L., and Krotkov, N.: Use of hyper-spectral visible and near-infrared satellite data for timely estimates of the Earth's surface reflectance in cloudy and aerosol loaded conditions: Part 1 – Application to RGB image restoration over land with GOME-2, Frontiers in Remote Sensing, vol. 2, https://doi.org/10.3389/frsen.2021.716430, 2022. a
Judd, L. M., Al-Saadi, J. A., Szykman, J. J., Valin, L. C., Janz, S. J., Kowalewski, M. G., Eskes, H. J., Veefkind, J. P., Cede, A., Mueller, M., Gebetsberger, M., Swap, R., Pierce, R. B., Nowlan, C. R., Abad, G. G., Nehrir, A., and Williams, D.: Evaluating Sentinel-5P TROPOMI tropospheric NO2 column densities with airborne and Pandora spectrometers near New York City and Long Island Sound, Atmos. Meas. Tech., 13, 6113–6140, https://doi.org/10.5194/amt-13-6113-2020, 2020. a
Kerr, G. H., Goldberg, D. L., and Anenberg, S. C.: COVID-19 pandemic reveals persistent disparities in nitrogen dioxide pollution, P. Natl. Acad. Sci. USA, 118, e2022409118, https://doi.org/10.1073/pnas.2022409118, 2021. a, b
Krotkov, N. A., McLinden, C. A., Li, C., Lamsal, L. N., Celarier, E. A., Marchenko, S. V., Swartz, W. H., Bucsela, E. J., Joiner, J., Duncan, B. N., Boersma, K. F., Veefkind, J. P., Levelt, P. F., Fioletov, V. E., Dickerson, R. R., He, H., Lu, Z., and Streets, D. G.: Aura OMI observations of regional SO2 and NO2 pollution changes from 2005 to 2015, Atmos. Chem. Phys., 16, 4605–4629, https://doi.org/10.5194/acp-16-4605-2016, 2016. a
Krotkov, N. A., Lamsal, L. N., Marchenko, S. V., Bucsela, E. J., Swartz, W. H., Joiner, J., and the OMI core team: OMI/Aura Nitrogen dioxide (NO2) total and Tropospheric Column 1-orbit L2 Swath 13x24 km V003, Goddard Earth Sciences Data and Information Services Center (GES DISC) [data set], https://doi.org/10.5067/Aura/OMI/DATA2017, 2019. a, b
Kuhlmann, G., Chan, K. L., Donner, S., Zhu, Y., Schwaerzel, M., Dörner, S., Chen, J., Hueni, A., Nguyen, D. H., Damm, A., Schütt, A., Dietrich, F., Brunner, D., Liu, C., Buchmann, B., Wagner, T., and Wenig, M.: Mapping the spatial distribution of NO2 with in situ and remote sensing instruments during the Munich NO2 imaging campaign, Atmos. Meas. Tech., 15, 1609–1629, https://doi.org/10.5194/amt-15-1609-2022, 2022. a
Lamsal, L. N., Krotkov, N. A., Celarier, E. A., Swartz, W. H., Pickering, K. E., Bucsela, E. J., Gleason, J. F., Martin, R. V., Philip, S., Irie, H., Cede, A., Herman, J., Weinheimer, A., Szykman, J. J., and Knepp, T. N.: Evaluation of OMI operational standard NO2 column retrievals using in situ and surface-based NO2 observations, Atmos. Chem. Phys., 14, 11587–11609, https://doi.org/10.5194/acp-14-11587-2014, 2014. a
Lamsal, L. N., Duncan, B. N., Yoshida, Y., Krotkov, N. A., Pickering, K. E., Streets, D. G., and Lu, Z.: U.S. NO2 trends (2005–2013): EPA Air Quality System (AQS) data versus improved observations from the Ozone Monitoring Instrument (OMI), Atmos. Environ., 110, 130–143, https://doi.org/10.1016/j.atmosenv.2015.03.055, 2015. a, b
Lamsal, L. N., Krotkov, N. A., Vasilkov, A., Marchenko, S., Qin, W., Yang, E.-S., Fasnacht, Z., Joiner, J., Choi, S., Haffner, D., Swartz, W. H., Fisher, B., and Bucsela, E.: Ozone Monitoring Instrument (OMI) Aura nitrogen dioxide standard product version 4.0 with improved surface and cloud treatments, Atmos. Meas. Tech., 14, 455–479, https://doi.org/10.5194/amt-14-455-2021, 2021. a, b
Le, C. and Hu, C.: A hybrid approach to estimate chromophoric dissolved organic matter in turbid estuaries from satellite measurements: A case study for Tampa Bay, Opt. Express, 21, 18849–18871, https://doi.org/10.1364/OE.21.018849, 2013. a
Lerot, C., Stavrakou, T., De Smedt, I., Müller, J.-F., and Van Roozendael, M.: Glyoxal vertical columns from GOME-2 backscattered light measurements and comparisons with a global model, Atmos. Chem. Phys., 10, 12059–12072, https://doi.org/10.5194/acp-10-12059-2010, 2010. a
Levelt, P., van den Oord, G., Dobber, M., Malkki, A., Visser, H., de Vries, J., Stammes, P., Lundell, J., and Saari, H.: The ozone monitoring instrument, IEEE T. Geosci. Remote, 44, 1093–1101, https://doi.org/10.1109/TGRS.2006.872333, 2006. a, b
Levelt, P. F., Joiner, J., Tamminen, J., Veefkind, J. P., Bhartia, P. K., Stein Zweers, D. C., Duncan, B. N., Streets, D. G., Eskes, H., van der A, R., McLinden, C., Fioletov, V., Carn, S., de Laat, J., DeLand, M., Marchenko, S., McPeters, R., Ziemke, J., Fu, D., Liu, X., Pickering, K., Apituley, A., González Abad, G., Arola, A., Boersma, F., Chan Miller, C., Chance, K., de Graaf, M., Hakkarainen, J., Hassinen, S., Ialongo, I., Kleipool, Q., Krotkov, N., Li, C., Lamsal, L., Newman, P., Nowlan, C., Suleiman, R., Tilstra, L. G., Torres, O., Wang, H., and Wargan, K.: The Ozone Monitoring Instrument: overview of 14 years in space, Atmos. Chem. Phys., 18, 5699–5745, https://doi.org/10.5194/acp-18-5699-2018, 2018. a, b, c
Li, C., Xu, X., Liu, X., Wang, J., Sun, K., van Geffen, J., Zhu, Q., Ma, J., Jin, J., Qin, K., He, Q., Xie, P., Ren, B., and Cohen, R. C.: Direct retrieval of NO2 vertical columns from UV-Vis (390–495 nm) spectral radiances using a neural network, J. Remote Sens., 2022, 9817134, https://doi.org/10.34133/2022/9817134, 2022. a, b, c
Liu, F., Tao, Z., Beirle, S., Joiner, J., Yoshida, Y., Smith, S. J., Knowland, K. E., and Wagner, T.: A new method for inferring city emissions and lifetimes of nitrogen oxides from high-resolution nitrogen dioxide observations: a model study, Atmos. Chem. Phys., 22, 1333–1349, https://doi.org/10.5194/acp-22-1333-2022, 2022. a
Mannino, A., Russ, M. E., and Hooker, S. B.: Algorithm development and validation for satellite-derived distributions of DOC and CDOM in the U.S. Middle Atlantic Bight, J. Geophys. Res.-Oceans, 113, C07051, https://doi.org/10.1029/2007JC004493, 2008. a
Marchenko, S., Krotkov, N. A., Lamsal, L. N., Celarier, E. A., Swartz, W. H., and Bucsela, E. J.: Revising the slant column density retrieval of nitrogen dioxide observed by the Ozone Monitoring Instrument, J. Geophys. Res.-Atmos., 120, 5670–5692, https://doi.org/10.1002/2014JD022913, 2015. a, b, c
Maxwell, A. E., Warner, T. A., and Fang, F.: Implementation of machine-learning classification in remote sensing: an applied review, Int. J. Remote Sens., 39, 2784–2817, https://doi.org/10.1080/01431161.2018.1433343, 2018. a
Munro, R., Lang, R., Klaes, D., Poli, G., Retscher, C., Lindstrot, R., Huckle, R., Lacan, A., Grzegorski, M., Holdak, A., Kokhanovsky, A., Livschitz, J., and Eisinger, M.: The GOME-2 instrument on the Metop series of satellites: instrument design, calibration, and level 1 data processing – an overview, Atmos. Meas. Tech., 9, 1279–1301, https://doi.org/10.5194/amt-9-1279-2016, 2016. a
NASA: NASA targets coastal ecosystems with new space sensor, RELEASE 19-065, https://www.nasa.gov/press-release/nasa-targets-coastal-ecosystems-with-new-space-sensor (last access: 1 August 2022), 2019. a
Platt, U.: Differential optical absorption spectroscopy (DOAS), in: Air Monitoring by Spectrometric Techniques, Chemical Analysis Series, edited by: Sigrist, M. W., John Wiley, New York, USA, vol. 127, 27–84, ISBN 978-0-471-55875-0, 1994. a
Platt, U. and Perner, D.: Measurements of atmospheric trace gases by long path differential UV/visible absorption spectroscopy, in: Optical and Laser Remote Sensing, edited by: Killinger, D. A. and Mooradien, A., Springer Verlag, New York, USA, 95–105, ISBN 978-3-662-15736-7, 1983. a
Postylyakov, O. V., Borovski, A. N., and Makarenkov, A. A.: First experiment on retrieval of tropospheric NO2 over polluted areas with 2.4-km spatial resolution basing on satellite spectral measurements, XXIII International Symposium, Atmospheric and Ocean Optics, Atmospheric Physics, 3–7 July 2017, Irkutsk, Russian Federation, in: Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol. 10466, 104662Y, https://doi.org/10.1117/12.2285794, 2017. a, b
Postylyakov, O. V., Borovski, A. N., Elansky, N. F., Davydova, M. A., Zakharova, S. A., and Makarenkov, A. A.: Comparison of space high-detailed experimental and model data on tropospheric NO2 distribution, XXV International Symposium, Atmospheric and Ocean Optics, Atmospheric Physics, 1–5 July 2019, Novosibirsk, Russian Federation, in: Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol. 11208, 112082S, https://doi.org/10.1117/12.2540770, 2019. a
Qin, W., Fasnacht, Z., Haffner, D., Vasilkov, A., Joiner, J., Krotkov, N., Fisher, B., and Spurr, R.: A geometry-dependent surface Lambertian-equivalent reflectivity product for UV–Vis retrievals – Part 1: Evaluation over land surfaces using measurements from OMI at 466 nm, Atmos. Meas. Tech., 12, 3997–4017, https://doi.org/10.5194/amt-12-3997-2019, 2019. a
Richter, A., Begoin, M., Hilboll, A., and Burrows, J. P.: An improved NO2 retrieval for the GOME-2 satellite instrument, Atmos. Meas. Tech., 4, 1147–1159, https://doi.org/10.5194/amt-4-1147-2011, 2011. a
Richter, A., Hilboll, A., and Burrows, J. P.: A multi-wavelength retrieval of tropospheric NO2 from GOME-2, Deutsche Physikalische Gesellschaft meeting, 17–21 March 2014, Berlin, Germany, http://www.iup.uni-bremen.de/doas/posters/dpg_2014_richter.pdf (last access: 1 August 2022), 2014. a, b
Schenkeveld, V. M. E., Jaross, G., Marchenko, S., Haffner, D., Kleipool, Q. L., Rozemeijer, N. C., Veefkind, J. P., and Levelt, P. F.: In-flight performance of the Ozone Monitoring Instrument, Atmos. Meas. Tech., 10, 1957–1986, https://doi.org/10.5194/amt-10-1957-2017, 2017. a
Shindell, D. T., Faluvegi, G., Koch, D. M., Schmidt, G. A., Unger, N., and Bauer, S. E.: Improved attribution of climate forcing to emissions, Science, 326, 716–718, https://doi.org/10.1126/science.1174760, 2009. a
Tack, F., Merlaud, A., Iordache, M.-D., Danckaert, T., Yu, H., Fayt, C., Meuleman, K., Deutsch, F., Fierens, F., and Van Roozendael, M.: High-resolution mapping of the NO2 spatial distribution over Belgian urban areas based on airborne APEX remote sensing, Atmos. Meas. Tech., 10, 1665–1688, https://doi.org/10.5194/amt-10-1665-2017, 2017. a, b
Tzortziou, M., Herman, J. R., Ahmad, Z., Loughner, C. P., Abuhassan, N., and Cede, A.: Atmospheric NO2 dynamics and impact on ocean color retrievals in urban nearshore regions, J. Geophys. Res.-Oceans, 119, 3834–3854, https://doi.org/10.1002/2014JC009803, 2014. a
Valks, P., Pinardi, G., Richter, A., Lambert, J.-C., Hao, N., Loyola, D., Van Roozendael, M., and Emmadi, S.: Operational total and tropospheric NO2 column retrieval for GOME-2, Atmos. Meas. Tech., 4, 1491–1514, https://doi.org/10.5194/amt-4-1491-2011, 2011. a
Vandaele, A. C., Hermans, C., Simon, P. C., Carleer, M. R., Colin, R., Fally, S., Merienne, M. F., Jenouvrier, A., and Coquart, B.: Measurements of the NO2 absorption cross-section from 42 000 cm−1 to 10 000 cm−1 (238–1000 nm) at 220 K and 294 K, J. Quant. Spectrosc. Ra., 59, 171–184, 1998. a, b
van Geffen, J., Boersma, K. F., Eskes, H., Sneep, M., ter Linden, M., Zara, M., and Veefkind, J. P.: S5P TROPOMI NO2 slant column retrieval: method, stability, uncertainties and comparisons with OMI, Atmos. Meas. Tech., 13, 1315–1335, https://doi.org/10.5194/amt-13-1315-2020, 2020. a, b, c
van Geffen, J., Eskes, H., Compernolle, S., Pinardi, G., Verhoelst, T., Lambert, J.-C., Sneep, M., ter Linden, M., Ludewig, A., Boersma, K. F., and Veefkind, J. P.: Sentinel-5P TROPOMI NO2 retrieval: impact of version v2.2 improvements and comparisons with OMI and ground-based data, Atmos. Meas. Tech., 15, 2037–2060, https://doi.org/10.5194/amt-15-2037-2022, 2022. a
van Geffen, J. H. G. M., Boersma, K. F., Van Roozendael, M., Hendrick, F., Mahieu, E., De Smedt, I., Sneep, M., and Veefkind, J. P.: Improved spectral fitting of nitrogen dioxide from OMI in the 405–465 nm window, Atmos. Meas. Tech., 8, 1685–1699, https://doi.org/10.5194/amt-8-1685-2015, 2015. a
Vasilkov, A., Qin, W., Krotkov, N., Lamsal, L., Spurr, R., Haffner, D., Joiner, J., Yang, E.-S., and Marchenko, S.: Accounting for the effects of surface BRDF on satellite cloud and trace-gas retrievals: a new approach based on geometry-dependent Lambertian equivalent reflectivity applied to OMI algorithms, Atmos. Meas. Tech., 10, 333–349, https://doi.org/10.5194/amt-10-333-2017, 2017. a
Vasilkov, A., Yang, E.-S., Marchenko, S., Qin, W., Lamsal, L., Joiner, J., Krotkov, N., Haffner, D., Bhartia, P. K., and Spurr, R.: A cloud algorithm based on the O2-O2 477 nm absorption band featuring an advanced spectral fitting method and the use of surface geometry-dependent Lambertian-equivalent reflectivity, Atmos. Meas. Tech., 11, 4093–4107, https://doi.org/10.5194/amt-11-4093-2018, 2018. a
Veefkind, J., Aben, I., McMullan, K., Förster, H., de Vries, J., Otter, G., Claas, J., Eskes, H., de Haan, J., Kleipool, Q., van Weele, M., Hasekamp, O., Hoogeveen, R., Landgraf, J., Snel, R., Tol, P., Ingmann, P., Voors, R., Kruizinga, B., Vink, R., Visser, H., and Levelt, P.: TROPOMI on the ESA Sentinel-5 Precursor: A GMES mission for global observations of the atmospheric composition for climate, air quality and ozone layer applications, Remote Sens. Environ., 120, 70–83, https://doi.org/10.1016/j.rse.2011.09.027, 2012. a
Werdell, P. J., Behrenfeld, M. J., Bontempi, P. S., Boss, E., Cairns, B., Davis, G. T., Franz, B. A., Gliese, U. B., Gorman, E. T., Hasekamp, O., Knobelspiesse, K. D., Mannino, A., Martins, J. V., McClain, C. R., Meister, G., and Remer, L. A.: The Plankton, Aerosol, Cloud, ocean Ecosystem mission: Status, science, advances, B. Am. Meteorol. Soc., 100, 1775–1794, https://doi.org/10.1175/BAMS-D-18-0056.1, 2019. a, b, c, d
Yang, K., Carn, S. A., Ge, C., Wang, J., and Dickerson, R. R.: Advancing measurements of tropospheric NO2 from space: New algorithm and first global results from OMPS, Geophys. Res. Lett., 41, 4777–4786, https://doi.org/10.1002/2014GL060136, 2014. a, b
Zakharova, S., Davydova, M., Borovski, A., Shukurov, K., Mukhartova, Y., Makarenkov, A., and Postylyakov, O.: Experiments on high-detailed mapping of tropospheric NO2 using GSA/Resurs-P observations: results, validation with models and measurements, estimation of emission power, in: Remote Sensing of Clouds and the Atmosphere XXVI, edited by: Comerón, A., Kassianov, E. I., Schäfer, K., Picard, R. H., Weber, K., and Singh, U. N., International Society for Optics and Photonics, SPIE, vol. 11859, 1185905, https://doi.org/10.1117/12.2600316, 2021. a
Zoogman, P., Liu, X., Suleiman, R. M., Pennington, W. F., Flittner, D. E., Al-Saadi, J. A., Hilton, B. B., Nicks, D. K., Newchurch, M. J., Carr, J. L., Janz, S. J., Andraschko, M. R., Arola, A., Baker, B. D., Canova, B. P., Miller, C. C., Cohen, R. C., Davis, J. E., Dussault, M. E., Edwards, D. P., Fishman, J., Ghulam, A., Abad, G. G., Grutter, M., Herman, J. R., Houck, J., Jacob, D. J., Joiner, J., Kerridge, B. J., Kim, J., Krotkov, N. A., Lamsal, L., Li, C., Lindfors, A., Martin, R. V., McElroy, C. T., McLinden, C., Natraj, V., Neil, D. O., Nowlan, C. R., O'Sullivan, E. J., Palmer, P. I., Pierce, R. B., Pippin, M. R., Saiz-Lopez, A., Spurr, R. J. D., Szykman, J. J., Torres, O., Veefkind, J. P., Veihelmann, B., Wang, H., Wang, J., and Chance, K.: Tropospheric emissions: Monitoring of pollution (TEMPO), J. Quant. Spectrosc. Ra., 186, 17–39, https://doi.org/10.1016/j.jqsrt.2016.05.008, 2017. a