Nitrogen oxides (NOx≡NO+NO2) in the upper troposphere (UT; ∼8–12 km) influence the oxidizing capacity of the atmosphere and global climate, as the formation and radiative forcing of tropospheric ozone are most efficient in the predominantly NOx-limited UT (Mickley et al., 1999; Bradshaw et al., 2000; Dahlmann et al., 2011; Worden et al., 2011). Sources of NOx to the UT include local emissions from lightning and cruising altitude aircraft, deep convective uplift of surface pollution, downwelling from the stratosphere, long-range transport, and chemical recycling of NOx from stable reservoir compounds (Ehhalt et al., 1992; Lamarque et al., 1996; Schumann, 1997; Jaeglé et al., 1998; Bradshaw et al., 2000; Bertram et al., 2007). The lifetime of NOx in the UT varies from a few hours to a few days depending on availability of hydrogen oxides (HOx≡OH+HO2) and peroxy radicals (RO2) to convert NOx to reservoir compounds (Jaeglé et al., 1998; Bradshaw et al., 2000; Nault et al., 2016).

Current understanding of UT NOx is erroneous, as demonstrated by misrepresentation in chemical transport models (CTMs) of the vertical distribution, relative abundance (ratios of NO / NO2), and absolute magnitude of UT NOx when compared to in situ measurements from research aircraft (Boersma et al., 2011; Travis et al., 2016; Silvern et al., 2018). Models are used to determine the contribution of ozone to anthropogenic climate change in the absence of reliable historical measurements (Pavelin et al., 1999). Models also provide prior information about the vertical distribution of NO2 for retrieval of vertical column densities of NO2 from space-based UV–visible instruments. Errors in these retrievals are particularly vulnerable to biases in modelled UT NO2, due to greater sensitivity of space-based observations to the UT than the middle and lower troposphere (Travis et al., 2016; Silvern et al., 2019). This impedes accurate top-down inference of air quality variability, surface concentrations, and precursor emissions (Stavrakou et al., 2013; Silvern et al., 2019). Models include heavily parameterized representation of lightning (Tost et al., 2007; Allen et al., 2010; Ott et al., 2010; Murray et al., 2012, 2013), the largest global influencer of NOx in the UT (Bradshaw et al., 2000; Marais et al., 2018), and may misrepresent the reaction kinetics and physical processing of NOx for the cold, low-pressure conditions of the UT (Chang et al., 2011; Henderson et al., 2011, 2012; Stavrakou et al., 2013; Amedro et al., 2019).

Observations that have been used to better understand UT NOx are mostly limited to research and commercial aircraft campaigns. For research aircraft, the record of observations in the UT since the early 1990s have been sustained almost exclusively by the NASA DC8 plane, with recent contributions from the German High Altitude and Long Range Research Aircraft (HALO) (Wendisch et al., 2016). There are also commercial aircraft campaigns, but these are mostly limited to heavily trafficked flight corridors that are often in the stratosphere at cruising altitude (Thomas et al., 2015; Stratmann et al., 2016). In situ measurements of NO2 in the UT can also be biased by interference from NOx reservoir compounds that thermally decompose to NO2 in the instrument inlet (Browne et al., 2011; Reed et al., 2016). Standard remote sensing products of NO2 from space-based nadir- and limb-viewing instruments provide global coverage, but either as a single piece of vertical information in the troposphere in the nadir as tropospheric column densities (Levelt et al., 2018) or as vertically resolved NO2 in the limb limited to NO2 abundances above the tropopause (Newchurch et al., 1996; Brogniez et al., 2002; Sioris et al., 2004; Brohede et al., 2007; Jones et al., 2012; Dubé et al., 2021).

Near-global research products of seasonal mean vertically resolved tropospheric NO2 have been retrieved by applying the cloud-slicing technique to partial columns of NO2 from the space-based Ozone Monitoring Instrument (OMI) (Choi et al., 2014; Belmonte Rivas et al., 2015). Cloud-slicing involves regressing clusters of partial NO2 columns above optically thick clouds against corresponding cloud-top pressures. The resultant regression slopes are converted to NO2 mixing ratios that represent average NO2 across the cloud-top altitude range (Ziemke et al., 2001). The advantages of cloud-slicing include enhanced signal over bright optically thick clouds (van der A et al., 2020) and removal of the dry stratosphere due to lack of clouds there. Near-global multiyear (2005–2007) seasonal means of UT NO2 from cloud-sliced OMI partial columns have been shown to reproduce the spatial variability of UT NO2 measured with bias-corrected NASA DC8 aircraft measurements of NO2 over North America, though at very coarse scales (seasonal, 32∘×20∘) (Marais et al., 2018). Even so, the OMI product confirms the dominant global influence of lightning on UT NOx and provides global constraints on lightning NOx production rates (280±80 mol NOx per lightning flash) and annual lightning NOx emissions (5.9±1.7 Tg N) (Marais et al., 2018). OMI pixels are at relatively coarse resolution (13 km × 24 km at nadir), and there is substantial data loss after 2007 due to the so-called row anomaly (Schenkeveld et al., 2017; Torres et al., 2018). The recently launched (October 2017) TROPOMI instrument on the Sentinel-5P satellite has the same spatial coverage as pre-row-anomaly OMI (swath width of 2600 km) but with a finer nadir pixel resolution of 7.2 km × 3.5 km (along track × across track) until 5 August 2019, further refined thereafter to 5.6 km × 3.5 km (Argyrouli et al., 2019). This offers better cloud-resolving capability and greater data pixel density than OMI with potential to retrieve finer-resolution NO2 in the UT.

Here we refine and test the cloud-slicing retrieval using synthetic partial NO2 columns from the GEOS-Chem CTM before retrieving UT NO2 from TROPOMI partial NO2 columns with cloud information from two distinct TROPOMI cloud products. Application of cloud-slicing to TROPOMI follows evaluation of TROPOMI total, stratospheric, and tropospheric columns with ground-based measurements of NO2 from Pandora and multi-axis differential optical absorption spectroscopy (MAX-DOAS) at free-tropospheric monitoring sites. We also evaluate TROPOMI UT NO2 with the OMI UT NO2 product.

Targeting cloudy scenes could yield representation errors in NO2 mixing ratios in the UT, due to the influence of clouds on NOx photochemistry (Holmes et al., 2019), large enhancements in NOx from lightning and convective uplift of surface pollution that accompany cloud formation (Price and Rind, 1992; Bertram et al., 2007), and low sampling frequency due to strict data filtering (Choi et al., 2014). We test the ability of the cloud-slicing technique to return accurate, representative mixing ratios of NO2 in the UT by applying this technique to synthetic partial columns from GEOS-Chem. The “true” NO2 used to evaluate cloud-sliced NO2 is obtained by averaging NO2 across the same vertical range as the cloud-sliced NO2 for the same cloudy model grid squares as are cloud-sliced (true cloudy UT NO2) and for all clear and cloudy model grid squares (true all-sky UT NO2).

Synthetic NO2 is from GEOS-Chem version 12.1.0 (the International GEOS-Chem User Community, 2018) simulated at a horizontal resolution of 0.25∘×0.3125∘ (latitude × longitude) extending over 47 vertical layers from the surface to 0.01 hPa for the nested domains available in version 12.1.0. These include North America (9.75–60∘ N, 130–60∘ W), western Europe (30–70∘ N, 15∘ W–61.25∘ E), and Southeast Asia (15–55∘ N, 70–140∘ E). Dynamic (3-hourly) boundary conditions are from a coarse-resolution (4∘×5∘) global GEOS-Chem simulation. The model is driven with NASA GEOS-FP assimilated meteorology and includes comprehensive emission inventories from anthropogenic and natural sources. These include local emissions of NOx in the UT from lightning as described by Murray et al. (2012) and from aircraft using the Aviation Emissions Inventory Code (AEIC) inventory detailed in Stettler et al. (2011). The model is simulated in boreal summer (June–August) when variability in UT NOx in all nested domains is dominated by lightning (Marais et al., 2018). The model is sampled daily at 12:00–15:00 local time (LT) to be consistent with the TROPOMI overpass time (13:30 LT). Two years (2016 and 2017) are simulated to increase data density. The model years predate TROPOMI, but this has no bearing on assessment of the cloud-slicing technique.

The cloud-slicing approach we apply to synthetic partial columns above synthetic clouds to estimate seasonal means of UT NO2 is the same as will be applied to TROPOMI, so model variables are only used if these are also available in or can be derived from publicly available TROPOMI data products. GEOS-Chem daily partial NO2 column densities (stratosphere + partial troposphere) and the corresponding GEOS-FP cloud-top pressures at 450–180 hPa and 0.25∘×0.3125∘ are gathered into grid squares of the target resolution of 4∘×5∘. Any 4∘×5∘ grids of gathered pixels with non-uniform overlying stratospheric column NO2 are discarded, as diagnosed with a strict stratospheric column NO2 relative standard deviation of 0.02. Variability in stratospheric NO2 is mostly from oxidation of nitrous oxide (N2O) in the mid-stratosphere (Crutzen, 1970). Its variability is dominated by solar insolation and stratospheric circulation but is also influenced by upwelling from the troposphere and downwelling from the mesosphere (Randall et al., 1998; Gruzdev and Elokhov, 2011). GEOS-FP thermal tropopause heights are used to determine the vertical extent of the stratosphere in the model. As many as 256 0.25∘×0.3125∘ partial columns can be gathered in a 4∘×5∘ grid, so we increase the number of possible cloud-sliced NO2 retrievals by subdividing clusters of at least 100 partial NO2 columns into clusters of at least 40. A threshold of 80 instead of 100 yields similar seasonal mean UT NO2. Subdividing the clusters of partial columns doubles the number of cloud-sliced NO2 data used to obtain multiyear seasonal means. Additional filtering is applied to clusters to remove extreme NO2 partial columns (partial columns falling outside the 10th to 90th percentile range) that have a large influence on regression of NO2 partial columns against cloud-top pressures, clusters with fewer than 10 partial columns after screening for extreme values, and clusters that do not extend across a sufficiently wide altitude range (cloud-top pressure range ≤140 hPa and standard deviation ≤30 hPa). GEOS-Chem cloud-top heights are diagnosed in the model as the pressure at the top edge of the highest model layer of GEOS-FP upward moist convective mass flux.

The slope of the relationship between cloud-top heights and partial columns for each cluster is estimated with reduced major axis (RMA) regression, and the standard deviation of the slope, obtained with bootstrap resampling, is used as the error estimate of the slope. Additional filtering is applied to retain slopes that have low relative error (relative error on the slope ≤1.0). We find that a relatively large proportion of slopes (15 %) are negative and would yield negative cloud-sliced UT NO2. Most of these occur over remote regions with low NO2 concentrations. These are diagnosed as slopes significantly less than zero (sum of slope and slope error <0) and removed. The retained slopes and errors (molec. cm-2 hPa-1) are converted to mixing ratios (pptv). The derivation of the mathematical expression for this conversion is detailed in Ziemke et al. (2001). The final step is to remove outliers (UT NO2 >200 pptv) caused by steep slopes. A threshold of 200 pptv is used, as this far exceeds the maximum seasonal mean UT NO2 of 145 pptv in the OMI cloud-sliced UT NO2 product (Marais et al., 2018). We find that only three cloud-sliced retrievals exceed 200 pptv. Seasonal means are obtained by Gaussian-weighting individual estimates of cloud-sliced UT NO2 to the pressure centre (315 hPa). Gaussian weights are calculated as exp⁡-(p‾-315)22×1352, where p‾ is the mean cloud-top pressure of the gathered points used in the cloud-sliced UT NO2 retrieval, 315 hPa is the pressure centre, and 135 hPa is the distance from the pressure centre to the edges.

The cloud-slicing retrieval adopted here is mostly similar to that applied to OMI to estimate mid-tropospheric NO2 at 900–650 hPa (Choi et al., 2014) and UT NO2 at 450–280 hPa (Marais et al., 2018). We extend the ceiling of the retrieval to 180 hPa (∼12.5 km) to better capture the vertical extent of the upper troposphere. Another notable distinction is that the method applied to OMI used vertical gradients of NO2 from the NASA Global Modeling Initiative (GMI) CTM to diagnose scenes with non-uniform NO2, to satisfy the assumption that NO2 is vertically uniform for conversion of regression slopes to mixing ratios (Ziemke et al., 2001; Choi et al., 2014). The threshold used for this is 0.33 pptv hPa-1. We dispense with this step, as its application to TROPOMI requires a model at a similar fine spatial resolution to TROPOMI, and CTMs may underestimate vertical NO2 gradients in the UT (Boersma et al., 2011; Travis et al., 2016; Silvern et al., 2018). Anyway, we find that the strict filtering applied to GEOS-Chem partial columns removes most (85 %) scenes with absolute NO2 vertical gradients ≥0.33 pptv hPa-1 for a target resolution of 4∘×5∘.

Figure 1 shows GEOS-Chem seasonal mean cloud-sliced and true cloudy UT NO2 at 4∘×5∘. The latter is also Gaussian-weighted to 315 hPa. The uncertainty in individual cloud-sliced values, estimated as the RMA regression slope error, ranges from 6 % to the imposed error limit, 99 %. This is reduced to <2 % for the multiyear seasonal means in Fig. 1 due to temporal averaging. Agreement between the cloud-sliced and true cloudy UT NO2 is shown in the scatterplot in Fig. 2. Successful cloud-sliced retrievals can exceed 35 for many grid squares, though these do not exhibit better agreement with the “truth” than the grid squares with fewer (<10) retrievals. The two datasets are spatially consistent (R=0.64) and exhibit similar variance (slope =1.1±0.1). The cloud-sliced UT NO2 has a small positive offset in background UT NO2 (intercept =2.3±1.2 pptv). On average, cloud-sliced UT NO2 is 17 % more than the true cloudy UT NO2, but this depends on the spatial resolution of the retrieved cloud-sliced product. Regression slopes increase from 0.87±0.03 for cloud-sliced UT NO2 obtained at 2∘×2.5∘ to 1.4±0.2 at 8∘×10∘, and the cloud-sliced UT NO2 is 4.1 % less than the true cloudy UT NO2 at 2∘×2.5∘ and 37 % more at 8∘×10∘. The increasingly positive bias with spatial resolution is because a greater relative proportion of clusters with NO2 vertical gradients ≥0.33 pptv hPa-1 are retained at coarser resolution. The percentage retained relative to all clusters with more than 10 data points and uniform overlying stratospheric NO2 increases from 9 % for 2∘×2.5∘ to 15 % for 4∘×5∘ and 23 % for 8∘×10∘. Maps of synthetic cloud-sliced UT NO2 at 2∘×2.5∘ and 8∘×10∘ are in Fig. S1 in the Supplement. Strict data filtering in the cloud-slicing steps removes 90 % of the clusters of GEOS-Chem partial columns for the 4∘×5∘ product. Most (33 %) data loss is due to the strict relative standard deviation threshold applied to stratospheric NO2. Cloud-slicing is very sensitive to this threshold. Relaxing it from a relative standard deviation of 0.02 to 0.03 increases data retention from 10 % to 17 % but increases the positive bias in cloud-sliced UT NO2 from 17 % to 45 %. This is due to an increase in the contribution of variability in the stratosphere to the cloud-slicing regression slopes.

Also shown in Fig. 1 is the true all-sky UT NO2 obtained for all (cloudy and clear) scenes across 450–180 hPa. Model grids with stratospheric influence are identified and removed using GEOS-FP tropopause heights that are updated hourly in the model. The true cloudy UT NO2 is 17 % more than the true all-sky UT NO2. Spatial resolution influences the size of this difference, increasing from 11 % at 2∘×2.5∘ to 22 % at 8∘×10∘. This suggests that isolating cloudy scenes induces a 11 %–22 % bias in seasonal mean NO2 that could be due to a combination of poor data retention (low sampling frequency of cloudy scenes), the influence of clouds on NOx photochemistry (Pour-Biazar et al., 2007; Holmes et al., 2019), and local enhancements in NOx from events like lightning and deep convective uplift of surface pollution that accompany clouds (Crawford et al., 2000; Ridley et al., 2004; Bertram et al., 2007).

Cloud-slicing applied to GEOS-Chem considers all cloudy scenes, whereas cloud-slicing of satellite observations is applied to partial columns above optically thick clouds, so that the clouds shield against contamination of NO2 from below the cloud. If we only consider synthetic partial columns above clouds with a physical (geometric) cloud fraction across 450–180 hPa of at least 0.7, the cloud-sliced UT NO2 positive difference is similar (18 %) to that obtained for all cloudy scenes, but half the number of data are retained. Cloud fractions retrieved with TROPOMI are effective or radiometric cloud fractions that are systematically more than the physical cloud fraction from the model (Bucsela et al., 2013; Laughner et al., 2018). Our results suggest that representation error is not sensitive to the cloud fraction threshold. Another distinction in GEOS-Chem and TROPOMI cloud variables is that the model provides the physical cloud-top height, whereas TROPOMI cloud retrievals that use models that assume clouds are uniform reflective boundaries retrieve cloud-top heights that can be ∼1 km lower than the physical cloud top (Joiner et al., 2012; Choi et al., 2014; Loyola et al., 2018a). We again apply the cloud-slicing algorithm to the simulated partial columns, but with the altitude of the cloud-top heights artificially lowered by 1 km. This approach assumes that the difference in altitudes of effective (radiometric) clouds and physical clouds is systematic and vertically and horizontally uniform. The difference between the resultant cloud-sliced UT NO2 and the true cloudy UT NO2 increases from 17 % to 24 %. This is because a 1 km decrease in cloud-top altitude leads to a larger relative increase in the column above high-altitude clouds than low-altitude clouds, leading to steeper regression slopes and larger UT NO2.

Pandora spectrometer systems provide observations of total and free-tropospheric columns of NO2 using direct sun, direct moon, and sky radiance observations (Herman et al., 2009; Cede et al., 2019). Those at high-altitude sites (>2 km or <800 hPa) have limited influence from the planetary boundary layer (typically extending to 1–2 km altitude) and so are used here to evaluate TROPOMI NO2 total columns of the free troposphere (middle and upper troposphere) and stratosphere. These include long-term Pandora instruments at Mauna Loa, Hawaii (19.48∘ N, 155.60∘ W, 4.2 km above sea level or a.s.l., ∼600 hPa); Izaña, Tenerife, Canary Islands (28.31∘ N, 16.50∘ W, 2.4 km a.s.l., ∼760 hPa); and Altzomoni, Mexico (19.12∘ N, 98.66∘ W, 4.0 km a.s.l., ∼620 hPa). Mauna Loa and Izaña are remote and have limited anthropogenic influence (Toledano et al., 2018), whereas Altzomoni is ∼70 km southeast of Mexico City and is often within the mixed layer of the city in the afternoon (Baumgardner et al., 2009) after the TROPOMI overpass. On average, multiyear mean tropospheric NO2 columns from OMI are ∼10×1015  molec. cm-2 lower over Altzomoni (<5×1015 molec. cm-2) than the city (>15×1015 molec. cm-2) (Rivera et al., 2013). At Izaña, there is also a MAX-DOAS instrument that we use to retrieve free-tropospheric columns of NO2 to assess these from Pandora and TROPOMI. MAX-DOAS offers vertical sensitivity in the troposphere and has been used extensively to determine free-tropospheric concentrations of NO2 at high-altitude sites (Gomez et al., 2014; Gil-Ojeda et al., 2015; Schreier et al., 2016). There are also Pandora instruments at remote sites in the northern middle and high latitudes in Eureka, Canada, and Ny-Ålesund, Norway, but the total columns of these sample the planetary boundary layer. The Eureka site is at 617 m altitude and the Ny-Ålesund site is at 18 m.

Pandora level 2 total and tropospheric columns are from the Pandonia Global Network (PGN, 2020). We use version 1.7 “nvs1” retrieval of total columns and “nvh1” retrieval of tropospheric columns (described below). Observations are for a full year (1 June 2019 to 31 May 2020) at Izaña. The data record is shorter at Mauna Loa (ends 29 March 2020) and Altzomoni (ends 9 March 2020). Total slant columns (NO2 abundances along the instrument viewing path) are retrieved by fitting a fourth-order polynomial to spectra at 400–440 nm using an NO2 effective temperature of 254.4 K. These are then converted to total vertical column densities by accounting for the geometry of the viewing path (Cede et al., 2019). The Pandora tropospheric NO2 columns have not yet been validated against other observations. Retrieval of these involves simultaneous retrieval of slant columns of NO2 and the O2-O2 dimer at multiple elevation angles (typically 0, 60, 75, 88, and 89∘). The O2-O2 dimer slant columns are used to calculate a representative air mass factor (AMF) that is applied to the difference in NO2 slant columns at multiple pointing elevation angles to calculate a tropospheric vertical column. The data also include estimates of the uncertainty on the total and tropospheric columns due to instrument noise and atmospheric variability (Cede et al., 2019). The NO2 effective temperature used in the total NO2 column retrieval is greater than the column average ambient temperature at high-altitude sites. This induces a positive bias in the total columns estimated by Verhoelst et al. (2021) to be ∼10 % that we address by decreasing the Pandora total columns and associated errors by 10 %. No correction is applied to the tropospheric columns, due to variable contribution of the troposphere to the total column.

MAX-DOAS vertical tropospheric columns of NO2 at Izaña are from RASAS-II sky radiance spectra for June 2019 to February 2020. The spectra are fitted for NO2 and O2-O2 in the wavelength range 425–490 nm, and slant columns are calculated as the difference between these spectra at high-sun (90∘ instrument elevation angle) and multiple elevation angles (1, 2, 3, 5, 10, 30, and 70∘) (Hönninger et al., 2004; Gil et al., 2008; Puentedura et al., 2012; Gomez et al., 2014; Gil-Ojeda et al., 2015). Vertical columns are estimated using optimal estimation that solves an ill-constrained problem by introducing prior information (Rodgers, 2000). Prior information for Izaña includes fixed (with altitude) aerosol extinction of 0.01 km-1 and NO2 of 20 pptv from the surface to the tropopause. Aerosol abundances at Izaña are sometimes influenced by windblown dust from the Sahara Desert, but are typically low (aerosol optical depth or AOD <0.05) (Gomez et al., 2014; Gil-Ojeda et al., 2015). The prior NO2 profile is within the range of background NO2 in the UT (10–20 pptv) (Marais et al., 2018) and MAX-DOAS NO2 concentrations previously retrieved at Izaña (20–40 pptv) (Gomez et al., 2014). Filtering is applied to remove vertical column retrievals with limited independent information (degrees of freedom for signal <1) and significant light path attenuation by aerosols (AOD >0.1) and clouds (effective cloud fraction >0.5). AOD is derived with MAX-DOAS O2-O2 dimer differential slant columns retrieved over the same wavelength range as NO2 (Frieß et al., 2006), and cloud fraction is from the Fast Retrieval Scheme for Clouds from the Oxygen A band version S (FRESCO-S) product provided with the TROPOMI NO2 product. Filtering removes 40 % of the retrieved vertical tropospheric NO2 columns at Izaña.

TROPOMI data are from the Sentinel-5P Pre-Operations Data Hub (S5P Data Hub, 2020). We use a year of NO2 data (1 June 2019 to 31 May 2020) from the level-2 offline (OFFL) product version 01-03-02. The data product includes NO2 abundances along the optical path from the sun to the instrument (the total slant column or SCDtot), NO2 vertical column densities in the stratosphere (VCDstrat), and the stratospheric air mass factor (AMFstrat). A detailed description of retrieval of SCDtot and VCDstrat is described in the product Algorithm Theoretical Basis Document (van Geffen et al., 2019) and by van Geffen et al. (2020). In brief, SCDtot values are obtained by spectral fitting of TROPOMI top-of-atmosphere radiances at 405–465 nm by accounting for light absorption by NO2 and other relevant gases. VCDstrat are from assimilation of TROPOMI and modelled total slant columns over locations diagnosed by the model to have limited tropospheric influence (predominantly remote oceans) (Boersma et al., 2004; Dirksen et al., 2011; van Geffen et al., 2019). The modelled slant columns are the product of vertical columns from the TM5-MP CTM (Williams et al., 2017) and AMFs calculated using TROPOMI viewing geometries and surface reflectivities. The CTM is simulated at 1∘×1∘ and driven with ECMWF meteorology updated every 3-hourly. SCDtot values are separated into a stratospheric (SCDstrat) and tropospheric (SCDtrop) component, and a tropospheric AMF (AMFtrop) is applied to SCDtrop to obtain tropospheric vertical columns (VCDtrop). AMFtrop accounts for viewing geometry, surface reflectivity, atmospheric absorption and scattering of light by trace gases and aerosols, and sensitivity to the vertical distribution of NO2. A vertically resolved correction is also applied to the AMFtrop to correct for the fixed NO2 effective temperature (220 K) used to retrieve SCDtot. The light path in the UT is relatively unobstructed by aerosols and, for cloud-slicing, would mostly be impacted by treatment of the reflectivity of optically thick clouds. We choose to use an AMF that only accounts for viewing geometry (AMFtrop,geo) due to uncertainties in the modelled vertical distribution of NO2 in the UT (Stavrakou et al., 2013; Travis et al., 2016) and representation errors from a model at coarser resolution (∼100 km) than TROPOMI (<10 km at nadir). Choi et al. (2014) found that OMI partial NO2 columns calculated with AMFtrop,geo above optically thick clouds in the mid-troposphere (650 hPa) were at most 14 % more than those calculated with a detailed AMF that assumed clouds are near-Lambertian surfaces with albedo of 0.8 and NO2 is constant with altitude. The effect of not including a temperature correction will be small in the UT where temperatures are ∼220 K anyway. To confirm this, we find that GEOS-Chem cloud-sliced UT NO2 values calculated with the TROPOMI AMF temperature correction expression in van Geffen et al. (2019) are only 6 % less than those in Figs. 1 and 2.

We calculate VCDtrop by first obtaining SCDtrop as the difference between SCDtot from the data product and SCDstrat calculated as the product of the reported VCDstrat and AMFstrat: 1SCDtrop=SCDtot-VCDstrat×AMFstrat. This we use to estimate the above-cloud VCDtrop using AMFtrop,geo that we calculate with the reported solar zenith angle (SZA) and viewing zenith angle (VZA) values: 2VCDtrop=SCDtropAMFtrop,geo=SCDtropsecSZA+secVZA. The TROPOMI VCDtot values we compare to Pandora are calculated as the sum of reported VCDstrat and our calculated VCDtrop (Eq. 2). Only data with quality flags (“qa_value” in the data product) of at least 0.45 are used. This removes data affected by sun glint, poor precision in the retrieval and radiances, and SZA >84.5∘ (van Geffen et al., 2019). Similarly, good-quality Pandora retrievals of total and tropospheric columns are identified as those with data quality flags of 0, 1, 10, or 11 (Cede et al., 2019), consistent with Ialongo et al. (2020). Coincident satellite and ground-based data are identified as TROPOMI pixels within a 0.2∘ radius (∼20 km) of the station and ground-based data ±30 min around the TROPOMI overpass.

Figure 3a–c compare collocated daily mean Pandora and TROPOMI total columns. This compares predominantly clear-sky observations from Pandora and all-sky observations from TROPOMI, as the larger sampling footprint of TROPOMI will include influence from clouds. TROPOMI cloud fractions from the FRESCO-S product coincident with Pandora range from cloud free (<0.1) to cloudy (0.8 for Mauna Loa and Izaña, 0.5 for Altzomoni) and average 0.3 at Mauna Loa and Izaña and 0.2 at Altzomoni. Errors on the daily means, obtained by adding in quadrature reported uncertainties of individual columns, are small at all sites. These vary from 0.1 % to 19 % for Pandora and 1.5 % to 16 % for TROPOMI. TROPOMI and Pandora total columns are temporally consistent (R=0.69 at Mauna Loa, R=0.87 at Izaña, R=0.67 at Altzomoni), but there is a systematic positive offset in TROPOMI (intercepts in Fig. 3a–c) ranging from 6.6×1014 molec. cm-2 at Mauna Loa to 9.3×1014 molec. cm-2 at Altzomoni. TROPOMI is on average 18 % higher than Pandora at Mauna Loa, 26 % at Izaña, and 38 % at Altzomoni. Verhoelst et al. (2021) also report a positive bias in TROPOMI total columns at the same Pandora sites of 6 % at Mauna Loa, 19 % at Izaña, and 28 % at Altzomoni for April 2018 to February 2020. Our higher values compared to Verhoelst et al. (2021) is because of the 10 % downward adjustment we apply to Pandora total columns. The difference in sampling footprints of space- and ground-based instruments can influence agreement between the two (Pinardi et al., 2020). We find though that the difference between TROPOMI and Pandora at Mauna Loa and Izaña is relatively insensitive to the choice of sampling coincidence. The difference is 17 %–20 % at Mauna Loa and 25 %–26 % at Izaña for a TROPOMI sampling radius range of 0.05–0.3∘ and for a Pandora sampling time window range of ±15–60 min. The comparison at Altzomoni, though, is very sensitive to the sampling radius due to proximity to Mexico City. There the difference increases from 22 % at 0.05∘ for 45 coincident points to 48 % at 0.3∘ for 76 coincident points.

At Mauna Loa, the free-tropospheric column contribution to the total averages 5.1 % (range of 0.2 %–16 %), according to Pandora. This is likely an overestimate, as few (33 %) Pandora tropospheric daily means could be estimated, as indicated by the grey points in Fig. 3a. The contribution of the free troposphere to the total column at the other sites is 8.3 % (0.2 %–38 %) at Izaña and 31 % (8 %–91 %) at Altzomoni. Due to the small contribution of the free troposphere to the total at Mauna Loa, we use the comparison of Pandora and TROPOMI at this site to evaluate the TROPOMI stratospheric column, as has been done previously (Verhoelst et al., 2021). At Mauna Loa, TROPOMI underestimates stratospheric NO2 variance by 13 % (slope =0.87±0.05). A regression slope less than unity suggests that higher-resolution features are smoothed by the coarser spatial resolution of the TM5-TMP model (1∘×1∘) and time resolution of the meteorology (3-hourly). The underestimate in stratospheric NO2 variance would lead to an overestimate in the relative contribution of the stratosphere to the total column for small column densities and vice versa. This would result in an overall overestimate in cloud-sliced UT NO2, increasing with decline in UT NO2 values, as the relative overestimate in column densities will be greater for high-altitude clouds than for low-altitude clouds. The 18 % higher TROPOMI than Pandora total columns at Mauna Loa are larger than and opposite in sign to the <10 % (-2×1014 molec. cm-2) meridional difference in TROPOMI stratospheric columns from the near-real-time (NRTI) NO2 product and those obtained with twilight measurements from the near-global Système d'Analyse par Observation Zénitale (SAOZ) network of zenith-scattered-light differential optical absorption spectroscopy (ZSL-DOAS) instruments (Lambert et al., 2019). The implied difference between SAOZ and Pandora stratospheric columns coincident with TROPOMI (Pandora < SAOZ) may be due to the need to account for time differences between the SAOZ measurements (twilight) and TROPOMI (midday) (Verhoelst et al., 2021). This difference warrants further investigation, as these ground-based measurements are crucial for validating space-based sensors that measure NO2.

We apply a variance correction to TROPOMI stratospheric columns by dividing these by 0.87 (the slope of the TROPOMI versus Pandora total columns at Mauna Loa in Fig. 3a) and subtracting the resultant mean increase in TROPOMI stratospheric columns of 3×1014 molec. cm-2. This reduces the intercepts in Fig. 3a–c to 4.4×1014 molec. cm-2 for Mauna Loa, 7.9×1014 molec. cm-2 for Izaña, and 7.3×1014 molec. cm-2 for Altzomoni (not shown). Likely causes for the remaining discrepancy between TROPOMI and Pandora could be a positive offset in the TROPOMI radiance intensity that is estimated to be 5 % of the total column or 0.1–1×1015 molec. cm-2 (van Geffen et al., 2020), challenges obtaining a Pandora reference measurement (atmospheric column without NO2) (Herman et al., 2009), and an overestimate in TROPOMI free-tropospheric NO2. The radiance intensity offset has been shown to mostly affect retrievals over open oceans (van Geffen et al., 2020), and an overestimate in free-tropospheric NO2 would have a larger effect on the total column comparison at Izaña and Altzomoni than at Mauna Loa.

Figure 4 compares time series of free-tropospheric column densities of NO2 at Izaña from Pandora, MAX-DOAS, and TROPOMI. As with the total columns, Pandora and MAX-DOAS are sampled 30 min around the satellite overpass and TROPOMI 0.2∘ around the site. We impose a modest threshold to only use TROPOMI tropospheric columns >4×1013 molec. cm-2 to mimic the detection limits of the instruments (Gomez et al., 2014) and mitigate the influence of TROPOMI data that would be susceptible to errors in distinguishing the stratosphere from the troposphere. This brings the lower-end TROPOMI values into better agreement with the ground-based values and has no effect on TROPOMI columns >2×1014 molec. cm-2. On average, Pandora is 14 % more than MAX-DOAS for coincident midday daily means, and the temporal correlation is modest (R=0.4). Temporal inconsistencies between Pandora and MAX-DOAS are due to challenges retrieving tropospheric columns routinely close to instrument detection limits (Gomez et al., 2014), lack of dynamic variability in the retrieved columns, and differences in the sampling extent of the two instruments. The MAX-DOAS sampling footprint, for example, shifts by at least 4∘ in latitude between its most northerly extent in winter solstices to its most southerly extent in summer solstices (Robles-Gonzalez et al., 2016). Most MAX-DOAS and Pandora data are at 1–4×1014 molec. cm-2, whereas the range for TROPOMI calculated using Eqs. (1) and (2) extends to ∼8×1014 molec. cm-2. The range is the same as the comparison of TROPOMI and shipborne MAX-DOAS tropospheric columns by P. Wang et al. (2020). In that study, TROPOMI was on average 4×1014 molec. cm-2 more than MAX-DOAS. In our comparison, TROPOMI free-tropospheric columns (red crosses in Fig. 4) are 77 % more than Pandora and 84 % more than MAX-DOAS. The overestimate is similar if the reported detailed tropospheric AMF is used instead of AMFtrop,geo (Eq. 2) to calculate TROPOMI tropospheric columns.

The stratospheric variance correction reduces the difference between TROPOMI and coincident ground-based measurements to 40 % compared to Pandora and 47 % compared to MAX-DOAS (not shown). This is due to an increase in the relative contribution of the stratosphere to total columns >2×1015 molec. cm-2 following the stratospheric column variance correction. A 40 %–47 % overestimate in free-tropospheric columns would induce a systematic positive bias in cloud-sliced UT NO2. To address the remaining difference between TROPOMI free-tropospheric columns and the ground-based observations, we decrease TROPOMI free-tropospheric columns by 50 % (red circles in Fig. 4), leading to a difference of -4 % with Pandora and 1 % with MAX-DOAS. There is no temporal correlation between daily coincident observations of TROPOMI and the ground-based measurements (R<0.1), consistent with the comparison of TROPOMI to shipborne MAX-DOAS (P. Wang et al., 2020).

Figure 3d–f compare Pandora to TROPOMI total columns after increasing TROPOMI stratospheric column variance by 13 % and reducing TROPOMI free-tropospheric columns by 50 %. This correction reduces the difference between TROPOMI and Pandora by just 3 percentage points at Mauna Loa and Izaña and 11 percentage points at Altzomoni. The variance at Altzomoni degrades from 0.96±0.16 to 0.82±0.14, but this is because the relatively few coincident points (76 compared to 308 at Izaña) are influenced by the single Pandora observation >5.5×1015 molec. cm-2 (coincident corrected TROPOMI is <4×1015 molec. cm-2) that may be detecting NO2 from fires that typically occur in December–February in the National Park where the instrument is located (Bravo et al., 2002; Baumgardner et al., 2009). The TROPOMI free-tropospheric column contribution at Mauna Loa and Izaña is more consistent with that from Pandora after applying the stratospheric and free-tropospheric column corrections, decreasing from 8 % to 6 % at Mauna Loa and 12 % to 7 % at Izaña. This is not the case for Altzomoni (decrease from 14 % to 9 %), due to anthropogenic influence from Mexico City that the coarser TROPOMI sampling footprint is not able to resolve. Points in Fig. 3 are coloured by season to show that all sites experience a modest decline in NO2 from summer (purple) to winter (cyan) due to the influence of solar variability on photochemical production of NOx in the stratosphere (Gil et al., 2008; Robles-Gonzalez et al., 2016) and seasonality in long-range transport and subsidence in the free troposphere (Gil-Ojeda et al., 2015). The distinct distribution of points in December–February compared to June–August and September–November at Mauna Loa suggests there may be seasonality in the size of the discrepancy between TROPOMI and Pandora stratospheric columns. The remaining TROPOMI positive offset of ∼4×1014 molec. cm-2 is consistent with the 2–4×1014 molec. cm-2 positive offset in TROPOMI stratospheric columns reported by P. Wang et al. (2020) from comparison to shipborne MAX-DOAS measurements. If the remaining offset is exclusively due to the stratospheric column, this would cancel in the cloud-slicing retrieval for clusters of partial columns with uniform stratospheric NO2.

In what follows, we use TROPOMI total columns with a 13 % increase in stratospheric column variance and a 50 % decrease in free-tropospheric columns that are based on comparison to Pandora and MAX-DOAS.

Seasonal mean UT NO2 values are obtained from TROPOMI for June 2019 to May 2020 at 1∘×1∘. The same cloud-slicing retrieval and data-filtering steps applied to synthetic spectra from GEOS-Chem (Sect. 2) are applied to corrected TROPOMI total columns. The corrections include a 13 % adjustment to the stratospheric column variance and 50 % decrease in tropospheric columns based on comparison to the three high-altitude Pandora sites (Sect. 3). The degradation in resolution of TROPOMI nadir pixels from 5.6 km × 3.5 km to the target resolution (1∘×1∘) is 400-fold compared to 250-fold for the synthetic experiment in Sect. 2 and a much greater (1300-fold) degradation in OMI nadir pixel resolution (13 km × 24 km) for the 5∘×8∘ product (Marais et al., 2018). The finer relative resolution we choose for TROPOMI cloud-sliced UT NO2 compared to OMI is informed by the synthetic experiment applied to GEOS-Chem and the superior cloud-resolving capability of TROPOMI than OMI. Cloud-slicing is applied to partial columns above optically thick clouds, diagnosed with an effective cloud fraction ≥0.7, as in Marais et al. (2018), to limit contamination from light transmitted through clouds. Though the cloud-slicing retrieval steps applied to GEOS-Chem and TROPOMI are the same, there are differences in the modelled and retrieved cloud parameters that we discuss below.

Two TROPOMI cloud-sliced UT NO2 products are derived using cloud-top heights and cloud fractions from distinct cloud products. The first are FRESCO-S cloud fractions and cloud-top pressures from the same data file as TROPOMI NO2. These are determined by minimizing the difference between measured and simulated spectra in the O2 A-band (758–766 nm) using lookup tables and assuming clouds are single layer Lambertian reflectors with an albedo of 0.8 (Wang et al., 2008; van Geffen et al., 2019). The second is the Optical Cloud Recognition Algorithm (OCRA) cloud fractions and Retrieval of Cloud Information using Neural Networks (ROCINN) cloud-top heights. OCRA cloud fractions are retrieved by determining the difference, in colour space, between cloudy and clear reflectances (Loyola et al., 2007, 2018a, b). ROCINN cloud-top heights are retrieved in kilometres by minimizing the difference between measured O2 A-band radiances and neural-network-trained radiances modelled using OCRA cloud fractions (Loyola et al., 2007, 2018b). In ROCINN, clouds are modelled as multiple optically uniform layers of light-scattering water droplets (the clouds-as-layers or CAL model). We convert the ROCINN-clouds-as-layers (ROCINN-CAL) cloud-top heights to pressures for cloud-slicing and comparison to FRESCO-S. FRESCO-S data are quality screened using the same qa_value threshold (0.45) as NO2. A qa_value threshold of 0.5 is used for OCRA and ROCINN-CAL. Snow and/or ice scenes potentially misclassified as clouds by FRESCO-S are identified as differences in reported scene and surface pressures >2 %, as in van der A et al. (2020), as having snow cover >80 % or classified as permanent ice cover. In what follows, we distinguish the two cloud-sliced TROPOMI UT NO2 products as FRESCO-S and ROCINN-CAL.

Figure 5 shows maps of seasonal mean FRESCO-S and ROCINN-CAL UT NO2 at 1∘×1∘. The spatial features are consistent with a combination of the density of lightning flashes (Cecil et al., 2014) and lightning properties such as flash footprint, duration, and energy (Beirle et al., 2014). These include elevated concentrations (>80 pptv) over Northern Hemisphere land masses in June–August, the year-round 40–60 pptv band over tropical landmasses that shifts meridionally with the Intertropical Convergence Zone (ITCZ), and relatively low concentrations (<30 pptv) over the remote Pacific Ocean. We find reasonable spatial correlation (R=0.4–0.6) between seasonal mean TROPOMI UT NO2 and lightning flash densities from the Lightning Imaging Sensor–Optical Transient Detector (LIS–OTD) in all seasons, except September–November for FRESCO-S (R=0.34). Seasonal LIS–OTD lightning flash densities at 1∘×1∘ are calculated from version 2.3 of the monthly climatology (OPeNDAP, 2020). RMA slopes from regressing TROPOMI UT NO2 against LIS–OTD lightning flash densities range from 0.3 to 0.6 pptv (106 flashes km-2 min-1)-1, and UT NO2 in the absence of lightning flashes in the tropics and subtropics (RMA intercepts) is very similar, 37–41 pptv, for both products in all seasons. In the cold polar regions in Fig. 5, UT NO2 values, limited to ROCINN-CAL, are near background (<30 pptv) as NO2 is preferentially present as NOx reservoir compounds such as peroxyacetyl nitrates (PANs) (Bottenheim et al., 1986). Large enhancements (NO2>80 pptv) over northern China and the northeast US in June–August and Australia in December–February most prevalent in the ROCINN-CAL product likely reflect contamination from surface pollution below clouds. These result from intense anthropogenic activity in North China and the northeast US (Zhao et al., 2013; Jiang et al., 2018; Z. Wang et al., 2020) and routine pyrocumulonimbus injection of fire plumes into the free troposphere and lower stratosphere during the extreme 2019–2020 fire season in Australia (Kablick et al., 2020).

FRESCO-S and ROCINN-CAL UT NO2 are spatially consistent for coincident grid squares (R of 0.82 to 0.88), though ROCINN-CAL UT NO2 are 4.2–9.1 pptv more than FRESCO-S UT NO2. As with the synthetic experiment, UT NO2 increases with degradation in resolution due to the relative increase in tropospheric columns with steep vertical gradients in NO2. Depending on the season, cloud-sliced UT NO2 values are 2 %–4 % more at 2∘×2.5∘ and 3 %–9 % more at 4∘×5∘ than at 1∘×1∘. Good quality retrievals and optically thick clouds with cloud-top pressures at 450–180 hPa account for ∼2 % of TROPOMI pixels using FRESCO-S and ∼3 % using ROCINN-CAL. Of these, 44 000–78 000 cloud-slicing retrievals are retained in each season for FRESCO-S and 118 000–177 000 for ROCINN-CAL (Fig. 5). Most data loss in the cloud-slicing retrieval is because of too few points (clusters <10) or a cloud-top pressure range <140 hPa. Discarded extreme values of cloud-sliced NO2>200 pptv are only 0.1 %–0.5 % of retained data. More cloud-sliced retrievals with ROCINN-CAL are due to greater abundance of optically thick clouds and clusters with greater cloud height range.

Figure 6 compares the meridional abundance of optically thick clouds in the UT from the two cloud products for June–August and December–February. The same information for the other two seasons is in Fig. S2. Both products retrieve effective (radiometric) cloud fractions. These are systematically more than the physical (geometric) cloud fractions from GEOS-Chem, though the two converge for optically thick clouds with physical cloud fractions approaching 1 (Stammes et al., 2008; Laughner et al., 2018). The number of OCRA optically thick clouds is always more (often double) than that of FRESCO-S in all seasons and across all latitudes. The greatest difference in the number of optically thick clouds tracks the ITCZ and is also typically at 45∘ N/S. The majority (61 %–62 %) of OCRA cloud fractions exceed 0.975 compared to 42 %–45 % for FRESCO-S. Loyola et al. (2018a) determined that OCRA cloud fractions retrieved over oceans are 0.1 unit more than those from retrievals like FRESCO-S that assume fixed cloud albedo. Differences over land are not as systematic and vary from negligible in the tropics and subtropics to >0.1 unit more in the Arctic (Loyola et al., 2018a). ROCINN-CAL retrieves cloud optical thicknesses alongside cloud heights. These exceed 20 for most (84 %–93 %) 1∘×1∘ grid squares used to cloud-slice TROPOMI, confirming that a cloud fraction threshold of 0.7 is sufficient to isolate optically thick clouds. The number of pixels in each cloud fraction threshold in Fig. 6 suggests that a stricter cloud fraction threshold of 1.0 applied to the ROCINN-CAL product might lead to a more consistent spatial distribution of UT NO2 to that from FRESCO-S in Fig. 5. The resultant ROCINN-CAL UT NO2 values using a cloud fraction threshold of 1.0 are in Fig. S3. Half the number of cloud-sliced retrievals are obtained, as expected, and there are fewer retrievals over Northern Hemisphere high latitudes than in Fig. 5. Those over the Southern Ocean in austral autumn and winter persist and may reflect enhanced occurrence of high-altitude clouds in these seasons over Antarctica (Verlinden et al., 2011). The average difference between ROCINN-CAL and FRESCO-S decreases from 5 %–8 % for the same cloud fraction threshold of 0.7 to 0.2 %–1.6 % using a cloud fraction threshold of 1.0 for ROCINN-CAL and 0.7 for FRESCO-S, likely due to reduced contamination of NO2 from below clouds with the stricter cloud fraction threshold.

Figure 7 compares gridded cloud product cloud-top pressures for June–August sampled where FRESCO-S cloud-top pressures are at 450–180 hPa and cloud fractions are at least 0.7. Cloud-top pressures are spatially consistent in the tropics (R=0.62 at 0–35∘ N, R=0.85 at 0–35∘ N) and midlatitudes (R=0.58 at 35–70∘ N, R = 0.63 at 35–70∘ S) but degrade north of 70∘ N (R=0.31). Variability in cloud-top pressures is similar for the two products in the tropics (regional mean standard deviation of 28–33 hPa at 0–35∘ N and 30–31 hPa at 0–35∘ S) but deviates in the subtropics and midlatitudes (18 hPa for FRESCO-S, 24–30 hPa for ROCINN-CAL) and more so in the Arctic (13 hPa for FRESCO-S, 54 hPa for ROCINN-CAL). There are no coincident data south of 70∘ S in June–August. In December–February (Fig. S4) south of 70∘ S there is a similarly weak correlation (R<0.1) and large difference in variability (19 hPa for FRESCO-S, 80 hPa for ROCINN-CAL). FRESCO-S does not account for scattering within and below clouds and so estimates the height as the optical centroid of the cloud (Joiner et al., 2012). The optical centroid is systematically lower in altitude (higher in pressure) than the physical cloud top, though FRESCO-S appears to be more consistent with ground-based observations than ROCINN-CAL for high-altitude cloud-top heights (Compernolle et al., 2020). Loyola et al. (2018a) determined that cloud-top altitudes from ROCINN-CAL were ∼1 km (range: 0.6 to >2 km) higher than those from a FRESCO-S-type approach that assumes clouds are single layers with fixed albedo. Our test of the effect of an artificial decrease in cloud-top altitude of 1 km for cloud-slicing synthetic GEOS-Chem partial columns (Sect. 2) suggests that 1 km lower altitude cloud tops in FRESCO-S should lead to larger UT NO2 than those from ROCINN-CAL, but the opposite is observed (Fig. 5). This suggests that the effect of other differences between the cloud products on the cloud-sliced UT NO2 must dominate. Regression slopes in Fig. 7 are less than unity, indicating that the difference in cloud-top pressures between the two products decreases with pressure (increases with altitude). The implication for cloud-sliced UT NO2 is greater global coverage with ROCINN-CAL, as clusters of TROPOMI pixels using ROCINN-CAL cloud-top pressures in the midlatitudes and polar regions overcome the 140 hPa cloud-top pressure range threshold imposed in the cloud-slicing algorithm. In the tropics and subtropics, ROCINN-CAL has less cloud-top pressure range than FRESCO-S for the same scenes. This leads to steeper cloud-slicing regression slopes for ROCINN-CAL and contributes to the 4–9 pptv greater UT NO2 than FRESCO-S in Fig. 5.

We evaluate TROPOMI UT NO2 with multiyear (2005–2007) seasonal mean cloud-sliced UT NO2 from OMI at 5∘×8∘ (Marais et al., 2018). OMI data are for 2005–2007, as this predates the substantial data loss caused by the row anomaly (Schenkeveld et al., 2017; Torres et al., 2018). The OMI product is retrieved in a similar manner to TROPOMI, except that the GMI CTM is used to diagnose and remove steep gradients in NO2 (≥0.33 pptv hPa-1), and the OMI retrieval ceiling is lower (280 hPa, ∼10 km) than TROPOMI (180 hPa, ∼12.5 km). In regions where lightning is prevalent, the vertical distribution of NO2 increases with altitude by 10–50 pptv across 280–180 hPa, as is observed with vertical profiles of NO2 from spring–summer research aircraft campaign measurements over the US (Boersma et al., 2011; Silvern et al., 2018). Strict filtering applied to cloud-slicing removes most scenes where the increase in NO2 with altitude exceeds 33 pptv across 280–180 hPa, based on the synthetic experiment with GEOS-Chem. The influence of more than a 10-year gap between the OMI and TROPOMI UT NO2 datasets on the comparison is challenging to quantify, due to paucity of routine measurements of NO2 in the UT. The contribution of changes in lightning activity should be small, as interannual variability is small (<5 %), and there is no discernible trend in the long-term record of satellite observations of lightning flashes (Schumann and Huntrieser, 2007). According to most climate models, there is an increase in lightning flashes due to global warming (Romps et al., 2014; Finney et al., 2016), though almost all of these models use the same positive relationship between lightning flashes and cloud-top heights first proposed by Price and Rind (1992). The few models that consider lightning flash dependence on upward mass flux predict a decline in lightning flashes (Finney et al., 2014, 2018).

Figure 8 evaluates spatial consistency between TROPOMI and OMI seasonal mean UT NO2 on the OMI grid (5∘×8∘) for TROPOMI cloud-sliced UT NO2 1∘×1∘ grid squares with at least 10 cloud-sliced retrievals. TROPOMI is spatially consistent with OMI in all seasons for both products (R=0.6–0.8). The TROPOMI background is 12–25 pptv more than OMI for FRESCO-S and 20–26 pptv more than OMI for ROCINN-CAL, based on the intercepts in Fig. 8. This may be due to the observed increase in NO2 with altitude across the sampling pressure ceilings of the two products (280 hPa for OMI, 180 hPa for TROPOMI), but other differences between the products may contribute. The OMI UT NO2 product uses cloud information derived from OMI O2-O2 slant columns. The signal from the O2-O2 dimer declines with altitude, increasing uncertainty in the retrieval with altitude (Veefkind et al., 2016). High-altitude clouds from OMI would have to be higher in altitude than TROPOMI to contribute to the positive offset in TROPOMI UT NO2 in Fig. 8, based on results from the synthetic test of lowering GEOS-Chem cloud-top heights by 1 km. The direction of the bias in OMI high-altitude cloud-top heights compared to lidar–radar measurements also does not appear to be systematic (Veefkind et al., 2016). The retrieval approaches of the TROPOMI (DOMINO product) and OMI (NASA Standard Product) slant columns are also distinct. DOMINO-retrieved NO2 column densities are typically greater than those from NASA (Hains et al., 2010; Dirksen et al., 2011). The regression slopes in Fig. 8 are closest to unity for June–August and March–May for FRESCO-S and September–November for ROCINN-CAL. The weaker variance in December–February in both TROPOMI products compared to OMI could reflect the need to account for seasonality in the stratospheric variance correction but also could be because we derive a stratospheric variance correction for a single site. The discrepancy between OMI and TROPOMI UT NO2 is much greater than in Fig. 8 if no correction factors are applied to TROPOMI (Fig. S5). TROPOMI UT NO2 background concentrations (intercepts from RMA regression of TROPOMI versus OMI) are 16–35 pptv more than OMI for FRESCO-S and 27–36 pptv more for ROCINN-CAL, and slopes exceed unity in all seasons (1.3–1.7 for FRESCO-S, 1.2–1.5 for ROCINN-CAL).

We have developed new products of NO2 in the upper troposphere (UT; ∼8–12 km) by cloud-slicing partial columns of NO2 from the space-based TROPOMI instrument. This involves regressing partial NO2 columns against cloud-top pressures and converting regression slopes to UT NO2 mixing ratios.

We first refined and tested representativeness of cloud-sliced UT NO2 by applying cloud-slicing to synthetic partial columns from the GEOS-Chem model. Synthetic cloud-sliced UT NO2 values are spatially consistent (R=0.64) with the synthetic truth, but preferentially sampling cloudy scenes and substantial data loss contribute to a resolution-dependent positive bias in cloud-sliced UT NO2 of 11 %–22 %.

Before applying cloud-slicing to TROPOMI, we evaluated TROPOMI with Pandora total columns at high-altitude sites (Mauna Loa, Izaña, Altzomoni) and Pandora and MAX-DOAS free-tropospheric columns at Izaña. We identified discrepancies between TROPOMI and ground-based NO2 measurements that include a 13 % underestimate in TROPOMI stratospheric NO2 variance and 50 % overestimate in TROPOMI tropospheric columns that would lead to an overestimate in cloud-sliced UT NO2.

We retrieved UT NO2 from TROPOMI by applying the refined cloud-slicing algorithm to corrected TROPOMI partial columns above optically thick clouds with cloud-top heights at 450–180 hPa using two alternate cloud products, FRESCO-S and ROCINN-CAL. ROCINN-CAL UT NO2 has more extensive coverage (0–70∘ N/S) than FRESCO-S (0–45∘ N/S) due to its greater abundance of optically thick clouds. Coincident UT NO2 values from the two products exhibit similar spatial distribution, but background UT NO2 from ROCINN-CAL is 4–9 pptv more than FRESCO-S. This is due to greater contamination from NO2 below clouds in ROCINN-CAL but also steeper cloud-slicing regression slopes for ROCINN-CAL, as cloud-top heights between the two products deviate with increasing cloud-top pressure. Ongoing validation studies are needed to resolve these differences.

Both products are spatially correlated with the existing coarse resolution (5∘ latitude × 8∘ longitude) Ozone Monitoring Instrument (OMI) product, except that TROPOMI is 16–36 pptv more than OMI, which we reason is due to the widely documented increase in NO2 with altitude from the OMI pressure ceiling (280 hPa) to that for TROPOMI (180 hPa) and differences in NO2 column density retrievals.

TROPOMI UT NO2 products presented here have the potential to provide routine, extensive, and consistent measurements of NOx in the UT and, as TROPOMI observations accumulate, aid in characterizing interannual and long-term variability in NOx in the undersampled UT.