The retrieval of NO2 (sub)columns from TROPOMI Earth nadir radiance and solar irradiance spectra is a three-step process relying on DOAS and on a chemical transport model (CTM)-based stratosphere–troposphere separation. The TROPOMI NO2 algorithm is an adaptation of the QA4ECV community retrieval approach and of the DOMINO/TEMIS algorithm , already applied successfully to heritage and current satellite data records (GOME, SCIAMACHY, OMI, GOME-2). In the first step, the integrated amount of NO2 along the optical path, or slant column density (SCD), is derived using the classical DOAS approach . In the second step, the retrieved SCD is assimilated by the TM5-MP CTM to allocate a vertical profile of the NO2 concentration, needed for the separation between stratospheric and tropospheric SCDs. This assimilation procedure favours observations over pristine, remote areas where the entire NO2 SCD can be attributed to the stratospheric component. Assuming relatively slow changes in the stratospheric NOx field, the model transports information to areas with a more significant tropospheric component. In the third step, the three slant (sub)column densities are converted into vertical (sub)column densities using appropriate air mass factors (AMFs). The CTM can be run either in forecast mode, using 1 d forecast meteorological data from the European Centre for Medium-Range Weather Forecasts (ECMWF), or in a more delayed processing mode, using 0–12 h forecast meteorological data. The former is used for near-real-time (NRTI) processing of the TROPOMI measurements, the latter for the offline (OFFL) production. For full technical details, the reader is referred to the Product Readme File (PRF), Product User Manual (PUM), and Algorithm Theoretical Basis Document (ATBD), all available at http://www.tropomi.eu/data-products/nitrogen-dioxide (last access: 5 January 2021). A detailed description and quality assessment of the derived slant column data have already been published by , and a publication on satellite intercomparison of vertical column data is under preparation . The current paper addresses the independent ground-based validation of vertical subcolumn densities in the troposphere and stratosphere and of the vertical total column. The S5P dataset validated here covers the nominal operational phase (Phase E2) of the S5P mission, starting in April 2018 and up to February 2020. No data obtained during the commissioning phase of the satellite have been used. Table  provides an overview of the processor versions to which this corresponds. They constitute as continuous a dataset as possible from May (NRTI) or October (OFFL) 2018 onwards. Combining interim reprocessing (RPRO) (May–October 2018) with OFFL, a coherent dataset with the OFFL processor v01.02.02 or higher can be obtained.

Besides very detailed quality flags, the S5P NO2 data product includes a combined quality assurance value (qa_value) enabling end users to easily filter data for their own purpose. For tropospheric applications (when not using the averaging kernels), the guideline is to use only NO2 data with a qa_value >0.75. This removes very cloudy scenes (cloud radiance fraction >0.5), snow- or ice-covered scenes, and problematic retrievals. For stratospheric applications, where clouds are less of an issue, a more relaxed threshold of qa_value >0.5 is recommended. These data filtering recommendations have been applied here, where the stricter requirement of qa_value >0.75 has been used for the total column validation as well. Again, further details on this can be found in the PRF, PUM, and ATBD.

Since the pioneering ages of NO2 column measurements from space with ERS-2 GOME in the mid-1990s, ground-based UV–visible DOAS measurements at twilight have served as a reference for the validation of NO2 total column data over unpolluted stations and of NO2 stratospheric column data from all nadir UV–visible satellites to date e.g.. Here as well, S5P TROPOMI stratospheric NO2 column data are compared to the correlative measurements acquired by ZSL-DOAS (Zenith-Scattered-Light Differential Optical Absorption Spectroscopy) UV–visible spectrometers e.g.and references therein. A key property of zenith-sky measurements at twilight is the geometrical enhancement of the optical path in the stratosphere , which offers high sensitivity to stratospheric absorbers of visible radiation and lower sensitivity to clouds and tropospheric species (except in the case of strong pollution events during thunderstorms or thick haze; see, for example, ). However, the geometrical enhancement also implies horizontal smoothing of the measured information over hundreds of kilometres, which requires appropriate co-location methods to avoid large discrepancies with the higher resolution measurements of TROPOMI, as discussed in Sect. . Various ZSL-DOAS UV–visible instruments with standard operating procedures and harmonized retrieval methods perform network operation in the framework of the Network for the Detection of Atmospheric Composition Change (NDACC; ). As part of this, over 15 instruments of the SAOZ design (Système d'Analyse par Observation Zénitale) are distributed worldwide and provide data in near-real time through the CNRS LATMOS_RT Facility . For the current work, ZSL-DOAS validation data have been obtained: (1) through the LATMOS_RT Facility (in near-real-time processing mode), (2) from the NDACC Data Host Facility (DHF), and (3) via private communication with the instrument operator. The geographical distribution of these instruments is shown in Fig. , and further details are provided in Sect. . Measurements are made during twilight, at sunrise, and at sunset, but only sunset measurements are used here for signal-to-noise reasons (larger NO2 column) and as these happen closer in time to the early-afternoon overpass of S5P. NDACC intercomparison campaigns conclude an uncertainty of about 4 %–7 % on the slant column density. After conversion of the slant column into a vertical column using a zenith-sky AMF, and for the latest version of the data processing, the uncertainty on the vertical column is estimated to be on the order of 10 %–14 % . Estimated uncertainties for all ground-based measurement types are summarized in Table . In Sect. , the photochemical adjustment required to correctly compare twilight with midday measurements is described.

Satellite tropospheric NO2 column data are compared classically to correlative measurements acquired by Multi-Axis Differential Optical Absorption Spectroscopy (MAX-DOAS) instruments . From sunrise to sunset, MAX-DOAS instruments measure the UV–visible radiance scattered in several directions and elevation angles, from which the tropospheric vertical column density (VCD) and/or the lowest part of the tropospheric NO2 profile (usually up to 3 km altitude, and up to 10 km at best) can be retrieved through different techniques see, for example,, with between 1 and 3 degrees of freedom. Their horizontal spatial representativeness varies with the aerosol load and the spectral region of the retrieval, from a few kilometres to tens of kilometres . Published total uncertainty estimates on the NO2 tropospheric VCD are of the order of 7 %–17 % in polluted conditions, including both random (around 3 % to 10 %, depending on the instrument) and systematic (11 % to 14 %) contributions . These ranges are more or less confirmed by the uncertainties reported in the data files, as visualized in Fig. . Nevertheless, differences in the reported uncertainties and in the actual measurement of the same scene between individual instruments are sometimes larger, and the main potential sources of these inhomogeneities are summarized below: -

Different uncertainty reporting strategy. The reported systematic uncertainty may include only that from the NO2 cross sections (approx. 3 %; UNAM, BIRA-IASB, MPIC, AUTH, IUPB), or it may include also a contribution from the VCD retrieval step (up to 14 % in JAMSTEC data and 20 % in KNMI data) and the aerosol retrieval Chiba U;.

MAX-DOAS data have been used extensively for tropospheric NO2 satellite validation, for instance for Aura OMI and MetOp GOME-2 e.g. by, as well as for the evaluation of modelling results .

Data are collected either through ESA's Atmospheric Validation Data Centre (EVDC; https://evdc.esa.int/, last access: 5 January 2021) or by direct delivery from the instrument principal investigators (e.g. within the S5PVT NIDFORVAL AO project). Currently, 19 MAX-DOAS stations have contributed correlative data in the TROPOMI measurement period from April 2018 to February 2020. Detailed information about the stations and instruments is provided in Sect. . A few contributing sites measure in several geometries (e.g. Xianghe measure in both MAX-DOAS and direct Sun mode; Bremen and Athens both report MAX-DOAS and zenith-sky measurements) or have multiple instruments (e.g. Cabauw and UNAM stations host both MAX-DOAS and Pandora instruments). This allows for detailed (sub)column consistency checks and in-depth analysis of the site peculiarities, beyond the scope of the present overview paper.

The Pandonia Global Network (PGN) delivers direct Sun total column and multi-axis tropospheric column observations of several trace gases, including NO2, from a network of ground-based standardized Pandora Sun photometers in an automated way. In this work, only direct Sun observations are used. These have a random error uncertainty of about 0.27 Pmoleccm-2 and a systematic error uncertainty of 2.7 Pmoleccm-2 . Studies at US and Korean sites during the DISCOVER-AQ campaign found a good agreement of Pandora instruments with aircraft in situ measurements (within 20 % on average; ), although larger differences are observed for individual sites .

Pandora data have been used before to validate satellite NO2 measurements from Aura OMI and TROPOMI .

For the current work, 25 sites have contributed Pandora data, collected either from the ESA Atmospheric Validation Data Centre (EVDC) (https://evdc.esa.int/, last access: 5 January 2021) or from the PGN data archive (https://pandonia-global-network.org/, last access: 5 January 2021). Only data files from a recent quality upgrade (processor version 1.7, retrieval version nvs1, with file version 004 and 005; see https://www.pandonia-global-network.org/home/documents/release-notes/, last access: 5 January 2021) were used, with 005 files (consolidated data) having precedence over 004 files (rapid delivery data). The most important change with the previous data release is a more stringent quality filtering. A total of 17 sites have provided measurement data newer than 3 months.

Except at low Sun elevation, the footprint of these direct Sun measurements is much smaller than a TROPOMI pixel. Therefore, as is the case with MAX-DOAS, a significant horizontal smoothing difference error can be expected in the TROPOMI–Pandora comparison, especially in the case of tropospheric NO2 gradients and when tropospheric NO2 is the largest contributor to the total column.

Three Pandora instruments (Altzomoni, Izaña, Mauna Loa) are located near the summit of a volcanic peak and are therefore not sensitive to the lower lying tropospheric NO2. In this work, their observations are compared to the TROPOMI stratospheric NO2 data (see Sect. ).

A potential source of inconsistencies between the different data products lies in the NO2 cross sections that are used. An overview of the different choices made is provided in Table . Most products use the cross sections published by , but there are differences in the choice of temperature at which to take the cross sections. The ZSL-DOAS measurements are processed with cross sections at a fixed 220 or 227 K, i.e. typical stratospheric temperatures. MAX-DOAS data are processed either with cross sections at room temperature (298 K, representing a typical tropospheric temperature) or using an orthogonalized set of cross sections at 298 and 220 K when both tropospheric and stratospheric slant columns are retrieved. As the scientific focus of the PGN up until processor version 1.7 (used for this study) was on measuring polluted conditions, i.e. in the presence of moderate to large tropospheric columns, the cross sections used in the processor are scaled to a fixed effective temperature of 254.4 K, which corresponds to the situation of approximately equal column amounts in the troposphere and stratosphere. The S5P retrievals use cross sections at 220 K but with an explicit correction for the temperature dependence of the NO2 cross sections in the AMF: space–time co-located daily ECMWF temperature profile forecasts are used to compute a height-dependent AMF correction factor. The temperature sensitivity parameterized in this correction is approximately 0.32 % K-1 . A posteriori temperature correction of the ground-based data is beyond the scope of this paper, so it must be kept in mind that this may contribute to differences between S5P and ground-based columns. Specifically, we could expect a small seasonal cycle in the stratospheric column comparisons of a few percent due to the seasonal variation in stratospheric temperature not being accounted for in the ZSL-DOAS data processing. PGN columns may either be overestimated by up to 10 % when the column is mostly stratospheric or underestimated by a similar order of magnitude when large tropospheric amounts are present. The MAX-DOAS data may be biased in either direction by a few percent when tropospheric and/or stratospheric temperatures differ strongly from the 298 and 220 K default temperatures.

To reduce mismatch errors due to the significant difference in horizontal sensitivity between S5P and ZSL-DOAS measurements, individual TROPOMI NO2 stratospheric column data (in ground pixels at high horizontal sampling) are averaged over the much larger footprint of the air mass to which the ground-based zenith-sky measurement is sensitive; see , , and for details. The length of this footprint if of the order of 300–600 km in the direction of the Sun, and the width is typically of the order of 50–100 km at mid-latitudes, depending on the duration of sunrise and sunset. Note that, as the TROPOMI stratospheric column is a TM5 output, its true resolution is actually much lower than the pixel size. To account for effects of the photochemical diurnal cycle of stratospheric NO2, the ZSL-DOAS measurements at sunset are adjusted to the early-afternoon S5P overpass time using a model-based correction factor. The latter is calculated with the PSCBOX 1D stacked-box photochemical model , initiated by daily fields from the SLIMCAT chemical transport model (CTM). The amplitude of the adjustment factor is sensitive to the effective solar zenith angle (SZA) assigned to the ZSL-DOAS measurements. It is assumed here to be 89.5∘ or, during polar day and close to polar night, the largest or smallest SZA reached, respectively. This photochemical correction factor is an average based on 10 years of the box-model simulations, and the range of values over these 10 years can be considered an uncertainty estimate. It varies between 1 % and 6 % at the sites considered here, the uncertainty being largest at high latitudes in local winter. This does however not contain any model uncertainty (in the sense of the accuracy of the model in representing the true photochemical variation during the day). Another way to estimate the uncertainty in the adjusted ZSL-DOAS data is by comparing the agreement between sunrise and sunset measurements when both are photochemically adjusted to the S5P overpass time. This does also contain co-location mismatch uncertainty due to transport of air occurring during the period between sunrise and sunset and due to the different air masses that are probed (east or west of the instrument respectively). Moreover, it also contains that part of the measurement uncertainty that is not systematic on a daily (or longer) timescale. We find that sunrise and sunset measurements typically agree within 6 % (standard deviation of the differences). Overall, the 10 %–14 % total uncertainty estimate already presented in Sect.  thus seems realistic.

Figure  illustrates the comparison between TROPOMI and ground-based ZSL-DOAS SAOZ NO2 data at the NDACC station at Observatoire de Haute-Provence (OHP) in southern France. The time series reveal a small negative median difference for TROPOMI, which is found to be a common feature across the network, but little seasonal structure. The correlation coefficient is excellent, and the histogram of the differences has an almost Gaussian shape.

Comparison results for the entire ZSL-DOAS network are presented in Fig. . This figure reveals occasionally larger differences in more difficult co-location conditions (e.g. enhanced variability at the border of the polar vortex) but no impact of the TROPOMI pixel size change on 6 August 2019. The latter result must be interpreted with care as, for these comparisons, multiple TROPOMI pixels are averaged over the ZSL-DOAS observation operator before comparison (see Sect. ), and as such any change in the noise statistics of individual pixels will be hidden.

Statistical estimators of the bias (median difference) and scatter per station are presented in box-and-whisker plots in Fig.  and in tabular form in Sect. . Across the network, S5P NRTI and OFFL stratospheric NO2 column data are generally lower than the ground-based values by approximately 0.2 Pmolec cm-2, with a station–station scatter of this median difference of similar magnitude (0.3 Pmolec cm-2). These numbers are within the mission requirement of a maximum bias of 10 % (equivalent to 0.2–0.4 Pmolec cm-2, depending on latitude and season) and within the combined systemic uncertainty of the reference data and their model-based photochemical adjustment. The IP68/2 dispersion of the difference between TROPOMI stratospheric column and correlative data around their median value rarely exceeds 0.3 Pmolec cm-2 at sites without tropospheric pollution. When combining random errors in the satellite and reference measurements with irreducible co-location mismatch effects, it can be concluded that the random uncertainty on the S5P stratospheric column measurements falls within the mission requirements of max. 0.5 Pmolec cm-2 uncertainty.

The potential dependence of the TROPOMI stratospheric column bias and uncertainty on several influence quantities has been evaluated. Figure  shows results for the solar zenith angle (SZA), the fractional cloud cover (CF), and the surface albedo of the TROPOMI measurement. This evaluation does not reveal any variation of the bias much larger than 0.4 Pmolec cm-2 over the range of these influence quantities.

Three of the PGN direct Sun instruments (see Sect. ) are located near the summit of a volcanic peak: Altzomoni (3985 m a.m.s.l.) in the State of Mexico, Izaña (2360 m a.m.s.l.) on Mount Teide on the island of Tenerife, and Mauna Loa (4169 m a.m.s.l.) on the island of Hawaii. At these high-altitude sites, the total column measured by the ground-based direct Sun instrument misses most of the tropospheric (potentially polluted) part and as such becomes representative of the TROPOMI stratospheric column. These sites have therefore been added to Fig. , illustrating that these comparisons based on direct Sun data yield similar conclusions as those based on zenith-sky data, that is, a minor negative median difference of the order of -0.2 Pmolec cm-2. It must be noted that, as discussed in Sect. , the PGN data are processed using cross sections at a temperature of 254.4 K, representative of a total column made of equal amounts of NO2 in the stratosphere and troposphere. This leads to columns which are about 10 % larger than if they had been processed with cross sections for 220 K. Future processing of the PGN data will address this, and it is expected that this will mostly remove the apparent negative bias for TROPOMI (but lead to a slight inconsistency with the ZSL-DOAS results).

TROPOMI data are filtered following the qa_value >0.75 rule as recommended in the associated PRF (see Sect. ). Then for each day, the pixel over the site is selected. MAX-DOAS data series are temporally interpolated at the TROPOMI overpass time (only if data within ±1h exist), and daily comparisons are performed. This short temporal window avoids the need for a photochemical cycle adjustment. Details on the comparison approach are described in for the validation of OMI and GOME-2 NO2 column data and in for the validation of the OMI QA4ECV NO2 Climate Data Record.

An illustration of the daily comparisons between TROPOMI and ground-based MAX-DOAS measurements between May 2018 and the end of January 2020 is presented in Fig.  for the Uccle station (Brussels, B, with moderate pollution levels). The two datasets have a correlation coefficient of 0.75 and a regression slope and intercept of 0.47 and 1.0 Pmolec cm-2 respectively. The (median and mean) difference of about -2.3 to -3.1 Pmolec cm-2 corresponds to a median relative difference of about -30 %.

Results for the entire MAX-DOAS network are presented in Fig. . This figure reveals mostly (but not only) negative differences, with a fairly significant variability but no clear seasonal features. No impact of the TROPOMI ground pixel size change on 6 August 2019 is observed.

Box-and-whisker plots for the whole network are shown in Fig. , with corresponding numeric values listed in Sect. . Based on measurements from these 19 MAX-DOAS stations, three different regimes can be identified: i.

Small tropospheric NO2 column values (median values below 2 Pmolec cm-2), e.g. at the Fukue and Phimai stations, lead to small differences. Typically, these stations show a small median bias (<0.5 Pmolec cm-2), but this can still correspond to up to a -27 % relative bias. The dispersion (IP68/2) of the difference is smaller than 1 Pmolec cm-2.

The overall bias (median of all station median differences) is -2.4 Pmolec cm-2, i.e. -37 %. The median dispersion is 3.5 Pmolec cm-2, while the site–site dispersion (IP68/2 over all site medians) is 2.8 Pmolec cm-2. Note that these network-averaged numbers are close to the numbers found for the polluted (Athens to Gucheng) sites. These results are within the mission requirement of a maximum bias of 50 %, but they exceed the uncertainty requirement of at most 0.7 Pmolec cm-2, which is only satisfied for the clean sites' ensemble. A discussion on the causes of these biases and sometimes large comparisons' spread is provided in Sect. .

Two key influence quantities for observations of tropospheric NO2 are aerosol optical depth (AOD) and cloud (radiance) fraction (CRF). The dependence of the differences between MAX-DOAS and TROPOMI tropospheric columns on these two influence quantities is visualized in Fig. . AOD is only retrieved in the processing of a handful of MAX-DOAS instruments, the others using climatological information, hence the limited subset in stations in panel (a) of this figure. No clear dependence of the bias on either property is seen, though in view of the relatively large scatter in these tropospheric column comparisons, this does not preclude more subtle dependencies. The impact of aerosol peak height would also be interesting to assess, but this is impossible to judge within the scope of the current paper as no such information is readily available.

As was done for the tropospheric column validation in Sect. , only S5P pixels with a qa_value of at least 0.75 are retained. The so-called summed product is used, i.e. the total column computed as the stratospheric plus the tropospheric column values. This summed column differs from the total column product. Only Pandonia measurements with the highest quality label (0 and 10) are used. The average column value within a 1 h time interval, centred on the S5P overpass time, is used. As the NO/NO2 ratio varies only slowly around the afternoon solar local time of the TROPOMI overpass, this small temporal window ensures no model-based adjustment is required. A 30 min time interval was tested as well, but this did not change the results significantly. Moreover, only TROPOMI pixels containing the station were considered.

An example of a time series of co-located TROPOMI and PGN total column measurements, and their difference, is shown in Fig. .

Results for the entire PGN network are presented in Fig. . This figure reveals that the difference, even in relative units, depends strongly on the total NO2 column, with low (or slightly positive) biases at low columns and markedly negative biases at high columns. No impact is observed for the TROPOMI ground pixel size switch of 6 August 2019.

Statistical estimators of the comparison results across the network are visualized in Fig.  and presented in tabular form in Table . One can distinguish roughly two different regimes. i.

The PGN median total column value is between 3 (Alice Springs) and 6 Pmolec cm-2 (New Brunswick). The absolute bias (median difference) is within ±0.2 Pmolec cm-2 in most cases (up to +0.5 Pmolec cm-2 at Egbert and Helsinki), while the median relative difference is within 5 % in most cases (up to ∼10 % at Alice Springs, Egbert, Inoe, and Helsinki). Canberra is a deviating case, with larger negative bias (-0.9 Pmolec cm-2; -20 %). The difference dispersion (IP68/2) roughly increases with increasing PGN NO2 median VCD, from 0.4–0.6 Pmolec cm-2 at the three cleanest sites to 1–2 Pmolec cm-2 at the other sites.

The median relative difference is mostly within (or bordering) the ±10% range for the sites with lower NO2 median total column values (Alice Springs to New Brunswick; Canberra is an exception), while it is negative and mostly outside this range, but still within ±50%, for the sites with higher NO2 median total column value (Buenos Aires to UNAM).

The overall bias over all sites (median over all site medians or site relative medians) is -0.5 Pmolec cm-2 (-7 %). The overall dispersion is 1.8 Pmolec cm-2, while the site–site dispersion (IP68/2 over all site medians) is 2.2 Pmolec cm-2.

It is however more useful to make the distinction between sites with low NO2 (Alice Springs to New Brunswick) and high NO2 (Buenos Aires to UNAM). For the low NO2 sites, the overall bias is 0.1 Pmolec cm-2 (2 %), the overall dispersion is 1.1 Pmolec cm-2, and the site–site dispersion is 0.2 Pmolec cm-2. For the high NO2 sites, the overall bias is -3.6 Pmolec cm-2 (-32 %), the overall dispersion is 3.3 Pmolec cm-2, and the site–site dispersion is 1.4 Pmolec cm-2.