Several cloud retrieval algorithms based on measurements in and around the O2 A-band at 760 nm have been developed for the GOME type of sensors: these include the ICFA (Initial Cloud Fitting Algorithm; Kuze and Chance, 1994), FRESCO (Fast REtrieval Scheme for Clouds from the Oxygen A-band; Koelemeijer et al., 2001; Wang et al., 2008), SACURA (Semi-Analytical CloUd Retrieval Algorithm; Rozanov and Kokhanovsky, 2004), UV/NIR (Diedenhoven et al., 2007), and ROCINN algorithms. These are all based on the independent pixel approximation (IPA), which is the assumption that the “radiative properties of a single satellite “pixel” are considered in isolation from neighbouring pixels” (definition of the American Meteorological Society). The IPA allows for the application of one-dimensional plane-parallel RT theory in the forward simulation of cloud-contaminated atmospheric scenarios. This is further discussed in Sect. 5.

The ICFA algorithm was used in the initial GOME data processing to derive the effective fractional cover. The FRESCO algorithm, also developed for GOME, is based on the calculation of transmittances (later, single-scattering radiances), and it retrieves effective cloud fraction and cloud top pressure, assuming a fixed cloud albedo of 0.8. The SACURA algorithm was developed initially for the SCIAMACHY instrument and then modified to handle also GOME measurements (Lelli et al., 2012). SACURA uses semi-empirical formulae from asymptotic radiative transfer theory to retrieve cloud optical thickness, cloud top height, liquid water path and other parameters. The UV/NIR algorithm uses information from 350 to 390 nm together with that from the O2 A-band to retrieve cloud fraction, cloud optical thickness, and cloud top pressure. The ROCINN algorithm (Loyola et al., 2007) is currently being used in the operational GOME and GOME-2 products, and it retrieves as primary quantities the cloud top height and cloud albedo.

The broadband polarisation measurements from GOME, SCIAMACHY, and GOME-2 are used for computing cloud fraction; see for example OCRA (Loyola et al., 1998; Lutz et al., 2016) and HICRU (Heidelberg Iterative Cloud Retrieval Utilities; Grzegorski et al., 2006). Enhancements to these algorithms have been introduced in recent years – see for example the detection of sun glint effects (Loyola et al., 2011; Lutz et al., 2016). For these instruments, the PMDs enable the cloud fraction to be retrieved at 8 times higher spatial resolution (10 × 40 km2) than that for the main science channels (80 × 40 km2) which are used for the retrieval of the trace gases.

There are three cloud-property algorithms in operational use for the OMI instrument (OMI has no O2 A-band measurements). The first (OMI/Aura Cloud Pressure and Fraction Raman Scattering, or OMCLDRR) uses the cloud screening effect on Fraunhofer filling signatures (due to inelastic rotational Raman scattering) in the region 346–354 nm to derive effective cloud fraction and cloud optical centroid pressure (Joiner and Vasilkov, 2006; Vasilkov et al., 2008; Joiner et al., 2012). This algorithm is based on the mixed Lambertian equivalent reflectivity (MLER) assumption. The second algorithm (OMI/Aura Cloud Pressure and Fraction O2–O2 Absorption, or OMCLDO2) uses reflectances in and around the O2–O2 absorption band near 477 nm (Acarreta et al., 2004; Veefkind et al., 2016); differential optical absorption spectroscopy (DOAS)-retrieved O2–O2 slant columns are compared with simulated look-up table entries to obtain effective cloud fraction and cloud pressure. The third algorithm (OMI/Aura Near UV Aerosol Optical Depth and Single Scattering Albedo, or OMAERUV) derives aerosol optical depth and single-scattering albedo (Torres et al., 2007) from radiances at 354 and 388 nm; the cloud fraction is computed as an intermediate step.

The OCRA colour space approach can be applied with three colours (RGB space) or two colours (GB space or RG space). For a given location (x,y), we define the reflectance ρ(x,y,λi) at wavelength range λi for the footprint of the measurement as 1ρx,y,λi=π⋅IλiE0λi⋅cosθ0, where Iλi and E0λi denote the measured earthshine backscattered radiance and the solar irradiance respectively, and θ0 is the solar zenith angle.

The reflectances used in this algorithm are derived from broadband measurements of backscattered radiance and extraterrestrial solar irradiance covering the spectral range of the GB colour system. The OCRA spectral ranges with TROPOMI/S5P are 405–495 nm for G and 350–395 nm for B.

The core of the algorithm is the construction of a cloud-free composite of multi-temporal (time series of measurements over the same location) reflectances that is independent of atmosphere and solar and viewing angles; this is indicated as the “internal store” in the flowchart of Fig. 1. For the offline creation of cloud-free reflectance composites in the GB case, the green and blue channel reflectances, which are denoted by the letters G and B respectively, are translated into normalised gb colour space via the relations 2g=ρ(x,y,λG)∑i=GBρ(x,y,λi),b=ρ(x,y,λB)∑i=GBρ(x,y,λi). If M is the set of n normalised multi-temporal measurements over the same location (xy), then a cloud-free (or minimum cloudiness) pixel gbCF∈M is selected using the brightness criterion gbCF-W≥gbk-W, for k=1,…n, where W is the white point (1/2, 1/2) in the gb-chromaticity diagram. This point refers to a situation where B and G are equal, i.e. where there is no wavelength dependence across the UV–VIS region – this is interpreted as a scene fully covered by cloud. Measurements under cloudy conditions are projected to the white point, and the measurement that is most distant from W is considered to be cloud-free. A cloud-free background, labelled “climatology” in Fig. 1, is constructed by merging cloud-free reflectances ρCF(λi) (corresponding to gbCF) at all locations. It should be noted here that the G and B cloud-free reflectances for a given grid cell are not determined independently as a minimum available reflectance over the whole monthly time range, but rather they are the interdependent reflectances belonging to an individual scene representing the largest distance from the white point in the gb-chromaticity diagram.

At the beginning of the TROPOMI/S5P mission, a monthly cloud-free background data set based on OMI measurements will be used, to be replaced by TROPOMI data as the mission unfolds.

The radiometric cloud fraction fc is determined by examining separations between measured GB reflectances and their corresponding cloud-free composite values: fc=3min⁡1,∑i=GBαλimax0,ρλi-ρCF(λi)-βλi2. This equation expresses the distance between actual measurements and the corresponding cloud-free scene in colour space. Scaling factors αλi=GB define the upper limit for reflectances under fully cloudy conditions, while offsets βλi=GB account for aerosol and other radiative effects in the atmosphere and as a lower limit basically define the cloud-free conditions. The “max” and “min” functions ensure that the cloud fraction is confined to the interval [0, 1].

The scaling and offset factors are determined offline using representative daily global satellite measurements (Lutz et al., 2016). The offsets are the histogram modes from the differences {ρλi-ρCF(λi)}, and the scaling factors are the inverses of the 99th percentile of the cumulative histograms from the differences {ρλi-ρCF(λi)}2. The temporal variability of the offsets and scaling factors is investigated by comparing several daily global histograms at different occasions throughout the year. Since no significant seasonal dependence is apparent, only one set of α and β per colour is used. It should be noted here that the offsets may partially compensate for extremely dark scenes (e.g. shadows) and radiative effects (e.g. absorbing aerosols), but a strict separation of aerosols and clouds is not done by OCRA.

Direct sunlight reflected by the ocean surface may reach the satellite sensor, enhancing the measured signal in a manner which contaminates cloud effects. Sun glint flagging was developed as a component of the operational OCRA algorithm (Loyola, 2011) for GOME-2/MetOp-A, and this treatment was further enhanced using the polarisation Stokes fractions (Lutz et al., 2016) to correct for the sun glint effect. Since TROPOMI does not provide polarisation information, a simplified sun glint flagging procedure will be used instead of this correction. First, those areas that might be affected by sun glint are marked using the viewing geometry conditions of the measurement: 4v=|Θo-Θ|-22+φo-φ-1802, where Θo and Θ are the solar and satellite zenith angles respectively, and φo and φ the solar and satellite azimuth angles (values are given in degrees). A sun glint flag is set whenever the value of “marker” v is larger than a given threshold. This threshold value will be determined dynamically when real TROPOMI data become available; for now, the OMI threshold value of 45 will be used initially.

Before we describe ROCINN itself, we remark on the initial wavelength registration (see Fig. 2). The wavelength grid of the measured solar irradiance E0 is recalibrated using a high-resolution solar reference Esol by first dividing the fitting window into sub-windows and computing for each sub-window j a wavelength shift Δλj between E0,j and Esol,j. The solar reference spectrum used for the wavelength calibration is the SAO2010 (https://www.cfa.harvard.edu/atmosphere/links/sao2010.solref.converted) produced for atmospheric measurements in the UV–VIS–NIR by Chance and Kurucz (2010). The calibration is applied on the reference spectrum by using DOAS fit methods as part of the UPAS processor. The recalibrated grid is then established by applying (at each original wavelength point) a shift value computed from a polynomial fit through the Δλj for the various sub-windows. The fitting is achieved for polynomials of a degree of 3 for S5P.

Note that during the inversion (see Sect. 4.5) a wavelength shift for the earthshine spectrum is fitted additionally.

The sun-normalised radiance R(λ) at wavelength λ is then defined as 5Rλ=IλE0λ, where Iλ and E0λ denote the measured earthshine backscattered radiance and solar irradiance spectra respectively, with both spectra registered on the recalibrated solar irradiance grid as noted above.

For ROCINN with CAL, the total sun-normalised radiance is taken to be a linearly weighted sum of independent radiances Rs for the clear-sky scene and RcCAL for the cloud-filled scene, with the weighting expressed through the radiometric cloud fraction fc. Both radiance contributions are calculated using standard one-dimensional radiative transfer models (RTMs).

The sun-normalised radiance for a cloudy scene is calculated with the cloud treated as a single scattering layer with geometrical extent characterised by cloud top height Zct and cloud base height Zcb (or alternatively the cloud geometrical thickness Hc=Zct-Zcb). The entire cloud is optically uniform with cloud optical thickness τc, and its scattering properties are determined through Mie-scattering calculations for water droplet particles (microphysical properties are discussed below). In the IPA, we may write sun-normalised CAL simulated radiances RsimCAL as RsimCALλ=fcRcCALλ,Θ,τc,Zct,Zcb,As,Zs6+(1-fc)Rsλ,Θ,As,Zs. Here, Θ denotes path geometry (solar and line-of-sight angles), and surface properties are the Lambertian albedo As and lower boundary height Zs.

Radiances for clear-sky and cloudy scenarios are calculated using the VLIDORT (Vector Linearized Discrete Ordinate Radiative Transfer) model (Spurr, 2006), at wavelengths in and adjacent to the O2 A-band. Details of the RTM calculations are given in Sect. 4.4 below.

A complete data set of simulated sun-normalised radiance templates is created offline for an appropriate range of viewing/solar geometries and surface geophysical scenarios, and for various combinations of cloud properties.

The inverse problem uses least-squares fitting with a generalised form of Tikhonov regularisation (details in Sect. 4.5). Retrieval in the O2 A-band with the four-element state vector τc,Zct,Zcb,fc is an ill-posed problem that requires additional information in order to obtain an inverse solution, as there are only two degrees of freedom for signal (DFS; Schüssler et al., 2014). For ROCINN_CAL, the retrieval state vector is just τc,Zct for cloud optical thickness τc and height Zct, a fixed cloud geometrical thickness of 1 km is assumed, and the radiometric cloud fraction fc is taken from OCRA.

ROCINN with CRB assumes that clouds are treated as Lambertian reflectors. The sun-normalised CRB simulated radiances RsimCRB are defined as 7RsimCRBλ=fcRcCRBλ,Θ,Ac,Zc+(1-fc)Rs(λ,Θ,As,Zs). The retrieval state vector for ROCINN_CRB is Ac,Zc for cloud albedo Ac and cloud height Zc; the radiometric cloud fraction fc is again from OCRA.

ROCINN is based on simulated sun-normalised radiances at wavelengths in and around the O2 A-band. Two sets of radiance templates were calculated using the CRB and CAL models. The cloudy-scene sun-normalised radiances RcCAL were calculated for a multi-layer atmosphere including multiple scattering in all layers. Mie scattering was used to generate cloud optical properties. Details may be found in Schüssler et al. (2014).

Simulated sun-normalised radiances Rsimλ are calculated using the vector VLIDORT multiple-scattering multi-layer discrete ordinate RTM (Spurr, 2006); the desired total intensity I will incorporate the effects of polarisation. The incorporation of a vector RTM is necessary not for TROPOMI itself but for the processing of data from GOME, SCIAMACHY, and GOME-2. In addition to the cloud layers, VLIDORT calculations are based on clear-sky optical properties for line absorption by oxygen molecules and Rayleigh scattering by air molecules.

For the line absorption, it is necessary to calculate line-by-line (LBL) radiances (typically at resolution 0.0015 nm for the range 758–771 nm) using line-spectroscopic information for the O2 A-band before convolution with the sensor slit function.

The spectroscopic data are taken from the HITRAN 2012 database (released in June 2013). Absorption cross sections are computed using LBL software from DLR (Schreier and Schimpf, 2001; Schreier, 2011), in which line absorption signatures are accurately modelled with the Voigt profile.

For Mie scattering calculations, we require knowledge of microphysical properties of clouds consisting of liquid water. The droplets are assumed to be randomly distributed within the cloud layer, and any possible inhomogeneity in the cloud is assumed negligible in the current version of CAL model. In addition, we have found that the consistency of cloud models (e.g. CAL or CRB) used in both the cloud and UV–VIS trace gas retrievals is far more critical than the optical properties selected for the RTM simulations of the CAL templates.

The drop size distribution is well approximated by the modified-Gamma size distribution function (Deirmendjian, 1964): 8nr=Cr-αexp-αγrrcγ, which is parameterised by the mode radius rc in units of microns (µm), and constants α and γ describe the shape of the distribution following Hess et al. (1998). In Eq. (8), C is the normalisation constant. Characteristic values for the low-level cloud (i.e. stratus/cumulus) parameterisation are mode radius rc=4.75µm and shape parameters α=5 and γ=1.61.

The cloud macro-physical properties (classifications of cloud top height and cloud geometrical thickness) are based on the tables in Wang et al. (2000). Details of this algorithm prototype can be found in Schüssler et al. (2014).

The cloud geometrical thickness is always constant and equal to 1 km; thus, the liquid water path of the cloud is then defined by the total number concentration. A single phase scattering function is used with the extinction cross section for spherical particles obtained by Mie theory (Van de Hulst, 1957; Bohren and Huffman, 1983). The complex refractive index (n+im) of cloud droplets was configured as n=1.33 and m=1.56×10-7 for liquid water at 758 nm (Hale and Querry, 1973).

The line-by-line RT calculations in the O2 A-band are computationally very demanding, and this precludes the deployment of online calls to VLIDORT during the processing of TROPOMI data. For this reason, RTM simulations for the range of S5P viewing conditions are performed in advance. Node points for the RTM are created using a “smart sampling” technique (Loyola et al., 2016) that minimises the number of calls to the RTM and at the same time optimally covers the input space. There are many millions of forward-model calculations required; this process is done offline and normally takes several weeks to complete. In the next step, the LBL simulations are convolved with the TROPOMI instrumental spectral response function, and the resulting radiances are used to train a neural network that accurately approximates the RTM template output with a mean average relative error below 1 % over the O2 A-band spectral window for all scene geometries. The node point generation, RTM simulation, and neural-network training are done using the smart sampling and incremental function learning technique (Loyola et al., 2016). The input space (surface properties, cloud properties, and geometries) is not sampled using a set of regular grids, but instead a smart sampling technique (Loyola et al., 2016) is used to optimise the distribution of multi-dimensional points within the (input) state space. The total number of computational nodes was of the order of some hundred thousand. The trained neural network that computes the O2 A-band sun-normalised radiances is used in the UPAS operational environment, and this enables ROCINN retrievals to be done very quickly.

If x is the state vector τc,Zct,As,fc comprising possible cloud parameters for retrieval, and b denotes a vector of auxiliary forward-model parameters (surface properties, viewing geometry, etc.), we write the measurement vector as yδ=Fx,b+δ, where F is the forward model and δ is the data error vector. The inverse problem defined by this equation is non-linear and ill-posed, and regularisation is required in order to obtain a solution with physical meaning. The degree to which the problem is ill-posed is partly characterised by the condition number cK=γmax/γmin of the Jacobian matrix K=dF/dx, where γmax and γmin are the largest and the smallest singular values of K respectively.

In the form of Tikhonov regularisation used here, the regularised solution xαδ minimises the objective functional: 9Fαx,b=12Fx,b-y2+αL(x-xa)2. Here, α denotes the regularisation parameter, and L is the regularisation matrix (Doicu et al., 2010). The functional is defined with the L2 Euclidean norm. The minimiser for Eq. (9) can be computed with Gauss–Newton methods.

In statistical inversion theory, the Bayesian approach or the optimal estimation method can be regarded as a stochastic version of Tikhonov regularisation. The maximum a posteriori solution coincides with the Tikhonov solution when the state vector x and the noise vector δ are Gaussian random vectors with covariance matrices Cx=σx2In and Cδ=σ2Im respectively, where σx and σ are the corresponding standard deviations, and In is the identity matrix of size n. In this case, the regularisation parameter α is the ratio of these two variances; that is, α=σ2/σx2.

As noted above, the operational ROCINN algorithm with CAL (or CRB) retrieves two cloud parameters: the cloud top height and cloud optical thickness (or cloud albedo) with the a priori cloud fraction taken from OCRA and the surface albedo from the MERIS black-sky climatology at 760 nm (Popp et al., 2011). Note that the cloud fraction and the surface albedo are included in the state vector with a very strong regularisation (i.e. only very small changes are allowed) in order to improve the fitting. These very small changes refer to the differences between the retrieved value of cloud fraction (and surface albedo) and their corresponding a priori value. The regularisation parameter for cloud fraction and surface albedo is very high (i.e. 2 orders of magnitude higher than that for cloud top height and cloud optical thickness/cloud albedo), and thus these retrieval parameters are always well within 1 % difference from their a priori values. The state vector includes additionally a single wavelength registration shift parameter that takes care of the Doppler effect. The inverse model requires the partial derivatives of the radiances with respect to the state vector elements, and these Jacobians are provided by the forward model.

Convergence is reached when either the residual Fx,b-y2 or incremental changes in the retrieved parameters Δx are smaller than pre-defined values (defaults: 5 × 10-3 and 5 × 10-5), or when the maximum number of iterations (default: 50) is reached. The default value for the regularisation parameter α is 1 × 10-4.

The equivalence between the Bayesian approach and the method of Tikhonov regularisation enables us to analyse the information content of the signal with respect to the retrieved parameters in a stochastic framework (Schüssler et al., 2014). DFS is a measure of the number of independent pieces of information in the measurement, and it gives the minimum number of parameters which can be used to define a state vector without loss of information. It is defined as the trace of the averaging kernel matrix, which represents the sensitivity of the retrieval to changes in the true state. The DFS can be computed as 10DFS=∑inγi2γi2+α, where γi2are the singular values of the matrix K.

Another useful criterion for the estimation of the retrieval quality is the Shannon information content (SIC), which is a measure of the incremental gain in information, defined as the entropy difference between the a priori and a posteriori states; the corresponding formula reads as follows: 11SIC=12∑inlog⁡1+γi2α. The accuracy of the regularised solution is represented by the mean square error matrix: Sα=εx†-xαδx†-xαδT≈In-Aαxαδ-xaxαδ-xaTIn-AαT12+σ2Kα†Kα†T, where x† is the exact solution or “true state”, xαδ the regularised solution, xa the a priori state vector, Aα the averaging kernel matrix, Kα† the generalised inverse, σ the noise standard deviation, α the regularisation parameter, and ε the expected value operator. Further information on the mean square error matrix and Tikhonov regularisation can be found in Doicu et al. (2010).

In order to evaluate the performance of the ROCINN retrieval algorithm in TROPOMI/S5P, a data set of synthetic TROPOMI measurements has been created. Synthetic spectra were computed using VLIDORT for a number of different scenarios characterised by various illumination and observation geometries, surface albedo, and cloudiness. In particular, the following input space has been covered using the smart sampling technique: surface height of [0–4] km, surface albedo of [0–1], cloud top height of [2–15] km, cloud optical thickness of [2–50], viewing zenith angle of [0–75] degrees, solar zenith angle of [0–90] degrees, and relative azimuth angle of [0–180] degrees.

In general, these closed-loop ROCINN_CAL retrieval results are excellent; the cloud top height results have no bias. The ROCINN_CRB retrieval was also applied to the same data set of synthetic spectra in order to obtain retrievals of the cloud height and cloud albedo. The results are shown in Fig. 3; as expected, the CRB cloud height is systematically below the simulated cloud top height, with a median difference of 1.2 ± 0.4 km.

Figure 4 shows the correlation from ROCINN_CAL retrievals of cloud optical thickness and ROCINN_CRB retrievals of cloud albedo. The differences in the cloud optical thickness are symmetrical about the zero-bias line, whereas the differences in the cloud albedo are slightly skewed towards negative values.

An important source of error is the spatial misregistration between the UV–VIS and NIR bands from TROPOMI. The combination of information from different spectral bands is not as straightforward as it appears, since the spatial regions covered by the ground pixels from different spectral bands do not match exactly. One method for combining information from different bands is by means of the TROPOMI coregistration mapping tables, which contain the fractions of overlapping areas between the source and target pixels (Sneep, 2015). For combinations based on OCRA UV bands 3, 4, and 5 and ROCINN NIR band 6, a static coregistration table suffices. However, this method implies a smoothing of the source band products.

For this purpose, a cloud coregistration inhomogeneity flag (CCIF) will be included in the S5P cloud products. This is determined as follows. First, a cloud coregistration inhomogeneity parameter (CCIP) is defined as the weighted averaged gradient of cloud fractions: 13CCIPj=∑iωijfci-fcj∑iωij, where the weights ωij correspond to the coregistration mapping values between UV bands (source, index i) and the NIR band (target, index j). The CCIF is defined as 14CCIFj=CCIPj>p, where p is a fixed threshold which has been set to 0.4 as the baseline, following extensive testing using the VIIRS (Visible/Infrared Imager and Radiometer Suite) cloud product on board the Suomi NPP (NPOESS Preparatory Project) resampled to the TROPOMI spatial grid.

The two-colour OCRA_GB model was tested against the three-colour OCRA_RGB model. The overall median cloud fraction difference is 0.026 and the mean 0.032 for a single day of GOME-2A measurements from 1 July 2012. For cloud fractions smaller than 0.1 the median and mean differences are 0.007 and 0.015 respectively.

Figure 8 shows the cloud fraction differences in a box-and-whisker plot. The bottom, inside band, and top of the box are the first, median, and third quartiles of the differences respectively.

As part of the S5P project we have adapted the OCRA algorithm to the OMI sensor (the precursor of TROPOMI).

OMI is a nadir-viewing push-broom spectrometer observing solar backscatter radiation in the ultraviolet and visible wavelength range up to 500 nm (Levelt et al., 2006). The swath width is 2600 km on the ground, encompassing more than 60 across-track pixels. The highest spatial resolution of 13 × 24 km2 (in normal mode) is achieved for the nadir pixels. OMI was launched in 2004 on the NASA Aura satellite platform.

The first step is the calculation of a monthly OCRA cloud-free background data set as described in Sect. 3.2; this was based on 3.5 years of OMI measurements from January 2005 to June 2008. The OMI cloud-free background data set for the month of August is shown in Fig. 9. Note that OMI is extremely stable, with almost no instrument degradation – this facilitates significantly the calculation of the OCRA cloud-free backgrounds.

The temporal resolution is 1 month; i.e. for a given grid cell, all measurements from a given month are aggregated in order to reflect seasonal surface variations. Finally, all data from the years 2005, 2006, 2007, and 2008 are considered for the final monthly maps. The spatial grid resolution is 0.2∘ latitude by 0.4∘ longitude. The cloud-free reflectance value for any given time and geolocation is found via linear interpolation between the two adjacent monthly cloud-free maps. This linear interpolation between monthly maps was found to give the best trade-off between the necessity to have as many measurements as possible per grid cell in order to ensure a cloud-free situation (i.e. a long timescale is desired) and the requirement to be sensitive to rapid changes in the surface conditions such as snowfall or melting (i.e. a short timescale is desired).

Secondly, the scaling and offset factors are computed following the procedure described in Sect. 3.3. The resulting scaling factors are αB=2.88 and αG=2.14, and the offset factors are βB=0.0138 and βG=0.0180.

For the comparison with MODIS, we used the following cloud products:

The OMAERUV product, provided by O. Torres. The cloud fraction is an ancillary product from the absorbing aerosol index algorithm based on OMI radiances at 388 and 354 nm (Torres et al., 2007).

Figure 10 shows the global cloud maps obtained with OCRA, OMAERUV, OMCLDO2, and OMCLDRR from OMI measurements on 16 July 2005; the four algorithms generate similar cloud features.

A quantitative comparison of the zonal mean cloud fractions from OMI and those derived with MODIS is presented in Fig. 11. The UV sensors are less sensitive to optically thin clouds than thermal infrared sensors; as expected, MODIS generates larger cloud fractions compared to those from OMI. The cloud fractions of OMCLDO2 and OMCLDRR are similar because both algorithms assume a fixed cloud albedo or reflectance of 0.8 for the retrieval, but overall the cloud albedo is significantly smaller (e.g. Lelli et al., 2012, report a mean global cloud albedo value of 0.63 based on GOME data from 1996–2003), and therefore the retrieved effective cloud fractions are significantly smaller than the MODIS geometrical cloud fractions. On the other hand, OCRA and OMAERUV do not need to assume a fixed cloud albedo, and their retrieved cloud fractions are larger than those from OMCLDO2 and OMCLDRR. OCRA and OMAERUV report cloud fraction values more representative of the radiometric cloud fraction based on top-of-atmosphere radiances measured by the instrument. We should emphasise here that a direct comparison between the MODIS geometric cloud fraction and the OMI-derived radiometric or effective cloud fractions should be treated with caution.

In preparation for S5P we have applied the ROCINN_CAL algorithm to GOME-2, and in this section we present for the first time the resulting cloud parameter retrievals.

GOME-2 is a nadir-viewing optical spectrometer that senses Earth's backscattered radiance and solar irradiance at UV–VIS–NIR wavelengths in the range 240–790 nm (Munro et al., 2016). The nominal full GOME-2 swath has a width of 1920 km in the direction perpendicular to the flight direction, and a single scan line has an extension of 40 km in the flight direction. The ground pixels have a spatial resolution of 80 × 40 km2. In addition, broadband PMDs provide an eightfold higher spatial resolution, i.e. 10 × 40 km2 for a selection of 15 spectral windows. Currently there are two GOME-2 operational sensors on board the EUMETSAT MetOp-A and MetOp-B satellites launched in 2006 and 2012 respectively; both GOME-2 sensors are operated in tandem, providing global measurements on a daily basis. A third GOME-2 sensor on board MetOp-C will be launched in 2018.

The VLIDORT line-by-line RTM simulations described in Sect. 4.4 are convolved with the GOME-2 instrumental spectral response functions, and these results are then used to train a neural network that accurately approximates the O2 A-band reflectances (Loyola et al., 2016). As noted in Sect. 4.5, the ROCINN cloud top height and optical thickness are retrieved using the Tikhonov inversion, taking as input the OCRA cloud fraction computed from the GOME-2 PMD measurements (Lutz et al., 2016).

Figure 12 shows the global cloud maps obtained with ROCINN_CAL and ROCINN_CRB from GOME-2A (GOME-2 on MetOp-A) measurements taken on 1 July 2012. As expected from the retrievals with synthetic data (Sect. 4.7) the cloud height retrieved using ROCINN_CRB is smaller than that retrieved using ROCINN_CAL; the CRB model retrieves the centroid of the cloud and not the cloud top (Joiner et al., 2012). The cloud optical thickness from ROCINN_CAL nicely correlates with the cloud albedo from ROCINN_CRB.

The histogram of absolute differences between the GOME-2A cloud heights on 1 July 2012 obtained with ROCINN_CAL and ROCINN_CRB is presented in Fig. 13. The CRB model underestimates the cloud top height with a median difference of 0.92 ± 0.75 km. These results are consistent with the retrievals obtained using synthetic data from Sect. 4.7.

The diagnostic quantities DFS and SIC for the test case of 1 July 2012 were found to lie in the ranges 1.2–4 and 2–11 respectively for ROCINN_CAL. These values are lower than those obtained for ROCINN_CRB (DFS between 2.1 and 4.3, SIC in the range 4–17), showing that the CAL-retrieved cloud quantities depend more on the a priori information.