The instantaneous aerosol DRE at the top of the atmosphere (TOA) is defined as the change in net (upwelling minus downwelling) irradiance, due to the introduction of aerosols in the atmosphere. Since the downwelling radiation is simply the incoming solar radiation, and restricting the discussion to smoke aerosols for which the extinction in the longwave radiation spectrum is small, the aerosol DRE for a cloud scene is 1DREaer=Fcld↑-Fcld+aer↑, where Fcld↑ is the shortwave upwelling irradiance in an aerosol-free cloud scene and Fcld+aer↑ is the shortwave upwelling irradiance of the same scene with both clouds and aerosols.

The aerosol DRE over clouds is determined from shortwave hyperspectral measurements of passive imagers, using measured reflectances of cloud scenes. The Earth reflectance is defined as the quotient of the upwelling radiance I(λ) and the downwelling solar irradiance E0(λ): 2R=πI(λ)μ0E0(λ), where μ0 is the cosine of the solar zenith angle θ0. If absorbing aerosols are present above the clouds, the measured scene reflectance R(λ)cld+aer will deviate from an aerosol-free cloud scene reflectance R(λ)cld. The reflectance difference is attributed to radiation absorption by the aerosols above the clouds, and the resulting direct radiative effect of these aerosols is quantified by integrating the reflectance difference over all wavelengths in the shortwave spectrum and all angles: 3DREaer=∫SWR(λ)cld-R(λ)cld+aerμ0E0(λ)B(λ,μ0)clddλ+ϵ, where R(λ)cld is a simulated aerosol-free cloud reflectance, representative of the measured scene with the aerosols removed. B(λ,μ0) is the anisotropy factor of a scene, which is a measure of the angular distribution of the reflected radiation for a scene and used to determine the radiance from a unidirectional reflectance measurement. This is determined from the modelled cloud scene and assumed to be unchanged by the aerosols over the clouds. ϵ represents all the instrument and retrieval errors of a single measurement. See Sect.  for a derivation of Eq. () and a comprehensive treatment of all its components.

The aerosol DRE follows from the integration of the radiance difference between the simulated aerosol-free cloud scene and measured aerosol polluted cloud scene over the solar spectrum. The integration is over the part of the shortwave spectrum where aerosols significantly absorb radiation. In case of combined OMI and MODIS reflectances, the integration limits are from the start of OMI measurements (about 270 nm) to the first of the MODIS channels that are used to invert cloud parameters (1246 nm), where the aerosol absorption is assumed to have become negligible.

The instantaneous aerosol DRE can also be derived for cloud-free scenes (substituting Fcld with Fclear in Eq. ). However, since the shortwave reflectance can be very small over dark scenes, the DAA method would produce very small numbers, yielding highly uncertain DREs. Therefore, the observations presented in this paper are restricted to cloud scenes only. Aerosol DRE for clear skies should be determined from observations of AOT in clear skies. Note that the more general all-sky direct radiative effect of aerosols in both clear and cloudy scenes is often derived as DREallsky=fcld⋅DREcld+(1-fcld)⋅DREclear e.g.. Here, DREcld is the direct radiative effect of all aerosols in a completely overcast atmosphere, DREclear the direct radiative effect of all aerosols in a cloud-free (Rayleigh) atmosphere, and fcld is the fraction of clouds. However, the validity of this equation, known as the independent pixel approximation , is dependent on pixel size and cloud homogeneity. The cloud fraction fcld is the fraction of an area where clouds appear with similar radiative properties. This may be true for satellites with sufficiently small pixels and homogeneous cloud fields. However, in this paper the aerosol DRE is derived from OMI, which has a relatively large footprint. For OMI an effective cloud fraction is derived (the OMCLDO2 product) , similar to the Fast Retrieval Scheme for Clouds from the Oxygen-A band (FRESCO) algorithm but using the O2–O2 absorption band at 477 nm, and the DRE is derived for OMI pixels with an effective CF > 0.3 to ensure sufficiently clouded scenes. The effective cloud fraction differs from the geometric cloud fraction in that it is radiatively equivalent to the brightness of the scene but assuming a thick cloud with a fixed albedo of 0.8. The reason is that for large OMI pixels, partial cloudiness and varying optical thickness cannot be discriminated. Usually, pixels with CF > 0.3 are fully covered with clouds. Therefore, COT and cloud droplet effective radius (CER) are retrieved assuming a completely clouded scene. Then, the aerosol DRE is computed using those cloud parameters again assuming complete cloud coverage. Although this is common for satellite cloud products, it should be understood that the OMI aerosol DRE dataset is not equivalent to DREcld above. A large part of the scenes with either small (geometrical) cloud fraction or small cloud optical thickness are not considered by selecting only scenes with effective CF >0.3. These scenes will have a small positive or negative aerosol DRE, as aerosol scattering dominates over dark surfaces. Therefore, the average OMI aerosol DRE in this paper is higher than the average true cloud or all-sky aerosol DRE. However, the dataset can be used to validate simulations of the aerosol DRE or other observational datasets where also scenes with CF > 0.3 are selected. For example, the SCIAMACHY aerosol DRE over clouds was compared to HadGEM2 simulations, which showed a clear underestimation of the aerosol DRE simulated by the model . A recent comparison with POLDER aerosol DRE for pixels with a cloud fraction larger than 0.3 shows that the aerosol effect could be even higher for thick plumes . The POLDER DRE correlated very well with SCIAMACHY and OMI–MODIS DRE but was even higher for very large values.

An illustration of the DAA technique is given in Fig. . The first step is the selection of suitable scenes, i.e. the selection of scenes with clouds; see above. To ensure the selection of (low-level) water clouds, only pixels with a cloud pressure larger than a threshold (e.g. 800 hPa) are selected. Step two is the determination of a measured scene reflectance spectrum. For SCIAMACHY this was trivial; the combination of OMI and MODIS reflectances is treated in Sect. . Step three is the retrieval of the cloud optical thickness and cloud droplet effective radius, using the SWIR part of the reflectance determined in step two, (e.g. R1.2µm and R2.1µm). The SWIR part of the lookup table (LUT) of reflectances is inverted to retrieve COT and reff . The fourth step is the simulation of the cloud scene reflectances in the ultraviolet (UV), visible, and SWIR part of the spectrum. This forward step is simplified using the same LUT as before, which contains reflectances at 18 wavelengths from 295 to 2130 nm; see Table . Once the simulated and measured cloud scene reflectances are available, the DRE is computed in step five, using Eq. (), and a measured or reference solar irradiance spectrum E0(λ).

A number of alternatives steps can be identified in this scheme. Firstly, the accuracy of simulating a cloud scene reflectance spectrum can be determined by adding an extra selection criterion in step one. The Aerosol UV-absorbing Index (AI) has been identified as a very good proxy for the presence of UV-absorbing aerosols in a (cloud) scene e.g.. By filtering for any cloud scene with a large AI, scenes with UV-absorbing aerosols above clouds are effectively filtered. If this criterion is added to step one, the remaining cloud scenes should yield a zero aerosol DRE. The (average) deviation from zero is a good estimate of the uncertainty in simulating the cloud scene reflectance. This is treated in Sect.  for OMI–MODIS pixels. Note, however, that the exact AI threshold value is dependent on the definition of the AI, which is different for different instruments and AI products and highly dependent on the calibration of the instrument. In the analysis in Sect.  the version 1.2.3.1 OMI Aerosol Product (OMAERO) AI at 354/388 nm was used, and it was found that a threshold of -1 was a better threshold for the removal of scenes with absorbing aerosols.

Secondly, the determination of COT in step three may be replaced by more accurate retrievals. In the current set-up, COT and reff are retrieved from the measured reflectance spectrum in step three. The SWIR measurements R1.2µm and R2.1µm are used to avoid biases due to absorption by aerosols, assuming that small particles do not effectively interact with radiation at those wavelengths. This works relatively well but is also a source of uncertainty for very thick plumes and larger particles. If unbiased cloud parameters can be obtained from other sources, e.g. from collocated dedicated cloud instruments, the DAA method may be improved, especially for thick aerosol plumes. It may even be extended to cases with desert dust above clouds, which are currently unsuitable because large mineral particles interact with radiation at SWIR wavelengths.

Originally, the DAA technique was applied to reflectance spectra from SCIAMACHY with a FRESCO effective cloud fraction larger than 0.3. SCIAMACHY was part of the payload of Envisat, launched in 2002 into a polar orbit with an Equator crossing time of 10:00 LT for the descending node. SCIAMACHY was designed to measure radiation in eight channels from 240 to 2380 nm at a spectral resolution of 0.2 to 1.5 nm . The radiance was observed in two alternating modes, nadir and limb, yielding data blocks called states, approximately 960×480 km2 in size. A state was divided into 13 swaths. In nadir mode, SCIAMACHY produced unique contiguous reflectance spectra from 240 to 1750 nm with an optical integration time of 1 s, by co-adding. By interpolating the spectra of pixels with an integration time of 0.25 s, a swath was divided into 16 pixels of approximately 60×30 km2. SCIAMACHY stopped delivering data in 2012.

In order to continue the DRE measurements, a combination of instruments can be used to determine a contiguous reflectance spectrum from the UV to the SWIR. A logical choice were instruments in the A-Train, which consists of several satellite platforms flying in constellation in a polar-orbiting, sun-synchronous orbit, crossing the Equator in the ascending node during the local afternoon (around 13:30 LT). The purpose is to allow the instruments on board the platforms to observe the same part of the Earth within minutes of each other. The time difference between the instruments within the A-Train is controlled by keeping the various satellites within control boxes, defined as the maximum distances to which the satellites are allowed to drift before correcting manoeuvres are executed.

The main focus here is the synergistic use of measurements from instruments on board the Aqua and Aura platforms. Aqua was launched in 2002 and Aura in 2004, following Aqua by about 15 min. A major orbital manoeuvre in 2008 of Aqua decreased the distance between the Aura and Aqua control boxes to about 8 min. The scene that is observed by both instruments is variable to a few minutes due to the time difference between Aura and Aqua.

In addition to the combined measurements from Aura and Aqua, a lidar on board the CALIPSO was used to illustrate the vertical distribution of the atmosphere. CALIPSO was launched in April 2006 and placed between Aqua and Aura. Therefore, it provides excellent collocation in time with the OMI and MODIS observations. The main payload of CALIPSO is CALIOP. It provides vertically resolved backscatter profiles of the atmosphere. Here, the Level 1B attenuated backscatter at 532 nm was used, to visualize the vertical distribution of clouds and aerosols of the atmosphere sampled by OMI and MODIS. Since the CALIOP across-track swath is very small, the measurements from CALIOP are representative of the centre of the OMI and MODIS swaths only. Note that CALIOP measurements are not needed for the DAA technique.

OMI , on board the Aura satellite, was designed to monitor trace gases in the Earth atmosphere, especially ozone. It was built as the successor to the ESA instruments GOME and SCIAMACHY and NASA's TOMS instruments e.g.. GOME and SCIAMACHY were the first space-borne hyperspectral instruments, measuring the shortwave spectrum from the UV to SWIR wavelength range (up until 800 nm for GOME), from which multiple trace gases, clouds, and aerosol parameters can be retrieved simultaneously. OMI was designed to measure the complete spectrum from the UV to the visible wavelength range (up to 500 nm) with a high spatial resolution and daily global coverage. The optical design of OMI is different from its predecessors, which used scanning mirrors. In OMI, the incoming radiation is projected onto a two-dimensional charge-coupled device (CCD). The radiation is split and mapped spectrally in one dimension of the CCD. In the other dimension, the across-track measurements are mapped. The across-track swath width is about 2600 km, resulting in a complete global coverage in 1 d. The spatial resolution of OMI is typically about 15×23.5 km2 at nadir to about 42×126 km2 for far off-nadir (56∘) pixels. However, the exact footprint size is complicated, which will be treated explicitly in Sect. . Since 2008, OMI suffers from progressive degradation, especially in far off-nadir pixels, called the row anomaly.

MODIS is an imaging spectroradiometer and a key instrument on board the Terra (EOS AM) and Aqua (EOS PM) satellites . MODIS acquires data in 36 spectral bands spanning the visible and infrared. Typical application of MODIS reflectances are measurements of the surface albedo, ocean colour and phytoplankton content, trace gases, clouds, and aerosols at a high spatial resolution. In this paper, only the shortwave spectral bands are used, which typically have a spatial resolution of 250 to 500 m and a band width of about 20 to 50 nm. The spatial and spectral specifications of the MODIS bands that are used in this paper are given in Table .

After selection of suitable cloud pixels (step one), a hyperspectral reflectance spectrum was constructed using collocated OMI/Aura and MODIS/Aqua pixels. Spectrally, OMI overlaps with MODIS at 459–479 nm (central wavelength 469 nm), which can be used to match the OMI reflectances in the visible channel and the MODIS reflectance in band 3. Spatially, the overlap is more complicated, since the OMI footprint is not uniquely defined due to the use of a polarization scrambler. The polarization scrambler projects four depolarized beams onto the detector CCD, which are slightly shifted with respect to each other, and therefore only the central point of the OMI footprint is uniquely defined. Furthermore, since the optics of OMI contain no moving mirror but project the incoming radiation onto the CCD detector array directly during a 2 s interval, the spatial response function of the OMI footprints is not box-shaped but rather Gaussian-shaped in two dimensions. About 74 % of the radiance received at a detector pixel is from within the corner coordinates; the rest of the signal is from outside the pixel corner coordinates. The OMI field of view was analysed in detail in  and . A 2-D Gaussian shape is used here to average MODIS reflectances across the OMI pixel, favouring pixels near the OMI centre and allowing for overlapping ground pixels.

The projections of radiation are slightly different in the two OMI UV channels and the OMI visible channel, resulting in slightly different ground pixels and wavelength grids, but these have not been accounted for. All computations were performed and reported relative to the wavelength grid and ground pixels of the OMI visible channel.

Two examples of OMI pixels tiled with MODIS pixels are shown in Fig. . Figure a shows an overview of the situation: a broken cloud field over the south-east Atlantic Ocean, west of Africa, with two OMI pixels: one in the stratocumulus cloud deck (red), and one at the cloud edge (blue). Figure b shows the MODIS pixels that are collocated with the OMI pixels, coloured by their weight in the averaging of the reflectance, which is the reflectivity convolved with the Gaussian function. Clearly, points close to the OMI pixel centre are favoured, but pixels beyond the corner coordinates also contribute to the radiation in the pixel. The cloud structure clearly has a large influence on the contributing pixels.

Figure c shows the combined OMI and MODIS reflectance of the fully cloudy scene (red), while Fig. d shows the combined OMI and MODIS reflectance of the broken cloud scene (blue). Clearly, there is a mismatch between OMI and MODIS for the broken cloud scene, which is caused by changes in the reflectance due to changes in the cloud fraction in the OMI footprint. The average reflectance of the scene has changed during the 15 min between overpasses of Aura and Aqua. The OMCLDO2 effective CF was 0.69 in the red pixel and 0.35 in the blue pixel. Fifteen minutes earlier, during the MODIS overpass, the geometric MODIS CF was around 0.99 and 0.98. Note that effective cloud fraction is generally lower than geometric cloud fractions. In order to get a contiguous reflectance spectrum, the average reflectance during the MODIS overpass is taken and OMI was scaled to match the MODIS average reflectance at 469 nm. Scaling MODIS to OMI seemed obvious at first, to have all parameters at the OMI grid and time. However, this resulted in very noisy data because scaled MODIS reflectances resulted in flawed cloud parameter retrievals at longer wavelengths and the accuracy of the DRE over clouds depends strongly on the accuracy of the cloud parameters. The derivation of cloud parameters is treated below.

The aerosol DRE retrieval over clouds is illustrated using a case of smoke over the south-east Atlantic Ocean in August 2006. Retrieval results from both SCIAMACHY and combined OMI–MODIS measurements on 10 August 2006 are shown in Fig. . August is the peak of the biomass burning season in southern Africa, and an extended smoke plume, originating from the African continent, drifts over the ocean in an elevated layer above a stratocumulus deck in the boundary layer.

The presence of the smoke can be observed in the RGB images of Fig. a and b as a grey haze over the continent, a darkening of the clouds, and high DRE values due to absorption of radiation by smoke above the stratocumulus cloud deck. This cloud deck is typical for this part of the ocean due to upwelling at the east part of the basin, cooling the sea surface. The stratocumulus cloud deck is persistent in the south and breaks up towards the Equator.

The vertical distribution of the aerosols and clouds is illustrated in Fig. c, using CALIOP attenuated backscatter at 532 nm along a track shown in Fig. a. It clearly shows the boundary layer stratocumulus clouds between 0 and 1 km altitude, rising towards the Equator, and a thick smoke plume between 1 and 4 km altitude. The strong returns are the surface at 0 km and cirrus clouds around 12 to 14 km.

The smoke consists of small particles, which scatter and absorb the incoming sunlight. Scattering dominates, and over a dark background like the ocean, the planetary albedo is increased due to the smoke. This will result in a negative direct radiative effect. However, over clouds the aerosol direct radiative effect becomes positive because the cloud optical thickness is large and the aerosols do not contribute much to the scattering of the sunlight. They do however, absorb radiation in the visible and UV part of the shortwave spectrum, reducing the planetary albedo, resulting in a positive aerosol radiative effect over clouds. This is quantified by the OMI–MODIS aerosol DRE over clouds (Fig. a).

The OMI–MODIS DRE reaches values of up to 100 Wm-2 in parts where smoke from the African continent is abundant. The values drop off to zero over clouds where the smoke plume is thinning and towards the cloud edges. The high and low values coincide well with concurrent measurements of SCIAMACHY DRE, shown in Fig. b. This figure shows the SCIAMACHY DRE overlaid on a MEdium Resolution Imaging Spectrometer (MERIS) RGB image, both on Envisat. Obviously, the spatial coverage of SCIAMACHY is much lower than OMI and MODIS, measuring in nadir mode only half of the time and having larger pixels. Consequently, the OMI–MODIS DRE is smoother with better coverage.

The location of the black OMI pixel (pointed at by the black arrow in Fig. a), coincides with the blue SCIAMACHY pixel in Fig. b (indicated by the blue arrow) and the black lines in Fig. c. The computation of the DRE using the DAA technique for these pixels is illustrated in Fig. . The OMI reflectance spectrum up to 500 nm of the black pixel is plotted in black, complemented with the average reflectance from collocated MODIS pixels (black dots). The variations in the reflectances of the individual MODIS pixels are shown by grey dots.

The retrieved cloud droplet effective radius for this OMI scene was 11.6 µm, and the cloud optical thickness was 6.7. The aerosol-free cloud reflectance spectrum for this scene, computed with these cloud parameters (step four), is shown by the red solid line in Fig. . By construction, the simulated reflectances match the MODIS-measured reflectances at 1.2 and 2.1 µm. Note that the average MODIS reflectance at 1.6 µm does not match the simulated reflectance, due to dead and bad pixels in this band.

Comparing the black and red lines in Fig. , differences can be observed between the simulated and measured reflectances by OMI and MODIS in the visible and UV. This is indicated by the yellow shaded area. The difference between the measured reflectance and the simulated scene reflectance is attributed to aerosol absorption by aerosols above the cloud layer in the real scene, which is not present in the simulated cloud-only scene, and used to compute the DRE following Eq. () (step five). The DRE derived for this OMI scene was 75.2 Wm-2.

In the same Fig. , the reflectance measured by SCIAMACHY is shown in blue, for the pixel indicated by the blue arrow in Fig. b. This is a scene which is at the same location as the OMI pixel in Fig. b but measured 3 h earlier. As can be seen in the RGB images, the cloud structures have changed rather considerably during this time, but the reflectance spectra from SCIAMACHY and OMI–MODIS are still remarkably similar. The DRE was also retrieved for this scene, using the SCIAMACHY reflectances at 1.2 and 1.6 µm. The cloud droplet effective radius during SCIAMACHY overpass was 8.3 µm and the cloud optical thickness was 14.2. The simulated aerosol-free cloud scene reflectance spectrum for these cloud parameters is shown in Fig.  as the red dashed line. The SCIAMACHY DRE using the reflectance difference between the simulated cloud scene and the measured scene is 87.6 Wm-2, which is slightly larger than observed by OMI. This is mainly due to the higher cloud optical thickness, for which the DRE is most sensitive.

The OMI–MODIS DRE is further compared with SCIAMACHY DRE over the south-east Atlantic area. SCIAMACHY has been used before to analyse the impact of smoke during the African biomass burning season on the radiation budget e.g.. Very high area-averaged instantaneous DREs were found in August 2006 of more than 80 Wm-2, which could not be reproduced by global climate models . These high DRE values have since been confirmed by POLDER measurements , which show even higher instantaneous DRE values than those with SCIAMACHY . The area-averaged instantaneous DRE over the south-east Atlantic was also determined from OMI–MODIS combined reflectances, and compared to the SCIAMACHY DRE (Fig. ). Only OMI pixels with an OMCLDO2 cloud fraction larger than 0.3 were selected, to ensure a sufficiently clouded scene, and only OMI pixels with an OMCLDO2 cloud pressure larger than 800 hPa, to exclude ice clouds. The maximum area-averaged instantaneous DRE from OMI–MODIS in August 2006 was 75.6±13 Wm-2. The SCIAMACHY data were similarly filtered, using a FRESCO cloud fraction larger than 0.3 and FRESCO cloud pressure larger than 800 hPa. The comparison is remarkably good, considering the much better OMI spatial coverage compared to that from SCIAMACHY. Pearson's correlation coefficient for the 7 d averaged DRE values from SCIAMACHY and OMI–MODIS is 0.9667. A fit between the two datasets showed that the DRE from OMI–MODIS was about 5 % lower than that retrieved from SCIAMACHY on average with an offset of 2.4 Wm-2.

Histograms of the DRE distribution during June to August 2006 are presented in Fig. . The average aerosol DRE over clouds was 25 Wm-2 with a standard deviation of 30 Wm-2 from OMI–MODIS measurements, while it was 28 Wm-2 with a standard deviation of 25 Wm-2 from SCIAMACHY measurements.

In order to provide an error estimate for the OMI–MODIS DRE measurements, the uncertainty ϵ in Eq. () is analysed in this section.