Insights into 3D cloud radiative transfer effects for the Orbiting Carbon Observatory

Clouds impose radiance perturbations upon Orbiting Carbon Observatory (OCO-2)-measured spectra. The Spherical Harmonic Discrete Ordinate radiative transfer Method (SHDOM) code is applied in both idealized bar cloud and scene-specific calculations of 1D and 3D radiances in order to understand 3D cloud effects for a wide range of gas vertical optical depths and solar- and sensor-viewing geometries for ocean and land scenes. SHDOM calculations for 36 scenes over the Amazon and the Pacific are co-analyzed with Moderate Resolution Imaging Spectroradiometer (MODIS) radiance-based cloud distance data and the OCO-2 Lite file rawXCO2 for both quality flag =0 (QF0; best quality) and quality flag =1 (QF1; poor quality) data. SHDOM calculations of the ocean and land scenes indicate that the 1D $/$ 3D radiance intensity ratios and rawXCO2 decrease concurrently as the nearest-cloud distance decreases towards zero, especially for the ocean glint QF1 data, which provide the clearest evidence of 3D cloud effects in OCO-2 retrievals. Yearly analysis of OCO-2 O₂ A-band continuum radiances indicate that 3D cloud-brightening events are predominant over cloud-shadowing events; therefore, 1D $/$ 3D intensity ratios are predominantly less than unity. Bias corrected (bcXCO2) at cloud distances between 0 and 20 km are calculated for 20^∘ latitude bands for 2015–2018. These zonal averages are used to calculate 3D-cloud-effect biases for bcXCO2 data (with a positive bias indicating that OCO-2 underestimates bcXCO2). Averages of 3D-cloud-effect biases, weighted by the number of Lite file data points in each of the nearest-cloud distance bins, in the Northern and Southern hemispheres, are 0.16 (1.31) and 0.26 (1.41) ppm (parts per million), respectively, over the ocean, and −0.13 (0.51) and −0.08 (0.47) ppm over land for QF0 (QF1) data.

1 Introduction

The Orbiting Carbon Observatory (OCO-2) measures the column-averaged atmospheric CO₂ dry-air mole fraction, referred to as XCO₂, on a global basis (Eldering et al., 2017). The project measurement goal is to obtain measurements of XCO₂ to the 0.25 % level, corresponding to a XCO₂ accuracy of 1 ppm (parts per million), since ambient XCO₂ is near 410 ppm. Biases in XCO₂ on regional scales as small as a few tenths of a ppm in XCO₂ can lead to spurious values of inferred CO₂ fluxes (Chevallier et al., 2010).

The OCO-2 satellite is in a sun-synchronous, 98^∘ inclination polar orbit and is comprised of three spectrometers centered in the O₂ A band (0.76 µm), weak CO₂ band (WCO₂; 1.6 µm), and strong CO₂ band (SCO₂; 2.06 µm). Spectral resolution $\mathit{\lambda }/\mathrm{\Delta }\mathit{\lambda }>\mathrm{17}\mathrm{000}$ (Crisp et al., 2017) ensures that individual spectral lines are observed. Each band contains 1016 data points, covering a wide range of optical depth. The OCO-2 experiment has several observing modes, namely ocean glint, land glint, land nadir, and target mode. In target mode, the spectrometers are commanded to observe a specific small geographical region. In ocean glint observations, the bright ocean glint spot is utilized to increase the observed radiance level. Land glint observations utilize glint observing geometry and are not restricted to water surfaces. The land nadir observations have a sensor view angle near 0.3^∘. We focus on ocean glint, land glint, and land nadir scenes in this paper.

The Atmospheric Carbon Observations from Space (ACOS v10) retrieval of CO₂ is based upon the measurement of surface pressure in the O₂ A band and CO₂ absorption in the WCO₂ and SCO₂ bands. The retrieval applies the optimal estimation retrieval methodology of Rodgers (2000). The operational retrieval (OCO-2 L2 ATBD, 2020; O'Dell et al., 2018) solves for a state vector with many elements, including XCO₂, surface pressure, reflectance, and aerosol. Spectroscopic line cross sections for O₂, CO₂, and H₂O are specified by the Absorption Coefficient (ABSCO) V5.1 data files (Payne et al., 2020).

XCO₂ generated by the operational retrieval is referred to as raw(XCO2). A post-retrieval processing step then bias corrects the raw(XCO2), yielding bias-corrected XCO₂ (bcXCO2). Bias correction (O'Dell et al., 2018) is achieved by comparing rawXCO2 to truth proxies, including ground-based XCO₂, measured by the Total Carbon Column Observing Network (TCCON; Wunch et al., 2017), ensemble model XCO₂, and small area analysis (in which differences in XCO₂ values and the area average are calculated). The differences between the raw(XCO2) and the truth proxies are related in a linear manner to several bias-correction parameters (dP over the ocean, dPfrac over land, CO2graddel, and DWS). dP is the difference (in hPa) between the retrieved (Pretrieved) and a priori (Papriori) surface pressure evaluated at the strong SCO₂ band geographic location, while dPfrac (in ppm) is as follows:

$$\begin{array}{}\text{(1)}& \text{dPfrac}=\text{raw}{\mathrm{XCO}}_{\mathrm{2}}(\mathrm{1.00}-\text{Papriori}/\text{Pretrieved}).\end{array}$$

CO2graddel is a measure of the difference in the retrieved and prior CO₂ vertical gradient. DWS is the sum of the vertical optical depths of the dust, water, and sea salt aerosol components. The bias-correction process takes the raw(XCO2) and increases these values by approximately 2 ppm. The version 10 OCO-2 Data Product User's Guide (Orbiting Carbon Observatory-2 & 3 (OCO-2 & OCO-3), 2020) discusses the details of the raw to bcXCO2 equations, which are dependent upon footprint–land–ocean specifics. The need to bias correct is due to instrument calibration, spectroscopic line uncertainty, and physics not included in the operational retrieval code. The uncertainty quantification of OCO-2 measurements is discussed in Connor et al. (2016) and Hobbs et al. (2017).

Raw and bc XCO₂, bias-correction variables, and other data are conveniently contained in OCO-2 Lite files, with one Lite file per day that includes all daily operational retrievals. Data quality is indicated by quality flag =0 (QF0; best quality) and quality flag =1 (QF1; poor quality) data flags. The OCO-2 team discourages use of QF1 data in XCO₂ studies. In this paper, we do examine QF1 data in addition to the QF0 data since the QF1 data provides insights into 3D cloud radiative effects.

Of the approximately 1 × 10⁶ daily observations which are collected by OCO-2, about 25 % are passed into the operational retrieval due to prescreening for scenes contaminated by clouds and heavy aerosol loadings. Two cloud preprocessors (Taylor et al., 2016) exclude many of the soundings. One preprocessor only uses the O₂ A band to provide a computationally quick determination of O₂ A-band surface pressure, which is compared to a priori meteorological data. An observation is excluded from the operational retrieval if the difference in surface pressure is greater than 25 hPa. The second preprocessor performs single-band retrievals of XCO₂ using both the WCO₂ and SCO₂ bands independently. If the ratio of the two columns deviates significantly from unity, then the observation is excluded from the operational retrieval. This often identifies scenes with aerosol contamination due to the spectral dependence of aerosol absorption.

Not all known physics, however, is included in the operational retrieval. The OCO-2 operational retrieval does not know if clouds are outside of the row (frame) of the eight side-by-side detector footprints. The OCO-2 orbital track sweeps out a continuous swath of footprints with a swath width of less than 20 km (see Fig. 2.2; OCO-2 L2 ATBD, 2019). The detailed shape of an individual footprint, of the order of 2 km, varies according to viewing geometry (Crisp et al., 2017). Clouds outside of the swath can scatter photons into the region of the footprints, but the OCO-2 experiment cannot determine the location of clouds outside of the swath. Until the wall-clock advantages of parallel computing can be implemented in an operational setting, 3D cloud effects will remain computationally expensive in an operational setting.

In this paper, we utilize MODIS radiance data at 250 m resolution to study 36 scenes in detail. The 250 m radiances and MODIS cloud mask data are used to specify the locations of clouds in the vicinity of OCO-2 observations. The 36 scenes include 12 ocean glint, 12 land nadir, and 12 land glint scenes. A visual examination of NASA Worldview (https://worldview.earthdata.nasa.gov, last access: 14 April 2023) MODIS Aqua imagery and a listing of Lite file latitude, longitude, and QF flags suggested scenes in which several dozen Lite file XCO₂ values are present in each scene. Many of the scenes have sun–cloud geometry in which light is reflected off a cloud feature oriented approximately perpendicular to the incident solar beam (Fig. 6 is an example), with clear-sky gaps between the clouds. This geometry is conducive to the study of 3D cloud effects. For global analyses, we use the nearest-cloud distance files discussed in Massie et al. (2021), which are available at Zenodo (referred to as 3D metric files; https://doi.org/10.5281/zenodo.4008765; Massie et al., 2020). As discussed in Massie et al. (2021), the nearest-cloud distance data are based upon an analysis of auxiliary files (Cronk, 2018) that contains MODIS 500 m radiances, cloud mask, and geolocation information matched to OCO-2 observation geolocation.

Previous studies of 3D radiative transfer (Merrelli et al., 2015) applied the Spherical Harmonics Discrete Ordinate Method (SHDOM) 3D radiative transfer code (Evans, 1998) to perturb OCO-2-like spectra and looked at OCO-2-like retrievals with and without the 3D radiance perturbations. Retrieved XCO₂ values were lower than clear-sky retrievals by 0.3, 3, and 5–6 ppm for surfaces characterized by bare soil, vegetation, and snow-covered footprints.

Massie et al. (2021) calculated differences in XCO_2TCCON and XCO_2Lite file for QF0 and QF1 data for several years of OCO-2 measurements. Denoting XCO₂${}_{\mathrm{TCCON}}-{{\mathrm{XCO}}_{\mathrm{2}}}_{\mathrm{Lite}\mathrm{file}}$ as Res_D at distance D from the nearest cloud, residual differences (Res_10 km−Res_D) varied between 0.0 and 0.4 ppm and 0.0 and 2.5 ppm for QF0 and QF1 data, respectively, as the nearest-cloud distance D varied from 10 to 0.5 km. The residuals (with positive values indicating a retrieval underestimate of XCO₂) are present in both raw and bc XCO₂ and therefore are residuals of 3D cloud effects that are not accounted for by the 1D operational retrieval framework. Massie et al. (2021) also demonstrated that an extension of the linear bias-correction methodology, in relation to adding 3D cloud metrics to the bias-correction parameters, only marginally improved the accuracy of bias-corrected XCO₂.

The recent Emde et al. (2022) study discusses 3D calculations that cover a range of cloud scenes over Germany and the surrounding countries, based on a large-eddy simulation, and also a set of box clouds with various solar zenith and viewing angles and optical depths. The calculations were carried out in the 400–500 nm and O₂ A-band spectral ranges. The figures in Emde et al. (2022) for the box cloud calculations are particularly instructive, as they illustrate how reflectance varies as a function of distance from cloud edges for viewing geometry in which clouds enhance the radiance field and for viewing geometry in which cloud shadow effects are present. The Emde et al. (2022) calculations are available at https://doi.org/10.5281/zenodo.5567616 (Emde, 2021). The study of Emde et al. (2022) motivated us to calculate a set of SHDOM idealized bar cloud 1D and 3D calculations.

This paper is the follow-on work of Massie et al. (2021) and expands the previous research in significant ways. Massie et al. (2021) focused on quantifying 3D cloud effects based on comparisons of bcXCO2 and TCCON data. The geographical distribution of TCCON sites, however, is concentrated over North America and Europe, with sparse coverage over the tropics and over the ocean. The current paper calculates 3D cloud biases as a function of latitude, using the Zenodo 3D metric files for 275 times more data points. The latitudinal dependence of 3D cloud biases is not addressed in Massie et al. (2021) but is addressed in this paper. Massie et al. (2021) demonstrated that 3D cloud biases frequently indicate an underestimation of XCO₂ as the nearest-cloud distance decreases. The current paper provides a physical reason for why this is the case. Massie et al. (2021) examined one 32 km × 32 km scene, while this paper examines 36 scenes over the ocean and land for a variety of viewing geometries over the Amazon and the oceans, areas which are problematic for OCO-2. The current paper presents side-by-side graphs of 1D $/$ 3D intensity ratios (and other variables) as a function of nearest-cloud distance to illustrate the nonlinearities that are present in the OCO-2 data files. The nonlinearities that are present in our graphs for several variables are nonlinearities which machine learning (ML) bias-correction methods (see Mauceri et al., 2023) will need to mitigate.

This paper is organized in the following manner. The data utilized in our study are discussed in Sect. 2. Details of the SHDOM 1D (IPA – independent pixel approximation) and 3D radiance calculations are reviewed in Sect. 3.1. In Sect. 3.2, SHDOM idealized bar cloud calculations are discussed, which provide insight as to the variation in 1D $/$ 3D intensity ratios as a function of nearest-cloud distance, gas vertical optical depth, solar zenith angle, cloud height, and surface reflectance. Partial derivatives in radiance with respect to changes in pressure, XCO₂, reflectance, and aerosol total optical depth are presented in a graphical form in Sect. 3.3 to illustrate the zero-order physics associated with 3D cloud effects. In Sect. 4, individual scenes over the ocean and land are discussed to illustrate how the 1D $/$ 3D radiance intensity ratio varies near and far away from clouds. In Sect. 5, SHDOM calculations for 36 scenes (12 ocean glint, 12 Amazon land nadir, and 12 Amazon land glint) are discussed. The QF0 and QF1 observations illustrate the relationships between nearest-cloud distance and key variables (1D $/$ 3D SHDOM radiance intensity ratios, raw(XCO2), dP, surface reflectance, and aerosol optical depth). In Sect. 6, yearly calculations of dP and raw(XCO2) are presented as a function of latitude and as a function of nearest-cloud distance. Section 7 discusses calculations of the zonal averages of 3D cloud radiative effect biases as a function of latitude for bcXCO2 over ocean and land. Section 8 presents our summary and conclusions.

2 Data

OCO-2 product files are available from the NASA Earthdata website (https://earthdata.nasa.gov/, last access: 14 April 2023) and archived in the GES DISC (Goddard Earth Sciences Data and Information Services Center) repository. The first part of the specific version 10 (v10) file path names are given here in parentheses to identify, in a general sense, the files used in our study. OCO-2 Lite files (oco2_LtCO2_..) contain raw and bc XCO₂, bias-correction variables (such as dP), and other information. The land/water flag is used in our study to identify ocean and land observations, and the operational-mode flag is used to identify glint and nadir observations. Preprocessor (oco2_L2ABP_..; referred to as L2ABP) files specify cloud flags and continuum radiances for all measurements (including those which are not successfully retrieved). Level 2 diagnostic files (oco2_L2Dia_..) specify full spectra in all three bands. Meteorological (oco2_L2Met..) surface pressure is specified by Goddard Earth Observing System Model, version 5, Forward Processing (GEOS-5 FP-IT) data, which is used to calculate the dP values. The ND characters in the granule-level file path names refer to nadir observations, while glint files contain GL as part of the file path name.

Co-analysis of MODIS and OCO-2 data is made possible by way of MODIS 250 m radiances (MYD02QKM..), geolocation (MYD03..), aerosol (MYD04..), cloud data (MYD06..), and weekly surface reflectance (MYD09..). The MY.. prefix indicates MODIS Aqua data files. OCO-2 flies 6 min in front of MODIS Aqua in the NASA A-train, and therefore, the MODIS cloud field in the intermediate 6 min may differ slightly from the cloud field that impacts OCO-2 observations. The MYD06 data file specifies cloud heights for a given scene. Cloud heights vary between 0.6 and 3.7 km for the scenes examined in this paper, with 1.4 km being the average.

The 3D metric files contain nearest-cloud distance and other 3D cloud metrics, which are discussed in Massie et al. (2021). The other metrics (H(3D), H(Continuum), and CSNoiseRatio) are based upon the calculation of the standard deviations of the MODIS radiance field, OCO-2 continuum radiance field, and sub-footprint radiance fields, respectfully. These metrics are measures of radiance spatial gradients, which will be non-zero in the presence of clouds. In this paper, we focus on nearest-cloud distance, since Várnai and Marshak (2009) demonstrated that MODIS reflectance at various wavelengths between 0.47 and 2.13 µm increases the closer one is to clouds (i.e., nearest-cloud distance is a previously proven 3D-cloud-effect metric). There are 3D metrics for each successful OCO-2 QF0 and QF1 retrieval for 2014–2019.

3 Calculations

3.1 The SHDOM radiative transfer code

The Spherical Harmonic Discrete Ordinate radiative transfer Method (SHDOM) 3D radiative transfer code (Evans, 1998; Pincus and Evans, 2009) calculates 1D (single column; independent pixel approximation) and 3D fields. In the 1D calculation, there is no exchange of photons between adjacent columns. In the 3D calculation, columns do exchange photons. Of particular interest to our paper is the scattering of photons from the cloud and ground to adjacent columns. This exchange of photons between columns yields a 3D Stokes radiation field which differs from the 1D Stokes field. The four components (I, Q, U, and V) of the Stokes field can be used to define the amount of linear and circular polarization in the radiation field. We focus on the total intensity I component of the Stokes vector.

Input to SHDOM includes the specification of the vertical (z axis) gas optical depth structure and the x–y–z mass content structure of the cloud and aerosol fields. In our calculations, the horizontal x–y grid has a grid spacing of 0.5 km, and the input gas files have a 1 km grid spacing from the ground to 30 km altitude. The number of x- and y-axes grid points is 64, so the full SHDOM x–y grid covers a 32 km × 32 km area. Since the cloud and aerosol input file has an altitude grid at sub-1 km spacing near the ground, SHDOM interpolates the gas field extinction/kilometer values to the sub-1 km vertical grid of the cloud and aerosol input file. Gas field extinction/kilometer values are calculated by utilizing the OCO-2 V5.1 ABSCO data files (Payne et al., 2020). The ABSCO molecular cross sections for O₂, CO₂, and H₂O, specified at a resolution of 0.010 cm⁻¹ ($\mathrm{5.9}\times {\mathrm{10}}^{-\mathrm{7}}$ µm in the O₂ A band), are tabulated as a function of 64 pressures and 17 temperatures. Extinction/kilometer values are calculated for each x–y–z grid point, according to each grid point's temperature and pressure. The SHDOM calculations do not iterate for the surface reflectance. A constant Lambertian surface reflectance in each band for land observations is specified (hardwired as an input to SHDOM), based on the Lite-file-retrieved values. These values produce SHDOM 3D top-of-atmosphere reflectance in good agreement with the observed (archived Lite file) reflectance. For ocean glint observations, the Mishchenko and Travis (1997) implementation of the Cox–Munk wind-speed-dependent surface reflectance formulation is used in the SHDOM calculations, with wind speed specified (hardwired as an input to SHDOM) based on the Lite file retrieval of the wind speed.

For a specified wavelength, SHDOM calculates the Rayleigh scattering, aerosol, and cloud optical parameters (optical depth, asymmetry parameter, and single scattering albedo) for each x–y–z grid cell. For the scenes discussed in this paper, the aerosol and cloud radii are 0.1 and 10 µm, respectively. Water droplet clouds have a cloud base of 0.1 km up to a specified cloud height (e.g., 1.4 km), with a vertically constant cloud droplet liquid water content (LWC; in g m⁻³ units). The aerosol has equal-valued extinction/kilometer values from the ground level to 1.8 km altitude. The input aerosol mass content values are adjusted such that the total vertical aerosol optical depth is near a desired value (in the 0.05–0.16 range and usually less than 0.1). Since the aerosol optical depth is small, surface reflectance should have more influence on the top-of-atmosphere radiance than aerosol. SHDOM calculates the aerosol and cloud optical depth parameters by applying a Mie code, based on the input cloud and aerosol file, and for aerosol, the complex index of refraction is selected based on a sulfate aerosol.

SHDOM is configured to write out the Stokes field at the top-of-the-atmospheric grid for a set of sun-observation azimuth angles, the solar zenith angle, and the sensor view angle, for 17 total vertical gas optical depths. For the O₂ A band, the gas optical depths vary from $\mathrm{8}\times {\mathrm{10}}^{-\mathrm{4}}$ to 4.0. The lowest gas optical depth in each band corresponds to the band continuum.

The observed radiances are directly related to the SHDOM 3D radiances, since actual atmospheric radiances are the result of 3D radiative transfer processes. The 1D radiance field is not available from OCO-2 measurements, though the 3D radiances approach the 1D values as cloud distances become very large. Also, the OCO-2 radiances are dependent upon a linear combination of the I, Q, and U Stokes components (see Eq. 3-40; OCO-2 L2 ATBD, 2020). We focus on the SHDOM total intensities to gain insights with regard to 3D cloud effects for a variety of scenes and do not make detailed comparisons of observed spectra and the SHDOM spectra.

3.2 Idealized bar cloud calculations

The choice of idealized bar cloud calculations is motivated by a visual examination of various NASA Worldview scenes over the ocean and the Amazon. The Amazonian clouds are frequently distributed in cloud streets (Fig. 6 is a good example). Ocean scenes, in which an elongated cloudy area is associated with adjacent clear-sky regions, are useful to study 3D cloud effects (Fig. 7 is a good example). The cloud distributions in both scenes can be geometrically approximated by one or more idealized bar clouds.

Figure 1 illustrates a pair of idealized bar clouds, referred to as the left and right bar clouds. The x width of each cloud is 3 km, and the bar clouds extend over the full y axis of the 32 km × 32 km scene. The clouds are assigned a specific solar zenith angle and cloud altitude (1.4 km). The Sun is along the x axis, and SHDOM is configured to calculate 12 sensor azimuth angles from 0 to 360^∘ in 30^∘ steps. Figure 1 illustrates one of the 12 azimuth angles, with the sensor to the right of the right bar cloud. The inclusion of the two clouds in the scene allows for an analysis of 1D and 3D radiances that result from scattering of photons off the right bar cloud, back towards the observation footprint (Obs), and for the case in which (for the left bar cloud) a cloud-shadowing effect is present if the observation point is close to the left bar cloud. The spatial extent of the cloud shadowing is dependent on the height of the cloud and the solar zenith angle. Analysis of the 1D and 3D radiances as a function of distance D from the bar clouds yields insights into the nature of 3D radiative effects as a function of gas vertical optical depth.

Figure 1 Idealized bar clouds for a cloud height of 1.4 km. The Sun and OCO-2 are to the left and right of the 32 km × 32 km SHDOM scene, and two observation points are at distance D from the left and right clouds. Two sensor beams are indicated by the upward-sloping arrows.

The 1D $/$ 3D ratio, and its variation in a scene, is fundamental to this paper, since it is a measure of the size of the 3D cloud effect in the scene. Figure 2 presents 1D $/$ 3D ratios as a function of cloud distance D from the right bar cloud for all three OCO-2 bands. Land nadir geometry is applied, with a solar zenith angle of 38^∘ (with atmospheric model specifics associated with the Amazon scene discussed in Fig. 6 below). The curves pertain to a low gas vertical optical depth near 0.01. The 1D $/$ 3D ratios approach unity as the cloud distance increases. The ratios for the optically thin regime, however, are not equal to 1.0 at the largest cloud distance, since photon paths are present in the 3D case in which light propagates into the top of the cloud and exits out the sides, adding to the 3D radiances in the regions between the clouds. As the cloud distance D decreases towards zero, the 1D $/$ 3D ratio becomes small, near 0.8. Notice that this drop off in the ratio is very nonlinear and takes place at cloud distances approximately less than 4 km. Also notice that the curves for the three bands are quantitatively different. This implies that a detailed understanding of 3D radiative effects requires attention to the details in each of the three OCO-2 bands.

Figure 2 Illustration of 1D $/$ 3D intensity ratios as a function of distance D (in km) from the right cloud in Fig. 1. The 3D cloud effects are present in all three OCO-2 bands, with the largest 3D effects (smallest 1D $/$ 3D ratios) in the O₂ A band. The ABSCO vertical gas optical depth is near 0.01.

From Fig. 2, it is apparent that 1D $/$ 3D ratios asymptote for a length scale of approximately 10 km. The periodic boundary conditions used by SHDOM therefore do not cause clear-sky pixels near the left cloud to be impacted by photons that, after being scattered by the right-side cloud, move across the right edge of the scene and reappear at the left edge. These considerations motivated our selection of the Fig. 1 geometry and the selection of a 32 km × 32 km SHDOM grid.

Considering the case where the 1D $/$ 3D ratio is less than unity due to 3D cloud effects, the OCO-2 experiment measures the true radiance, which is a 3D radiance, since the real atmosphere exchanges photons between adjacent columns. The operational OCO-2 retrieval calculates 1D column radiances and inserts no physics due to adjacent column 3D cloud effects. With 1D $/$ 3D less than unity, the retrieval needs to enhance the 1D radiance by modifying variables (such as surface reflectance, aerosol, surface pressure, and XCO₂) in order to bring forward model and observed radiances into agreement. This is illustrated below in Sect. 5, in relation to Figs. 8, 9, and 10.

Figure 3 displays 1D $/$ 3D ratios as a function of cloud distance D for the left and right bar clouds for the O₂ A-band continuum (gas optical depth near 0.008). The Sun and sensor are along the −x and +x branches of the x axis. For the left bar cloud (Fig. 3a), the ratio decreases slightly as the cloud distance decreases from large values and then increases drastically (towards 5.4) as distance D approaches zero. This behavior is due to cloud-shadowing effects. For graphical convenience, the 1D $/$ 3D cloud-shadowing ratios greater than 1.1 are set to a maximum value of 1.1, and the minimum ratio of 0.45 was set to 0.70, so that y axis ranges are the same in both panels. The lowest 1D $/$ 3D ratio in Fig. 3a is due to an increase in 3D photons originating (leaking from) the adjacent cloud column. The 1D radiance is not susceptible to cloud shadowing, since cloud shadowing originates from the sunward-adjacent column. The 3D radiance is susceptible to the shadowing from the adjacent column. The 3D radiance fields are therefore susceptible to both cloud-shadowing (dimming) and cloud-brightening effects (Fig. 3b). The prevalence of cloud-brightening versus cloud-shadowing effects is discussed in Sect. 4.

Figure 3 The 1D $/$ 3D intensity ratios as a function of distance D for (a) left and (b) right idealized bar clouds and for the O₂ A-band continuum (vertical gas optical depth is 0.0008). Cloud shadows are present in panel (a) (with ratios greater than unity), and 3D cloud brightening is present in panel (b). The idealized bar clouds have the same altitude, pressure, and temperature profile as that of the detailed SHDOM calculation in Fig. 6 for 22 June 2015 at 6.52^∘ S and longitude −55.98^∘ W. Ratios greater than 1.1 in panel (a) were reset to 1.1, and the minimum of 0.45 was reset to 0.75 in order to have the same y axis range in both panels.

Figure 4 indicates the sensitivity of 1D $/$ 3D ratios to gas vertical optical depth, solar zenith angle, cloud height (cloud vertical extent), and surface reflectance. The curves are those from the cloud-brightening calculations for the right bar cloud. The largest sensitivity is due to the gas optical depth (Fig. 4a). The sensitivity is largest for the smaller nearest-cloud distances. As the gas optical depth increases, the ratios become closer to unity, and the curves drop off to lower ratios at increasingly smaller cloud distances. For the case where optical depths become very large, the ratio approaches unity for all cloud distances. This is reasonable, since at very large gas optical depths the vast majority of the photon paths are located at large heights above the surface, and these photons do not interact with the low-level clouds.

Figure 4 All parts of this figure pertain to a cloud-brightening situation near the right-hand-side cloud. (a) Dependence of O₂ A-band 1D $/$ 3D ratios on the ABSCO gas vertical optical depth and as a function of nearest-cloud distance. The cloud-top height is 1.4 km, the Lambertian surface reflectance is 0.30, and the aerosol vertical optical depth is 0.16. (b) Dependence of 1D $/$ 3D ratios on the solar zenith angle. The ABSCO gas vertical optical depth is 0.0008, and the Lambertian surface reflectance is 0.30. (c) Dependence of 1D $/$ 3D ratios on cloud vertical thickness, labeled by the cloud-top height, with a cloud base at 0.1 km. The ABSCO gas vertical optical depth is 0.0008, the Lambertian surface reflectance is 0.30, and the solar zenith angle is 40^∘. (d) Dependence of 1D $/$ 3D ratios on the Lambertian surface reflectance. The ABSCO gas vertical optical depth is 0.0008, and the solar zenith angle is 40^∘. (e) Dependence of 1D $/$ 3D ratios on the cloud LWC (gm m⁻³) value. (f) Dependence of 1D $/$ 3D ratios on the aerosol size distribution and indices of refraction.

The second-largest sensitivities are due to the solar zenith angle (Fig. 4b) and band cloud height (Fig. 4c). The sensitivity to solar zenith angle is reasonable, since, for larger solar zenith angles, photons are scattered off of the sides of the clouds, while for a solar zenith angle near zero, the 3D radiative effect is constrained by photon paths passing through the top of the cloud (followed by some exiting of photons to the side) and/or secondary paths (Sun to surface, surface to cloud, and cloud back to surface).

In Fig. 4c, the sensitivity to cloud thickness is illustrated (labeled by the cloud-top height) for a cloud base of 0.1 km. As the vertical extent of the cloud is increased, there is more side surface area present, increasing the number of sun-to-cloud to surface photon events. As noted by Taylor et al. (2016), the cloud preprocessor does a good job of screening for clouds, though the preprocessor can pass some cases in which low-level clouds are present.

Figure 4d indicates that 1D $/$ 3D ratios are not sensitive to surface reflectance for the land nadir view geometry. Since the 1D path (sun to surface to sensor) is dependent on the surface reflectance, and the 3D situation (with added sun–cloud–surface–sensor paths) is also dependent on the surface reflectance, some cancellation in the surface reflectance term is expected. The 1D $/$ 3D ratios were also calculated for several cloud LWC values (Fig. 4e), sulfate, sea salt (Fenn et al., 1985), and brown carbon aerosol (due to biomass fires; Alexander et al., 2008; Fig. 4f). While there are important variations in the details shown in Fig. 4, all panels display a noticeable nonlinear decrease in 1D $/$ 3D ratios as the nearest-cloud distance decreases.

A version of Fig. 4 for left bar clouds (not shown) has curves that differ from the right bar cloud calculations of Fig. 4 in the same way that Fig. 3a and b differ. The 1D $/$ 3D intensity ratio curves decrease as the nearest-cloud distance decreases, and then each curve begins to increase towards values greater than unity a few kilometers from the shadow side of the left bar cloud. The nearest-cloud distance for these inflection points varies from 2 to 4 km for solar zenith angles between 20 and 50^∘ solar zenith angle (SZA). This is reasonable, since the cloud shadow extends a distance H_cld tan(SZA) for a cloud height in kilometers of H_cld. A comparison of the two graphs indicates that the left bar curves are closer together (in the y axis separation) for the four solar zenith angle curves than the solar zenith angle curves in Fig. 4, while the LWC curves have more y axis separation near the inflection points. The Zenodo site (https://doi.org/10.5281/zenodo.7655136; Massie, 2023) contains the numerical data for Fig. 4 and its left bar cloud equivalent.

3.3 Radiance perturbation sensitivity

Figure 5 illustrates the sensitivity of radiances to perturbations in XCO₂, surface pressure, surface reflectance, and total optical depth. The x axis variable specifies the total vertical optical depth (gas plus aerosol) for each band, with each vertical optical depth corresponding to a specific ABSCO wavelength, while the y axis variable specifies the radiance perturbation. The lowest vertical optical depths are in the continuum portion of the spectra, while the largest vertical depths are in absorption lines. The information in Fig. 5 is calculated from monochromatic, high spectral resolution ABSCO data. Monochromatic radiance derivatives are presented in Fig. 5.

Figure 5 Radiance perturbations based upon an Amazon nadir scene atmospheric profile using idealized bar clouds. The total vertical optical depth on the x axis is the sum of the gas and aerosol vertical optical depths. Each vertical optical corresponds to a specific ABSCO wavelength. The lowest vertical optical depths are in the continuum portion of the spectra, while the largest vertical depths are in absorption lines. The observation point is 4 km from the cloud. The “A” character in the y axis label stands for the 1D perturbation radiance or the 3D radiance. Aerosol (Aer) optical depths and surface reflectivity (Refl) are perturbed by 10 % from the Table 1 baseline values, while the surface pressure (P) is perturbed by 2 hPa, and CO₂ in the WCO₂ and SCO₂ bands is perturbed by 1 ppm.

Denoting R as the radiance intensity, the 3D curves are 100 ($R_{\mathrm{3}\mathrm{D}}-R_{\mathrm{1}\mathrm{D}})/R_{\mathrm{1}\mathrm{D}}$ values in each panel, where 3D and 1D refer to SHDOM 3D and 1D calculations. The model atmosphere (temperature profile, gas, aerosol, and cloud optical depth structure) is the same in the 1D and 3D curves, but the 3D radiances are those due to SHDOM accounting for the exchange of photons between columns. Once specified, the model atmosphere is fixed in a SHDOM simulation. The observation point (see Fig. 1) is 4 km to the left of the right bar cloud, with the Sun (sensor) to the left (right) of the observation point. The other curves are partial derivatives of the 1D radiances; i.e., $\mathrm{100}(R_{\mathrm{1}\mathrm{D}\mathrm{perturbation}\mathrm{case}}-R_{\mathrm{1}\mathrm{D}\mathrm{baseline}\mathrm{case}})/R_{\mathrm{1}\mathrm{D}\mathrm{baseline}\mathrm{case}}$. The model atmosphere is the same in the 1D baseline and 1D perturbation cases, except for a perturbed value in one variable. Baseline conditions and perturbations are specified in Table 1. The solar zenith and sensor view angles are 38 and 0^∘, respectively, in all of the calculations. The atmospheric profile corresponds to the Amazon scene associated with Fig. 6 that is discussed in Sect. 4. Sulfate aerosol extends from ground to 1.8 km, with a total aerosol optical depth near 0.067 in the O₂ A band, cloud LWC is 0.30 g m⁻³, and the idealized bar clouds have a cloud top at 1.4 km.

Table 1 Baseline and perturbations of the Fig. 5 calculations.

Figure 6 (a) MODIS radiance field (units are in $\mathrm{W}{\mathrm{m}}^{-\mathrm{2}}{\mathrm{sr}}^{-\mathrm{1}}\mathrm{\mu }{\mathrm{m}}^{-\mathrm{1}}$) on 22 June 2015 over the Amazon, as a function of longitude and latitude, and the location of v10 Lite file observations, marked by the □ (QF0) and X (QF1) symbols. The observations are between the clouds (the irregular green and yellow areas). The direction of the incident sunbeam is from the northwest, and north is at the top. (b) SHDOM calculation of 1D $/$ 3D ratios for the 22 June 2015 Amazon scene. The smallest 1D $/$ 3D ratios are located on the sunward side of clouds (white). Note that panel (b) only covers a portion of the spatial extent of panel (a).

Figure 5 illustrates that an increase in radiance comes about (in a partial derivative sense, with other variables held constant) if the surface reflectance or aerosol optical depth is increased, if the surface pressure is decreased, or if XCO₂ is decreased in the WCO₂ and SCO₂ bands. Away from clouds (in the absence of radiance dimming due to cloud shadows), the 3D effect increases radiance. Since the 3D effect is substantial at all optical depths, it is expected that the retrieval will definitely adjust the surface reflectivity and/or aerosol in the state vector, since these variables have radiance partial derivatives that are also non-zero at all optical depths. The surface pressure perturbations in all three bands, and the WCO₂ and SCO₂ XCO₂ perturbations, have radiance partial derivatives that are small at small gas optical depths and appreciable only at the larger vertical optical depths. Figure 5 does not indicate how the 3D radiance perturbation is accounted for in the operational retrieval by perturbations in surface reflectance, aerosol, surface pressure, and XCO₂. This question is addressed below in Sect. 5.

Figure 5 illustrates that OCO-2 observations are susceptible to 3D cloud effects. The 3D effect is present in all three bands, increasing 1D continuum (smallest optical depth) radiances by 3 %, 2 %, and 1% in the O₂ A, WCO₂, and SCO₂ bands when the observation point is 4 km from the right bar cloud. The spectral variations in the 3D radiance perturbations in Fig. 5 are distinct (different from the other perturbations), which forces adjustments in the retrieval state vector variables. The 3D perturbations are larger (smaller) for smaller (larger) cloud distances. As stated in the introduction, the OCO-2 measurement goal is to measure XCO₂ to the 1 ppm level. Perturbations in XCO₂ of 1 ppm, however, perturb radiances on the 1 % level only at large optical depths and less so at smaller optical depths. If the 3D radiance perturbations were substantially smaller than the radiance perturbation corresponding to a 1 ppm increase in XCO₂, and if the spectral variations in the 3D radiance perturbations were not different from the other perturbations, then the OCO-2 observations would not be susceptible to 3D cloud effects, but this is not the case. Since 40 % of all OCO-2 observations are within 4 km of clouds, 3D cloud radiative effects impact many OCO-2 observations (Massie et al., 2021).

The wavelengths selected in Fig. 5 are representative. A different set of wavelengths would produce derivatives, especially for the pressure and CO₂ derivatives in the SCO₂ band at optical depths greater than 2 that differ from those shown in Fig. 5. The key point of Fig. 5 is that the pressure and CO₂ derivatives are negative, ranging from 0 % to −1 % and are of similar absolute size to the 3D radiance perturbations, which vary from 0 % to 3 % for an observation 4 km from the nearest cloud. Figures 4 and 5 are the only figures in this paper that present information that relates to non-continuum wavelengths.

Figure 5 illustrates the zero-order physics associated with 3D radiative transfer. The discussion then needs to proceed to ask how the operational retrieval responds to the 3D radiance enhancements due to 3D cloud-brightening effects. Since the operational retrieval does not insert any 3D radiative transfer physics into the retrieval, there is a needed adjustment of the state vector element values to bring forward model and observed radiances in agreement. The operational retrieval obtains a solution state vector with the specified surface pressure, surface reflectivity, aerosol, and XCO₂, which brings forward model radiances in line with observed radiances. There is no reason to assume that only the surface pressure, surface reflectivity, and aerosol state vector variable values are numerically adjusted by the forward model, thus yielding the needed radiance enhancement due to 3D cloud radiance brightening, with XCO₂ also not being numerically adjusted by the forward model, with contributions to the needed radiance enhancement.

4 Amazon and ocean glint scenes

Figure 6 presents the detailed MODIS radiance field for a scene over the Amazon on 22 June 2015. The direction of the incident sunbeam is from the northwest at the solar zenith angle of 38^∘, while the OCO-2 sensor angle is 0^∘. The altitude, pressure, and temperature model atmosphere used in the simulation is derived from the oco2_L2MetND file, with specifics listed in Table 1. Clouds are specified by an analysis of the MODIS MYD02QKM 250 m radiance, MYD03 geolocation, and MYD06 cloud fields. The MYD06 cloud field identifies some clouds, and these clouds are used in conjunction with the MYD02QKM radiances to establish a cloud radiance threshold. The MYD06 cloud field, however, does not identify all clouds in a scene. This is apparent by examining the MODIS 250 m radiance field and the MYD06 cloud field (Massie et al., 2017). Once the cloud radiance threshold is established from an examination of the MODIS radiance field, then clouds are assigned to all x–y grid points if the MODIS radiance is greater than the threshold value. In Fig. 6a, clouds are present when the radiance is greater than 80 $\mathrm{W}{\mathrm{m}}^{-\mathrm{2}}{\mathrm{sr}}^{-\mathrm{1}}\mathrm{\mu }{\mathrm{m}}^{-\mathrm{1}}$.

Note that the locations of the Lite v10 file data points are between the clouds and are indicated by the □ and X symbols for the QF0 and QF1 retrievals. The cloud preprocessor does a good job of screening for observations over clouds, and the operational retrieval does not converge successfully for these data points.

Table 2 (first column) presents statistics for the latitude range 10^∘ S to 3^∘ N on 22 June 2015, which includes many more data points than those displayed in Fig. 6a. Of 5162 OCO-2 observations, 589 observations (11 %) were successfully retrieved, with 40 % and 60 % QF0 and QF1 retrievals, respectively. Approximately 80 % of the retrievals are located within 4 km of clouds.

Table 2 Statistics of various quantities (in %) for single-day and 2016 yearly averages based upon an analysis of L2ABP preprocessor data files.

^a Total number Obs is the total number of observations in the latitude–longitude range, as specified by the L2ABP file. ^b Number of retrievals is the number of successful retrievals, as specified by the Lite file. ^c Percentage cloud <4 km is the percentage of retrievals for which the cloud distance is less than 4 km. ^d Percentage geom, 2 km, is the percentage of retrievals associated with cloud shadows for the cloud height in 0 to 2 km altitude range.

The prevalence of cloud-brightening versus cloud-shadowing effects for the Amazon scene is revealed in Table 2. Table 2 specifies the percentage of total retrievals which are associated with cloud shadows, assuming that the cloud heights are 2, 4, 6, or 8 km. The percentages for each cloud height are calculated based on the algorithm described in Appendix A. The algorithm utilizes O2ABP preprocessor cloud flags to identify clouds and clear observations and OCO-2 Level 1B data files that specify O₂ A-band continuum radiances. The continuum radiances and cloud flags are used together to specify clear and cloudy radiance thresholds.

Of the 589 successful retrievals for the Table 2 Amazon 150622 case, only eight retrievals (1.3 %) are associated with shadows, and the other retrievals ($\mathrm{100}\mathit{\%}-\mathrm{1.3}\mathit{\%}=\mathrm{98.7}\mathit{\%}$) are associated with cloud brightening, assuming that all cloud heights are 8 km in vertical extent. The percentages are lower for the assumed lower cloud heights. The retrievals associated with cloud shadows are QF1 data points, while Table 2 indicates that retrievals associated with cloud brightening have QF0 percentages between 34 % and 60 % for the various cases. The additional columns of Table 2 indicate that the Pacific glint observations (the 12 June 2016, 160622 case, and discussed below) and yearly averaged percentages over the Amazon and Pacific in 2016 are less than 4 %, even if all cloud heights are 8 km. Cloud heights, however, are less than 8 km. Application of NASA Giovanni (https://giovanni.gsfc.nasa.gov/giovanni/, last access: 14 April 2023) analysis of MODIS MYD08 data files yields histograms (not shown) of cloud-top temperatures and pressure means which correspond to cloud-top heights between 1 and 2 km for the 150622 and 160622 cases and heights between 2 and 3 km for the 2016 Amazon and Pacific yearly averages. Cloud brightening is therefore prevalent compared to cloud shadowing.

Figure 6b presents the SHDOM calculation of 1D $/$ 3D ratios for the Amazon scene for a gas vertical optical depth of 0.0008 (an optical depth in the O₂ A-band continuum). Since the sunbeam direction is from the northwest, the sunward side of clouds are located on the northwestern side of the clouds. The v10 Lite file soundings have 1D $/$ 3D ratios in the 0.56–0.96 range, with an average of 0.91. Successful OCO-2 retrievals are therefore susceptible to significant 3D cloud radiative perturbations of the order of 9 %.

Figure 7 presents a glint scene at 10^∘ N on 12 June 2016 over the Pacific, with a solar zenith angle of 24^∘ and sensor view angle of 19^∘. The □ and X symbols mark the locations of the Lite file QF0 and QF1 retrievals in the MODIS radiance field. The date is near the summer solstice, and the sunbeam direction is from the northwest. More QF0 data (the □ symbols) are located to the south of the cloud in the center of the frame, with fewer □ symbols on the sun-reflective side of the cloud (the region northwest of the cloud). This situation is not, however, generally the case, since a visual examination of figures similar to Fig. 7 for the other ocean glint scenes listed in Table 3 did not show this behavior. An examination of the NASA Worldview imagery for the Fig. 7 scene did indicate that there are more very small cloud remnants north of the main cloud region, with a very clear region south of the main cloud. The visual examination of the 12 scenes does indicate that QF1 data points are consistently closer to clouds than the QF0 data points.

Figure 7 (a) MODIS radiance field (units are $\mathrm{W}{\mathrm{m}}^{-\mathrm{2}}{\mathrm{sr}}^{-\mathrm{1}}\mathrm{\mu }{\mathrm{m}}^{-\mathrm{1}}$) on 12 June 2016 over the ocean, as a function of longitude and latitude, and the location of v10 Lite file observations, marked by the □ (QF0) and X (QF1) symbols. The observations are between the clouds (the irregular green and yellow areas). The direction of the incident sunbeam is from the northwest, and north is at the top. (b) SHDOM 1D $/$ 3D ratio field for the 12 June 2016 ocean glint scene. Notice the increase in the 1D $/$ 3D ratios as the distance from the clouds increases, with clouds corresponding to areas with 1D $/$ 3D ratios greater than unity. Panel (b) only covers a portion of the spatial extent of panel (a).

Table 3 Specifics of the multiple scenes.

^a OG, LN, and LG refer to ocean glint, land nadir, and land glint observing modes. ^b 160612 refers to 12 June 2016.

Figure 7b presents the SHDOM 1D $/$ 3D ratios for this oceanic scene. Figures 7 and 6 are similar in that the smallest 1D $/$ 3D ratios are located close to clouds. The v10 Lite file data points have 1D $/$ 3D ratios in the 0.81–0.99 range, with an average of 0.98, which is larger than the Fig. 6b Amazon scene average v10 1D $/$ 3D ratio of 0.91. The spatial extent of the lowest 1D $/$ 3D ratios near cloud edges in Fig. 7 is less than the spatial extent of the lowest 1D $/$ 3D ratios in Fig. 6b, which motivated the difference in the Figs. 6b and 7b color bar scales.

5 Analysis of multiple scenes

Lite file variables and SHDOM 1D and 3D radiance fields are analyzed for 36 individual scenes (12 ocean glint, 12 land nadir, and 12 land glint geometry). The specifics for the scenes are given in Table 3. A range of solar zenith angle from 20 to 55^∘ characterizes the ocean glint scenes. The land scenes are situated over the Amazon. It is of interest to study Amazon scenes, since there are relatively few successful QF0 data points over the Amazon, and the Amazon is of large importance to the global carbon cycle. The majority of completed retrievals over the Amazon are QF1 retrievals. Several Amazon scenes were chosen purposely to make sure that there were at least some QF0 data points in the scenes.

Figure 8 presents results for individual sun glint retrievals over the Pacific. QF0 and QF1 data points are given by the green (^*) and blue (+) symbols. In Fig. 8a, SHDOM 1D $/$ 3D ratios for continuum O₂ A band (for the smallest gas optical depth) versus the nearest-cloud distance taken from the 3D metric file are graphed. The 1D $/$ 3D ratios are near unity for cloud distances greater than 4 km. The ratios become smaller, with smallest values near 0.3, as the nearest-cloud distance approaches 0. The largest number of QF0 data points are for large cloud distances, while the largest number of QF1 data points are for small cloud distances.

Figure 8 Individual Lite file QF0 (^*; green) and QF1 (+; blue) SHDOM continuum 1D $/$ 3D, OCO-2 dP, raw(XCO2), total aerosol optical depth, Albedo1 (for the O₂ A band), and delta wind (see text), as a function of nearest-cloud distance for the 12 ocean glint scenes.

In the operational OCO-2 bias-correction processing step, a specified limited range for 31 variables determines if a retrieval is a QF0 data point (see Table 3.4 of the Orbiting Carbon Observatory-2 & 3 (OCO-2 & OCO-3), 2020). As an example, a retrieval is a QF0 data point if dP is between −7.5 and 8 hPa for land observations for the v10 data files.

In Fig. 8b, the dP values for QF1 data points take on large negative values as the cloud distance approaches zero. The interpretation of Fig. 8a and b is that the 1D $/$ 3D ratio becomes small as the 3D cloud effect enhances (brightens) the radiances. In order for the retrieval to match the forward model with the observed radiance, the retrieval decreases the surface pressure to smaller values (compared to the meteorological surface pressure field) at small nearest-cloud distance.

To place the various scene XCO₂ values onto a common framework, we calculate the average QF0 XCO₂ for a 6^∘ range of latitude centered on the scene's latitude. The average is then subtracted from the QF0 and QF1 rawXCO2 for a specific scene, and these adjusted rawXCO2 values are placed into our graphs. In Fig. 8c, the adjusted rawXCO2 varies from −2 to 2 ppm for the QF0 data at all nearest-cloud distances. For the QF1-adjusted rawXCO2 data, the values take on increasingly negative values as nearest-cloud distance decreases. The concurrent decrease in the 1D $/$ 3D ratios and rawXCO2, as nearest-cloud distance decreases, is evidence of the 3D cloud effects in the OCO-2 retrievals, especially for the QF1 observations.

Figure 8d displays total aerosol optical depth, which takes on increasingly larger values as nearest-cloud distance decreases for the QF1 data. The percentages of QF0 and QF1 data points at cloud distances of less than 4 km are 51 % and 86 % of the total number of QF0 and QF1 data points, respectively. Since each scene has a different inherent aerosol optical depth, it is expected that there will be several green (^*) sets of data points (the rightward directed spikes of total aerosol optical depth in Fig. 8d).

In the v10 retrieval, surface reflectance is represented by the sum of the Cox–Munk surface glint and Lambertian surface terms. Figure 8e displays the retrieved O₂ A-band Lambertian surface reflectance (Albedo1) values added to the Cox–Munk term. (The v10 Lite files do not specify the total surface reflectance values over the ocean, while the files do specify the total surface reflectance over land). Figure 8f displays delta wind values (the difference in retrieved wind speed and a priori wind speed in m s⁻¹ units). Since an increase in wind speed generally leads to a smaller surface reflectance, the positive delta wind (Fig. 8f) and Albedo1 values for QF1 data indicate that the retrieval selects decreasing Cox–Munk and increasing Lambertian contributions to the total surface reflectance as nearest-cloud distance varies from large to small values.

Figure 9 displays graphs for the Amazon land nadir scenes. The behavior of the data points is similar to the ocean glint data, though the range of the nearest-cloud distance is smaller than for the ocean glint scenes. Since the selected ocean and Amazon scenes were picked in a purposeful manner, this range difference is not generally true (see the next section for the yearly analyses). The 1D $/$ 3D ratios in Fig. 9a, however, do decrease as the cloud distance decreases. As cloud distance decreases, the adjusted XCO₂ values take on increasingly negative values (Fig. 9c). The range in QF1 XCO₂ for cloud distances greater than 5 km is larger for the Amazon land nadir scenes. Standard deviations for ocean glint XCO₂ are generally smaller than land XCO₂ (see Fig. 18 of O'Dell et al., 2018, and Table 4 of Massie et al., 2021). The percentages of QF0 and QF1 data points at cloud distances less than 4 km are 74 % and 84 % of the total number of QF0 and QF1 data points, respectively.

Figure 9 Individual Lite file QF0 (^*; green) and QF1 (+; blue) SHDOM continuum 1D $/$ 3D, OCO-2 dP, raw(XCO2), total aerosol optical depth, Albedo1 (for the O₂ A band) and Albedo2 (for the WCO₂ band) data, as a function of nearest-cloud distance for the 12 Amazon nadir scenes.

From Fig. 5 and Table 1, a change in 0.02 and 0.01 in the O₂ A band and WCO₂ surface reflectance yields an increase in radiance, which is 3 and 5 times as large, respectively, as the 3D radiance perturbation, so small changes in the QF0 and QF1 surface reflectance values (referred to as albedo values in the Lite files and in Figs. 9 and 10) add a sufficient increase in radiance that brings forward model and observed radiances into agreement. There is a noticeable difference in Figs. 8 and 9 in that the ocean glint scenes have dP of less than −10 hPa for the QF1 data, while there are a few dP values of less than −10 hPa in Fig. 11 for the land nadir scenes.

Figure 10 Individual Lite file QF0 (^*; green) and QF1 (+; blue) SHDOM continuum 1D $/$ 3D, OCO-2 dP, raw(XCO2), total aerosol optical depth, Albedo1 (for the O₂ A band) and Albedo2 (for the WCO₂ band) data, as a function of nearest-cloud distance for the 12 Amazon glint scenes.

Figure 11 Zonal averages of nearest-cloud distance over ocean and land for 2016 QF0 and QF1 data.

Figure 10 displays Amazon land glint data. The 3D metric files have nearest-cloud distance data for 4 of the 12 land glint scenes (the LG 150625 scenes, as specified in Table 3). Nearest-cloud distance values for the LG 200603 and LG 200610 scenes in Table 3 were calculated based on Lite file longitude and latitudes and 250 m MODIS radiance fields. SHDOM 1D $/$ 3D ratios and rawXCO2 decrease as cloud distance decreases, while total aerosol optical depth increases, especially for the QF1 data. In contrast to Fig. 9, the retrieval selects O₂ A band and WCO₂ band surface reflectivity at small cloud distances for some (∼ 12) of the data points, which are smaller than the surface reflectivity at large cloud distance.

The 3D cloud effects in ocean glint, Amazon nadir, and Amazon glint observations are evident in Figs. 8, 9, and 10, since 1D $/$ 3D ratios and raw(XCO2) concurrently decrease as nearest-cloud distance decreases, especially for the QF1 data points. Using Fig. 5 as a rough guide, an increase in aerosol, an increase in surface reflectance, and decreases in surface pressure and XCO₂, yield positive radiance perturbations. There are differences over ocean and land retrievals in that dP variations are smaller over land (compared to the large decrease in dP for the ocean glint retrievals; Fig. 8b) and that there are some retrievals for land glint observations in which decreases in surface reflectance are present as nearest-cloud distance decreases. There is commonality in Figs. 8, 9, and 10 in that the y axis spread in dP, XCO₂, aerosol, and delta wind is largest as nearest-cloud distance decreases.

The fact that cloud brightening dominates in Fig. 8, compared to cloud shadowing, has important implications for the Fig. 10 OCO-2 retrievals. From Fig. 3a, cloud shadows are associated with SHDOM 1D $/$ 3D ratios greater than unity. Figure 8a has observed Lite file retrievals over the ocean, with SHDOM 1D $/$ 3D ratios decreasing to low values as nearest-cloud distance decreases, with little evidence of ratios greater than unity. The 1D $/$ 3D intensity ratio averages are 0.98 and 0.96 for the QF0 and QF1 data points. The radiance brightening is associated with retrieval dP (retrieved − meteorological field) values less than 0, with a sharp decrease in dP for small cloud distances. Figure 10a, b, and c are consistent with the presence of 3D cloud effects in OCO-2 data, since SHDOM 1D $/$ 3D ratios, retrieval dP, and raw(XCO2) decrease as the nearest-cloud distance decreases. The cloud brightening is accounted for by the retrieval by a combination of increases in aerosol optical depth (Fig. 10d), and decreases in dP and XCO₂, as the nearest-cloud distance decreases. This is most apparent in the QF1 data points in Fig. 8. This is also illustrated by Fig. 11 in Massie et al. (2021) for a larger set of data points.

6 Yearly analysis

Generalization in differences in ocean glint, land nadir, and land glint are best made from an analysis of yearly averaged data, using the daily Lite file and 3D metric files for each full year of OCO-2 data during 2015–2018. Since the 3D metric files do not include 1D $/$ 3D SHDOM ratios, which are only available for the 36 scenes discussed above, the nearest-cloud distance is the primary 3D metric utilized in this section.

The nearest-cloud distance averages for 2016 are presented in Fig. 11 for ocean glint, land nadir, and land glint observations. It is apparent that clouds are closer on average for OCO-2 observations over the ocean than over land. This is especially true for the QF1 data, which are approximately 4 km on average from clouds over the ocean. The 4 km mark is important, since the curves in Figs. 2–4 become very nonlinear at distances near and below the 4 km cloud distance. The cloud distance is noticeably smaller over the Equator, over both land and ocean, with a symmetrical appearance with respect to the Northern and Southern hemispheres.

Averages of dP for several 10^∘ latitude bins in the tropics are displayed in Fig. 12, with −10 referring to the 10 to 0^∘ S latitude bin, etc. The QF1 ocean glint curves are best defined (Fig. 12a), with large negative dP values at cloud distances less than 5 km and dP between −1 and 1 hPa for cloud distances greater than 5 km. This signature is evidence of 3D cloud effects that impact OCO-2 radiances. The data points in Fig. 8b are consistent with the Fig. 12a dP tropical averages. The land nadir and land glint dP curves (Fig. 12b and c) in the tropics, however, increase in dP by several hectopascals as the cloud distance decreases, with dP values in the positive 0 to 4 hPa range.

Figure 12 The dP averages as a function of nearest-cloud distance. The value of −10 refers to the 10–0^∘ S latitude band.

Averages (denoted as Ave) of rawXCO2 for the same tropical latitude bins in Fig. 12 are displayed in Fig. 13 for 2016. The QF1 XCO₂ curves are again best defined for the ocean glint curves, with fewer oscillations in the curves than for those over land. The raw(XCO2) decreases by 0.3 to 0.7 ppm as the cloud distance decreases in the tropics for the QF0 data and decreases by 1.2 to 2.3 ppm for the QF1 data. Since the 3D cloud-retrieval bias is given by the difference Ave(20 km) − Ave (0 km), the 3D cloud-retrieval biases for QF0 and QF1 data are between 0.3 and 0.7 ppm and between 1.2 to 2.3 ppm, respectively, in the tropics. (The 3D cloud-retrieval bias is the raw(XCO2) bias introduced by the retrieval and does not refer to the retrieval of cloud properties). The Fig. 13 average curves indicate that the operational retrieval underestimates rawXCO2 in the tropics. The rawXCO2 land nadir and land glint averages for QF0 data decreases by 0.3 to 0.6 ppm, as cloud distance decreases, and by 1.5 to 3.0 ppm for the QF1 data.

Figure 13 The raw(XCO2) averages as a function of nearest-cloud distance. The value of −10 refers to the 10–0^∘ S latitude band.

A visual examination of the graphs (not shown) of the average dP and raw(XCO2) curves for 2015, 2017, and 2018 display many similar qualitative features to those displayed by the 2016 curves in Figs. 12 and 13. While XCO₂ has increased during 2015–2018 (by approximately 2.4 ppm yr⁻¹), the shapes of the curves are qualitatively similar from year to year.

Figure 14 displays ΔdP averages (i.e., dP (near 20 km) − dP (near 0 km) differences) as a function of latitude for 2015–2018, with −40 referring to the 40 to 30^∘ S latitude bin, etc. Instead of just using the dP values at 20 and 0 km cloud distance, averages for 18–21 and 0–3 km are calculated to bring more data into the averaging process. The ocean glint QF0 ΔdP averages are slightly positive at all latitudes. The ocean glint QF1 ΔdP averages are consistently positive at all latitudes, with values between 2 and 4 hPa for latitudes southward of 30^∘ N. There is a hemispherical asymmetry in the ΔdP land values, with negative ΔdP QF1 values near −4 hPa at 20^∘ S and values near zero in the Northern Hemisphere. The reason for this asymmetry is not known.

Figure 14 Latitudinal averages of dP (near 20 km) − dP (near 0 km) for 2015–2018 for QF0 (solid line) and QF1 (broken line) data. OG, LN, and LG refer to ocean glint, land nadir, and land glint observing modes. The value of −40 refers to the 40–30^∘ S latitude bin.

Figure 15 displays ΔrawXCO2 averages (i.e., raw(XCO2) (near 20 km) − raw(XCO2) (near 0 km) differences) as a function of latitude for 2015–2018. The ΔrawXCO2 averages are an appropriate measure of 3D cloud-retrieval biases (in ppm) that are present in raw(XCO2) OCO-2 data files. Table 4 specifies the range of these biases for the six lines in Fig. 15. The ocean glint QF1 ΔrawXCO2 biases are consistently above 1 ppm, with a latitudinal average near 1.5 ppm. This indicates that ocean glint QF1 data is underestimated by an amount roughly equal to the 1 ppm OCO-2 measurement goal. The ocean glint QF0 data has an average ΔrawXCO2 near 0.4 ppm. Land glint and land nadir QF1 average ΔrawXCO2 is positive in the Northern and Southern hemispheres, while the QF0 average ΔrawXCO2 is positive (0.25 ppm) in the Southern Hemisphere and negative (−0.25 ppm) in the Northern Hemisphere. This asymmetry is likely related to the ΔdP hemispherical asymmetry present in the Fig. 12 curves.

Figure 15 Latitudinal averages of raw(XCO₂) (near 20 km)  − raw(XCO₂) (near 0 km) for 2015–2018 for QF0 (solid line) and QF1 (broken line) data. OG, LN, and LG refer to ocean glint, land nadir, and land glint observing modes. The value of −40 refers to the 40–30^∘ S latitude bin.

Table 4 Minimum, average, and maximum ranges (in ppm) of the six curves in Fig. 17.

The 3D cloud effects are expected to asymptote to zero as the cloud distance becomes very large. The ocean glint averages are the clearest evidence of 3D cloud effects (Figs. 12a–b and 13a–b) in the tropics. With respect to latitudinal averages, the ocean glint QF0 (QF1) latitude average ΔrawXCO2 biases (Fig. 15) are near 0.4 (1.5) ppm, while land nadir and land glint biases are near 0.25 and −0.25 ppm for QF0 data in the Southern and Northern hemispheres.

7 3D bcXCO2 cloud effect bias mitigation

This section discusses how one can calculate an empirical lookup table of 3D-cloud-effect biases and use it to correct bcXCO2 for 3D cloud effects that are present in the v10 Lite files. The v10 3D-cloud-effect biases for bcXCO2 data will differ from the v11 Lite file bcXCO2 biases. The v11 data are currently in the production phase.

Figure 16 displays a graph of QF0 ocean glint bcXCO2 as a function of nearest-cloud distance D. Using the 20–0^∘ S band as an example, and a nearest-cloud distance of 3 km, the 3D-cloud-effect bias is given by a bias B(D) value, calculated by drawing a line on the y axis that specifies the asymptotic bcXCO2 value at large nearest-cloud distance and the difference bcXCO2(20 km) − bcXCO2(3 km). The B(D) values can be calculated for all nearest-cloud distances and all latitude bands. Figure 17 presents curves of these biases for the six latitude bands for QF0 and QF1 data. The Zenodo archive (https://doi.org/10.5281/zenodo.7655136; Massie, 2023) has separate ASCII files of the QF0 and QF1 curves and ASCII files of the 3D-cloud-effect biases for QF0 and QF1 data for land nadir and land glint data. The files specify a tabulation of the nearest-cloud distance versus 3D-cloud-effect bias. Positive (negative) biases indicate that the operational retrieval and post-retrieval bias processing underestimates (overestimates) bcXCO2.

Figure 16 Latitudinal variation in bcXCO2 as a function of nearest-cloud distance. The black lines illustrate how one calculates the bias B value for each cloud distance for each latitude band. Using the −20 to 0^∘ latitude band as an example, the value of B is 0.25 ppm when the nearest-cloud distance is 3 km.

Figure 17 The 3D-cloud-effect biases (ppm), calculated from Fig. 16 B values, as a function of latitude for (a) QF0 and (b) QF1 data.

There are many data points in the averages displayed in Fig. 16. For the 20–0^∘ S band example, the number of data points N(D) in the nearest-cloud distance bins of 1 and 20 km are 2.38×10⁶ and 181 909. For a bcXCO2 error of 0.39 ppm (calculated from a 50^∘ S to 50^∘ N average of Lite file XCO₂err standard errors for the Fig. 7 date), the 2σ 95 % confidence limits of the determination of the bcXCO2 averages for these bins are 0.0006 and 0.0017 ppm, assuming uncorrelated errors. The 95 % confidence limits of the bcXCO2 averages for the 40–60^∘ N band are also small (0.0017 and 0.0036 ppm). The increase in the bcXCO2 averages in the 40–60^∘ N band is inherently present in the Lite file bcXCO2 and is not due to too few data points in the nearest-cloud distance bins.

Figure 17 indicates that the 3D-cloud-effect biases are largest (and consistently positive) for the ocean glint QF1 data. QF1 3D-cloud-effect biases are generally larger than the QF0 biases and generally positive for ocean glint, land nadir, and land glint observations. For QF0 data, the biases are both positive and negative. As an example of negative biases, Fig. 17 biases are negative for the 40–60^∘ N latitude band, since bcXCO2 increases as nearest-cloud distance decreases.

To correct a Lite file bcXCO2 value of 3D-cloud-effect bias, the 3D metric data (available from https://doi.org/10.5281/zenodo.4008765; Massie et al., 2020) can be used in conjunction with the latitude-dependent 3D-cloud-effect bias data B(D) (available from https://doi.org/10.5281/zenodo.7655136; Massie, 2023). Given a nearest-cloud distance D (from the 3D metric file data or calculated from MODIS or geosynchronous satellite radiance data), the corrected bcXCO2 value is then bcXCO2 +B(D).

Latitudinal averages of 3D-cloud-effect biases, calculated by numerically weighting the biases B(D) by the number of observations N(D) in each of the nearest-cloud distance bins D for each of the latitude bands (the ratio $\sum B\left(D\right)N\left(D\right)/\sum N\left(D\right)$) are presented in Fig. 18 for ocean glint QF0 and QF1 data. Since there are more observations at small nearest-cloud distances compared to large nearest-cloud distances, the weighted biases are dominated by the left side of the curves in Fig. 16. Biases are larger over the Southern Hemisphere than over the Northern Hemisphere. This may be due to the fact that there are more TCCON observations in the Northern Hemisphere.

Figure 18 The 3D-cloud-effect average biases as a function of latitude for QF0 and QF1 data. The average biases are those based on Fig. 17 data, weighted by the number of observations in each of the nearest-cloud distance bins for observations in 2015–2018. The value of −40 refers to the 40 to 20^∘ S latitude bin, etc.

Table 5 presents the weighted biases and the biases for the 0 to 1 km nearest-cloud distance bin. For Fig. 16, the biases for the 0 to 1 km bin are the largest possible biases. The calculations that generated Table 5 were carried out with and without temporal corrections. Since XCO₂ is increasing at 2.5 ppm yr⁻¹ (and at larger values in the high northern latitudes), and since we analyze bcXCO2 data for the years between 2015 and 2018, the average year temporal value for, e.g., nearest-cloud distances of 3 and 20 km, may differ. Knowing the temporal values for each latitude band and each nearest-cloud distance bin and the temporal bcXCO2 trend (in ppm yr⁻¹; calculated from a linear fit to bcXCO2 for the 4 years), the bcXCO2 average for a specific latitude band and nearest-cloud distance bin can be temporally corrected to a common latitude-band-specific temporal value (see Appendix B for details). The C columns of Table 5 include the temporally corrected biases, and the nC columns are biases with no temporal corrections. Figures 16–18 display temporally corrected bcXCO2 calculations. A graph of bcXCO2 (not shown), comprised of bcXCO2 not temporally corrected, is very similar to Fig. 16.

Table 5 The bcXCO2 3D cloud biases (ppm) for the Northern (0–60^∘ N, NH) and Southern (60–0^∘ S, SH) hemispheres and for the 60^∘ S and 60^∘ N range (NH+SH). A positive (negative) bias indicates that the operational retrieval and post-retrieval bias processing underestimates (overestimates) bcXCO2 due to 3D cloud radiative effects. WC and WnC refer to observation-number-weighted biases with (C) and without (nC) temporal corrections. DiffC and DiffnC refer to the bcXCO2(20 km) − bcXCO2(0 km) bias differences, for nearest-cloud distances of 20 and 0 km, with and without temporal corrections. The distance of 0 km refers to the 0 to 1 km cloud distance bin. See Sect. 7 for a discussion of this table.

An inspection of Table 5 indicates that averages of weighted 3D-cloud-effect temporally corrected biases in the Northern and Southern hemispheres are 0.16 (1.31) and 0.26 (1.41) ppm, respectively, over the ocean and −0.13 (0.51) and −0.08 (0.47) ppm over land for QF0 (QF1) data. All of the QF1 biases in Table 5 are positive, while the QF0 biases are positive over the ocean and mostly negative over land.

8 Conclusions

While 1D radiative transfer theory is extensively covered in papers and textbooks, 3D radiative transfer rule-of-thumb knowledge is not well established. The calculation of idealized bar clouds (Fig. 1) provides insight as to how 1D $/$ 3D ratios vary as a function of 3D cloud metrics. Of the various possible cloud metrics to consider, nearest-cloud distance readily comes to mind since the 3D cloud effect obviously becomes small if clouds are far away from observation points. Figure 2 indicates that the 1D $/$ 3D ratio is closest to unity far away from clouds and decreases towards smaller values as cloud distance D decreases and that the O₂ A band will likely have the largest 3D cloud effect, followed by the WCO₂ and SCO₂ bands. Figure 2 also indicates that the 1D $/$ 3D ratios are appreciable in size. With 80 % of the Amazon retrievals within 4 km of clouds (Table 2), O₂ A-band 1D $/$ 3D ratios are less than 0.96.

The Fig. 4 curves from the idealized bar cloud SHDOM calculations are presented to convey insights into the factors which modulate 3D cloud effects. Figure 4 indicates the sensitivity of 1D $/$ 3D ratios as a function of vertical gas optical depth, solar zenith angle, cloud-top height (cloud vertical extent), surface reflectance, cloud LWC, and aerosol composition. The sensitivity is largest for the gas vertical optical depth. While there are important variations in the details shown in Fig. 4, all panels display a highly nonlinear decrease in 1D $/$ 3D ratios as nearest-cloud distance decreases.

The OCO-2 cloud preprocessor does a very good job of screening observations near and over clouds. The preprocessor, however, does not necessarily identify clear-sky observations that are impacted by 3D cloud effects located between low-altitude clouds. Figures 6 and 7 illustrate typical scenes over land and ocean in which successful OCO-2 retrievals are located between clouds. Figures 8, 9, and 10 illustrate that 3D cloud effects usually occur within 4 km of clouds, since 1D $/$ 3D ratios and raw(XCO2) concurrently decrease at nearest-cloud distances less than 4 km.

Cloud-brightening events are prevalent compared to cloud-shadowing events. A yearly analysis of Amazon and Pacific regions (Table 2) yields retrieval percentages associated with cloud shadowing to be less than 4 % for cloud heights less than or equal to 8 km. The unequal percentage of cloud-brightening (96 %) versus the cloud-shadowing 4 % percentage in the retrieved observations imposes an asymmetry in the imposed 1D $/$ 3D intensity ratio radiance perturbations, with fewer cloud-shadowing events compared to cloud-brightening events.

The 3D cloud effects likely are more important for ocean glint observations than for land nadir observations, since nearest-cloud distances are smaller over the ocean than land (see Fig. 11). We assert that the predominant presence of QF1 data over the Amazon, which is subject to many low-altitude clouds, is in part due to the difficulties imposed by 3D radiative transfer upon the 1D interpretation of measured Amazon OCO-2 radiances.

Figure 16 illustrates how one can calculate a 3D-cloud-effect bias lookup table for bcXCO2 data, with Fig. 17 presenting ocean glint table lookup data. The application of the lookup table data (which can be accessed from https://doi.org/10.5281/zenodo.7655136; Massie, 2023) is illustrated in Fig. 18, which indicates that the operational processing underestimates QF0 bcXCO2 over the Southern Ocean on average by 0.3 ppm, relative to observations free of 3D cloud effects (as estimated from observations near 20 km to the nearest-cloud distance). Though the QF0 hemispherical biases are between −0.31 and 0.26 ppm (Table 5), biases the in bcXCO2 on regional scales as small as a few tenths of a ppm in XCO₂ can lead to spurious values of inferred CO₂ fluxes (Chevallier et al., 2010). The operational retrieval and post-retrieval processing consistently underestimates QF1 bcXCO2 by 0.3 to 1.4 ppm relative to observations free of 3D cloud effects (Table 5).

While Massie et al. (2021) focused on comparisons of bcXCO2 and TCCON, the analysis of 275 times more bcXCO2 data between 2015 and 2018 (without reference to TCCON data) enabled calculations of 3D-cloud-effect biases as a function of latitude. The biases are larger in the Southern Hemisphere. This is possibly due to the fact that there are fewer TCCON observations in the Southern Hemisphere. The magnitude of the 3D-cloud-effect biases discussed in Massie et al. (2021) and this paper are similar in size, with QF1 biases generally larger than QF0 biases. Since the post-retrieval bias-correction process exclusively uses QF0 data, and dP and dPfrac variables, which are correlated with nearest-cloud distance (Massie et al., 2021), it is expected that the QF0 biases will be small. The post-retrieval bias-correction process indirectly accounts for 3D effects, but Fig. 16 and the Table 5 entries indicate that 3D-cloud-effect biases remain in the Lite file data.

Future work includes the development of a quick parameterization of 1D $/$ 3D ratios as a function of aerosol and cloud optical depth, given an arbitrary geospatial distribution of clouds. This work will examine a wider range of parameters, such as cloud height, aerosol height, and aerosol composition, in addition to an examination of scenes not covered in this paper, such as brighter surfaces. Aerosol characteristics are of interest, since Bell et al. (2023) used OCO-3 data to demonstrate that swath bias is related to aerosol characteristics and viewing geometry. While our paper focuses on OCO-2, the results are applicable to OCO-3 on the International Space Station. Geosynchronous satellite radiance and cloud data can be used to derive nearest-cloud distance for studies related to OCO-3 and 3D cloud effects.

Appendix A

The following paragraphs discuss the specifics of the algorithm used to calculate the percentage of cloud-brightening and cloud-shadowing events.

Using the Amazon observations on 22 June 2015 (150622) as an example, the L2ABP preprocessor data file specifies the cloud flag (0= clear, 1= cloudy, and 2= undetermined), and O₂ A-band continuum radiances for each OCO-2 measurement are specified by the OCO-2 Level 1B data files. Average clear and cloudy continuum radiances (Clear_bin and Cloudy_bin) are determined in 0.5 latitude bin steps for the 10^∘ S to 3^∘ N latitude range. Some of the latitude bins will be fully cloudy. These bins are not used in the calculation of a Clear_ave radiance average. Some of the latitude bins have too few clouds and are excluded, since it is of interest to determine clear radiances in the vicinity of clouds. A Clear_ave radiance average is calculated from the Clear_bin averages when the percentage of clear flags for a latitude bin is greater than 50 %. A similar calculation is done for the Cloudy_ave average. Guided by SHDOM calculations, for bar clouds similar to Fig. 1 with cloud tops at 2, 4, 6, and 8 km, the 3D radiances are analyzed to determine an average observation to cloud radiance ratio Ratio_obs,cloud, with Ratio_obs,cloud determined from 3D radiance (distance D from the cloud) $/$ SHDOM 3D radiance (position located over the cloud) ratios. These calculations yield a SHDOM threshold Ratio_threshold near 0.30. Since there is a range of Cloudy_bin values for the various latitude bins on an individual day, Ratio_threshold is conservatively increased arbitrarily to 0.60 (which will overestimate the cloud shadow percentage geom values in Table 2).

For a specific observation point, L2ABP locations surrounding the observation point are examined. A 10 km × 10 km box surrounds the observation point, and the algorithm loops over L2ABP data file longitude I and latitude J indices. Ratio${}_{\mathrm{obs};I,J\mathrm{point}}$ values, equal to the O₂ A-band L2ABP radiance_obs point, divided by the O₂ A-band L2ABP radiance_{I, J point}, are calculated. The distances Distance${}_{\mathrm{obs};I,J\mathrm{point}}$ from the observation point to the I,J positions are calculated. The angles Angles${}_{\mathrm{obs};I,J\mathrm{point}}$ between the fixed solar vector (observation point to the Sun's location) and geometry vectors (from the observation point to the L2ABP I,J positions are also calculated. For a given cloud height (e.g., 4 km), the solar zenith angle and cloud height determines the X_shadow spatial length (in km) that the shadow corresponds to.

In general, if (a) the positions of the Sun, L2ABP file I,J position, and observation point are in a line with the I,J point in the middle of the line, if (b) Distance${}_{\mathrm{obs};I,J\mathrm{point}}$ is less than X_shadow, if (c) Ratio${}_{\mathrm{obs};I,J\mathrm{point}}$ is less than Ratio_threshold, if (d) the I,J point has a L2ABP cloud flag equal to 1 (cloudy case), and if (e) Angles${}_{\mathrm{obs};I,J\mathrm{point}}$ is less than a threshold angle difference (e.g., 30^∘), then the observation point is associated with a shadow. The code-wise loops check to see if a shadow is associated with each observation point and then calculates the percentage of retrievals that are associated with shadows for each cloud height.

Appendix B

Temporal corrections of bcXCO2 for the years 2015–2018 are incorporated into Table 5 and Figs. 16–18.

For each latitude band, the bcXCO2 trend T_lat (ppm yr⁻¹) is calculated utilizing QF0 bcXCCO2 data from all nearest-cloud distance D (km) values. The trends are of the order of 2.5 ppm yr⁻¹, though higher (3.0 ppm yr⁻¹) at higher northern latitudes. The trends T_lat from the QF0 bcXCO2 calculation are applied to both QF0 and QF1 bcXCO2.

For a specific latitude band, the observation-number-weighted temporal averages Y_lat,D are calculated for each nearest-cloud distance D bin, with Y_lat,D calculated using the day of the year information for each individual data point. The Y_lat,D are used to calculate an average time Y_lat ave for each latitude band.

To produce Figs. 16–18, each bcXCO2 average (before temporal correction) for each D bin was modified by adding the correction term $T_{\mathrm{lat}}(Y_{\mathrm{lat},D}-Y_{\mathrm{lat}\mathrm{ave}}$) for each bin D. The − sign is applied, since T_lat is positive, with positive- and negative-signed temporal corrections for data in 2015 and 2018.