Overview of OCO-2 data
The OCO-2 instrument measures radiances in the molecular oxygen (O2)
A-band (0.765 µm), the “weak” CO2 band at 1.61 µm, and the
“strong” CO2 band at 2.06 µm. The OCO-2 instrument is an imaging
spectrometer that collects eight samples, or “spatial footprints” across a
narrow (0.8∘) swath track. Observations are taken in three different
modes: (1) “Nadir”, where the spacecraft points the instrument's aperture
at the ground directly downward along the orbit track; (2) “Glint”, where
the spacecraft points instrument's aperture near the “glint spot” where
sunlight is specularly reflected by the surface, near the specular
reflection point for sunlight; and (3) “Target”, where the spacecraft points
the instrument aperture at a stationary surface target, such as a validation
site or city.
Nadir observations usually return useful measurements only over land. Glint
observations return useful data over both land and ocean. Here, we
discriminate land-glint and ocean-glint observations because they have
different error statistics. We do not evaluate Target data in this analysis
due to spurious statistics that are observed with the Target data.
As discussed in Boesch et al. (2006), Connor et al. (2008), and O'Dell et al. (2012 and
references therein), total column estimates of XCO2, are derived from OCO-2
observed radiances using a Bayesian optimal estimation approach that depends
on CO2, all the geophysical parameters or interferences that affect the
radiances in these bands, and a priori statistics of the atmosphere and these
interferences.
We use version 7 of the OCO-2 data, the first OCO-2 product distributed for
general users. These data, like those described for GOSAT data in Wunch et al. (2011), are bias-corrected using a fit to retrieved aerosol optical depth
and the retrieved vertical CO2 gradient based on comparisons between
OCO-2 and total column measurements from the ground-based Total Carbon
Column Observing Network (TCCON) and regions where XCO2 variations are
expected to be small relative to the measurement uncertainties (Wunch et al., 2011,
2017). We find that use of the bias-corrected data greatly improves
comparisons between expected variability within a neighborhood and the
actual observed variability (see Appendix). Data quality is evaluated using
a variety of metrics that depend on the estimated cloud, aerosol, and
surface properties, convergence, and known statistics of the retrieved
XCO2 values (e.g., Mandrake et al., 2013). Data quality flags are given as
“warn levels” with values ranging from 0 (best) to 19 (worst). Data with
lower warn levels are more likely to represent the statistics of the
observed CO2 whereas data with higher warn levels are likely too
strongly affected by interfering effects. The warn levels are primarily
evaluated empirically; for these reasons we conservatively use only data
with warn levels of 10 or smaller to ensure that the corresponding errors
are likely well characterized:
https://docserver.gesdisc.eosdis.nasa.gov/public/project/OCO/OCO2_DUG.V7.pdf.
We find empirically that use of data with warn levels less than 10 improves
the comparison between the calculated uncertainties and observed variance as
discussed in the Appendix.
Evaluation of uncertainties
We evaluate the uncertainties of the XCO2 observations by examining the
variations of XCO2 within small neighborhoods of approximately 10.5 by 100 km in size. Within a neighborhood there are about 190 observations
that are taken consecutively. After warn level filtering, this “small
neighborhood” test set is composed of approximately 1.5 million land-nadir
soundings, 1.0 million land-glint soundings, and 5.0 million ocean-glint
soundings. We only select neighborhoods that contains at least 50 soundings
that pass these criteria. There are approximately 39 000 small neighborhoods
in total across the three modes. stretching from approximately 30∘ S to 30∘ N.
The strict filtering used in this analysis (warn levels < 10), and
the need for at least 50 measurements per bin limits this analysis to
latitudes between 30∘ S and 30∘ N, primarily over drier, subtropical regions
over land but no obvious preferential distribution over the ocean (not
shown).
As discussed in O'Dell et al. (2012), a CO2 profile is simultaneously
estimated with all other geophysical parameters that affect the observed
radiance such as aerosols, albedo, and surface pressure. The
“column-averaged dry air mole fraction” of CO2 or XCO2 is then
calculated by applying the column operator (e.g., Connor et al., 2008; Worden
et al., 2015) to the estimated CO2 profile. As discussed in Rodgers (2000),
Worden et al. (2004), Connor (2008), and Bowman et al. (2006), when this
nonlinear retrieval converges to a solution, the estimated XCO2 can be
written as
X^=Xa+hTAxxx-xa+hTAxyy-ya+hTGn+hTG∑iKiδi,
where X^ is the estimated total column for CO2,
X^a is the a priori value used to help regularize the retrieval, and
the vector x is the “true” CO2 profile in units of volume mixing ratio
(VMR), discretized onto the forward model atmospheric pressure grid used to
calculate the transfer of radiation needed to model the observed radiance.
The xa is the a priori for the CO2 profile. The vector “y” contains all
the other parameters that are simultaneously estimated with x such as aerosol
properties, surface albedo, surface pressure. The vector “n” is the actual
noise in the radiance. The quantities x,y, and n are not known exactly, only their
statistical properties can be estimated. The vector “h” is the column
operator which maps a profile on the pressure grid defined by “x” into a
dry air total column. The averaging kernel matrix A describes the
sensitivity of the estimate to each retrieved parameter (Rodgers, 2000). In
Eq. (1) the averaging kernel matrix is composed of two parts, Axx
and Axy, described by
A=AxxAxyAyxAyy.
For example Axx describes the sensitivity (or ∂x^∂x) of the estimated CO2 on each level,
x, to its true value, whereas Axy describes the sensitivity of the
estimated CO2 on each level, x, to all other simultaneously estimated
parameters, e.g., aerosols. The matrix G is the gain matrix,
which is the derivative of the estimated CO2 on each level, x, to the
observed radiance, L (or G=∂x^∂L). The matrix K is the Jacobian, or
sensitivity of the observed radiance to a parameter (e.g., K=∂L∂x). The last term, δ,
describes the error in all parameters that are not estimated for this
retrieval, but are assumed constant, such as absorption coefficients or
instrument functions (e.g., Connor et al., 2008). The mean CO2 column is
written as
X^mean=Xa+hT1N∑j=1NAjxj-xa+1NhT∑j=1NAxyjyj-ya+1N∑j=1NhTGjnj+∑i,jKi,jδi,j,
where N is the number of observations within the small neighborhood and for
simplicity we assume the column operator h is constant across the domain.
For the next three sections, we test the following hypotheses regarding the
observed distributions within the collection of “small neighborhoods” and
their calculated uncertainties:
H1: observed variability in small neighborhood is due to natural XCO2
variability.
H2: observed variability in small neighborhood is due to measurement
noise.
H3: observed variability is correlated.
H4: observed variability within a small neighborhood is described by a
slowly varying bias that is not explained by natural XCO2 variability.
We look at the variability with respect to the neighborhood mean in two
ways: (1) for small neighborhoods, the predicted errors for a neighborhood
are averaged from the observations that comprise that neighborhood, making
the statistics technically a sum of Gaussians, and (2) the variability with
respect to the neighborhood mean, sorted by predicted error and aggregated
over many neighborhoods – the statistics in this case should be Gaussian;
however the locality of the analysis is somewhat reduced.
To evaluate whether measurement noise in the radiances is the primary factor
driving variability within a small neighborhood we first assume that the
terms Axy(yj-ya) and systematic errors Ki,jδi,j do not vary. Based upon these approximations, the difference
between an observation and its mean is given by
X^obs-X^mean=δobs=δXCO2+Gobsnobs-1N∑jNGjnj,
where δXCO2=hTA(xobs-xmean) and
is the difference between the individual “true” XCO2 and the mean of the
“true” XCO2 values within the neighborhood. Assuming that the measurement
noise is spatially uncorrelated, the variance within the small neighborhood
(e.g., Bowman et al., 2006) is
VarX^obs-X^mean=σobs2=σXCO22+σnoise2+1N2∑j=1Nσj2-2Nσk2,
where σnoise2=GKSkGKT
is the measurement uncertainty due to noise. The σXCO2 is the
variability of the true XCO2 within the small neighborhood. The Sk is
the spectral instrumental noise covariance and is calculated during
calibration of the instrument. The individual σnoise values are
provided for each measurement in the OCO-2 product files. For large N,
Eq. (5) is approximately equal to
σXCO22+σnoise2.
H1: observed variability is due to natural XCO<inline-formula><mml:math id="M100" display="inline"><mml:msub><mml:mi/><mml:mn mathvariant="normal">2</mml:mn></mml:msub></mml:math></inline-formula> variability
In order to test whether natural variability, or σXCO22, affects
the observed variance of XCO2 within each neighborhood we examine
XCO2 estimates from the NASA GMAO high-resolution free-running GEOS-5
CO2 simulation available at
https://gmao.gsfc.nasa.gov/global_mesoscale/7km-G5NR/data_access. We use model fields that
correspond to each measurement within each neighborhood. There is a spatial
mis-match because the model fields are at 7 km × 7 km resolution whereas the
OCO-2 data are taken every 3 km; however, we discount the role of spatial
resolution because our results do not fundamentally change when smoothing
the data from 7 km × 7 km (lat/long) to 14 km × 7 km (lat/long). The dates
of the model run (2006) also do not match the dates of the OCO-2 data
(2014–2015). However, since we are looking to quantify the approximate range
of natural variability we would not expect inter-annual differences in
XCO2 from winds or fluxes to substantively alter our conclusions when
looking at XCO2 over a large swath of the Earth; as discussed next,
comparisons with other data and model are consistent with this conclusion.
From the GMAO model, we find that natural variability typically accounts for
about 0.08 ppm of the variability within a ∼ 100 km
neighborhood over land and about 0.06 ppm within a typical ∼ 100 km neighborhood over the ocean (see the Appendix). We subsequently
assume that natural variability has negligible impact on our conclusions as
we find it is on average much smaller than the observed variability.
The calculated, observed, and modeled uncertainties for land-nadir observations. The black circles are the observed
distributions and red circles are modeled distributions. (a) Comparison between expected and actual error when binning
all the data by their calculated uncertainty; the solid black line is the one-to-one line. (b) Comparison between calculated
and random error for each neighborhood (black) versus model (red) when using observed random error from Fig. 1a. (c) Same as
Fig. 1b but now adding a correlation between adjacent data points. (d) Same as Fig. 1b but now accounting for distribution of
observed gradients across the neighborhoods used for analysis.
![]()
Observed and modeled distributions for land-glint data.
![]()
H2: observed variability in small neighborhood is due to measurement noise
We next compare observed variability across all the neighborhoods to the
calculated uncertainties using two approaches. In the first approach we
gather all observations that have approximately the same calculated
measurement uncertainty due to noise, σnoise2, (to within
0.01 ppm) as provided in the OCO-2 product files and compare to the actual
variability of these observations. The steps for this comparison are as follows:
Calculate the δobs or difference between an observation and its
mean within a small neighborhood as shown in Eq. (4).
Collect all of the δobs values from all neighborhoods used in
this analysis whose corresponding σnoise values (measurement
uncertainty) are the same to within 0.01 ppm and bin them as a function of
σnoise. There are typically about 1000 observations per
σnoise bin.
Compare the standard deviation of the collection of δobsvalues
within each bin to the expected standard deviation due to noise or,
σnoise. Based on Eq. (5) we should expect to get a linear,
one-to-one relationship if the dominant parameter affecting the variability
within a small neighborhood is noise.
The results of these comparisons for land-nadir, land-glint, and ocean-glint
observations are shown in the upper left panels of Figs. 1, 2, and 3
respectively. These results show the calculated measurement uncertainty due
to noise has skill – i.e., there is a linear relationship between calculated
and actual error. However, over land the observed random variability is
approximately 0.4 ppm larger than the variability expected from noise. We
discount synoptic variations in XCO2 as discussed in the previous section.
Other sources of variability could be due to the strong nonlinearities in
the retrieval (e.g., Kulawik et al., 2008) or local variability between the true
and a priori in the interferences, or non-retrieved parameters. Over the ocean there
appears to be an even stronger one-to-one relationship between the
calculated uncertainty and the actual uncertainty except for calculated
uncertainties less than approximately 0.25 ppm which show a strong inverse
relationship. We find that these observations (not shown) tend to occur in
the tropics in cloudy regions and that the observations tend to have very
high signal-to-noise ratios.
We next test whether the calculated measurement noise is a useful value for
predicting the expected distribution of observations within a neighborhood.
Because each δobs is drawn from a distribution with a different
variance, we treat the sample of each set of observations, δ1,δ2,…δN, as being drawn from an
uncorrelated distribution with individual variances σobs2.
Accordingly, the variance of this sample should be the average of the
individual variances σobs2:
Var|[X^obs-X^mean]||=δ1,δ2,…δN=1N∑jNσj2.
The top right panel of Fig. 1 shows a comparison of the observed variance
of the XCO2 distributions (using the left side of Eq. 6) within each
neighborhood (black circles) versus the expected variance in XCO2 using the
right side of Eq. (6). Each black symbol represents a single
neighborhood. In contrast to the top left panel of Fig. 1, this result
suggests that the measurement error has no skill in predicting the observed
variance of XCO2 within a neighborhood.
We next test whether the observed variance, versus that due to measurement
noise or sampling, explains the upper right panel of Figs. 1, 2, and 3. To
perform this test, we perform the following steps:
Within each neighborhood, replace the calculated measurement error with the
“actual” measurement error as shown by the solid red line in the upper
left panel of Figs. 1, 2, and 3, for each observation.
Create a simulated distribution of observations based on this new
uncertainty.
Randomly sample (or take) one of these observations → label this the
“modeled” observation.
Repeat steps 1–3 for all observations in the neighborhood.
Calculate the variance of this “modeled” set of observations for each
neighborhood.
Observed and modeled distributions for sea-glint data.
![]()
Distribution of XCO2 values between time steps for the set of
observations from each “small neighborhood” used in this analysis.
![]()
The red dots in Figs. 1b, 2b, and 3b show the modeled distributions using
the steps discussed above. The modeled distribution is more consistent with
the mean of the observed distribution relative to the one-to-one line.
However, it is clear from this simulation that errors due to random noise
and sampling do not explain the observed variance for each neighborhood
although the distribution of variances for the ocean show much better
agreement relative to the land distributions.
H3: uncertainties are correlated
We next test whether observed correlations in the data could explain the
distributions of the data within a neighborhood. Figure 4 shows the joint
distribution of the XCO2 anomaly and a 0.3 s lagged anomaly in a
neighborhood (consistent with observation sampling). If the data were
uncorrelated then the joint distribution should be circular; the asymmetric
distribution therefore implies that the errors, as empirically described by
the differences, are correlated. Figure 5a and b show that autocorrelation
is observed both in time for measurements made of the order of 1 s of
each other, and with respect to the spatially adjacent “footprints”, the eight
simultaneous measurements made by the OCO-2 instrument at each time. The
range of correlations for the different observation types, land nadir, land
glint, and ocean glint are 0.45, 0.43, and 0.28 as a function of footprint
and 0.31, 0.34, and 0.24 as a function of time.
(a) Correlation of differences across pixels between observed minus
mean within a neighborhood. (b) Correlation between observations for a single pixel.
![]()
In order to test whether these observed correlations could explain the
distributions shown in Figs. 1, 2, and 3, we conservatively use a
correlation coefficient of 0.7 for all observations (an extreme case). We
then use the following procedure, building on the steps described in the
previous section.
Within each neighborhood replace the calculated measurement error with the
“actual” measurement error as shown in the upper left panels of Figs. 1,
2, and 3 for an observation.
Starting with the first observation (in time) within a neighborhood for
Footprint #1, sample a value for the observation from the distribution of
“actual” measurement errors. Label this the “modeled” observation.
For all subsequent observations in time for Footprint #1, sample each
“modeled” observation from a distribution that is correlated with the
modeled observation at the previous time step and has a variance
corresponding to the “actual” measurement error.
For observations in Footprints #2–8, sample each modeled observation
from a distribution correlated with the modeled observation at the same time
step in the previous (adjacent) footprint, again with a variance
corresponding to the “actual” error.
Calculate variance of this “modeled” set of observations, for each
neighborhood.
As can be seen in the lower left panels of Figs. 1, 2, and 3, adding
correlations to the data makes the comparison worse because the modeled
distributions become much narrower relative to the modeled distributions in
the upper right panels of these figures. Our conservative choice of a 0.7
correlation between observations at adjacent times and footprints
illustrates this effect clearly. We therefore conclude that while
correlations are empirically observed in the data, they cannot completely
explain the observed distributions within the small neighborhoods.
H4: observed variability within a small neighborhood is described by a
slowly varying bias that is not explained by natural XCO<inline-formula><mml:math id="M132" display="inline"><mml:msub><mml:mi/><mml:mn mathvariant="normal">2</mml:mn></mml:msub></mml:math></inline-formula> variability
We next examine whether “non-random” uncertainties could explain the
observed distributions in the upper right panels of Figs. 1, 2, and 3. For
example, as shown in Eq. (1), the jointly retrieved parameters (y-ya)
might remain constant across a neighborhood but the averaging kernel
associated with this term, which is given by
Axy=∂x∂L∂L∂y=GKy, can vary
across a neighborhood as the pointing angle varies. The effect of
non-retrieved parameters such as instrument effects or spectroscopy on the
estimate can vary for the same reason.
The difference between XCO2 and the mean value (or delta
XCO2) for one of the small neighborhoods (or areas) used in this
analysis. The left panel shows the differences for each footprint (FP),
representative of one of the OCO-2 observations. The right panel shows the
observed distribution (actual) and one calculated if the distributions were
representative of the calculated random error.
![]()
Figure 6 shows the variation of XCO2 across one of the ocean
neighborhoods for all eight OCO-2 footprints (denoted by “FP”). The right
panel shows the observed distribution in black relative to the mean
XCO2 of the neighborhood. For reference, the red dashed line in the
right panel indicates the expected distribution if only the calculated
random noise explained the variability. The slope shown in Fig. 6
represents an extreme case but demonstrates that observations can pass the
set of quality flags but still show this unlikely behavior over the ocean.
Figure 7a shows the distribution of all slopes across all land-nadir
neighborhoods used in this study and different fits (Gaussian, Lorentz,
Laplace) to the distribution. The Laplace distribution provides the best
overall fit so we use its functional form as a simple, convenient
description of the shape of the sharply peaked slope distribution. More
complex models such as Gaussian mixtures might also describe the shape of
this distribution of slopes as drawn from several distinct “populations”
of neighborhoods, but we leave such an analysis to future work. For
comparison, Fig. 7b shows the expected distribution using the GMAO model
XCO2 fields described earlier. As with the OCO-2 data shown in Fig. 7a, the histogram in Fig. 7b describes the distribution of XCO2
gradients across 100 km neighborhoods spatially corresponding to the OCO-2
data. The expected distribution of natural variability of XCO2 across
the 100 km neighborhoods is much smaller than observed (as with the
conclusions about natural variability discussed for H1); we therefore do not
expect that the natural carbon cycle can explain these observed variations.
(a) The distributions of slopes of the observed XCO2 gradients
across all the small neighborhoods corresponding to land-nadir observations.
(b) The expected distribution of slopes in XCO2 based on the
GMAO high-resolution model.
![]()
For land-nadir, land-glint, and ocean-glint data the variance of the slopes
is given by 1.28 ppm 100 km-1, 1.12 ppm 100 km-1, and
0.48/100 km
respectively. As shown in the Appendix, these values are much larger than
the gradients expected from natural variability, such as the latitudinal
gradient in XCO2. Consequently, we expect these gradients to be related
to interferences in the XCO2 data.
To test whether these slowly varying changes explain the distribution of
XCO2 within small neighborhoods we follow the same steps described in
Sect. 3.2 and 3.3 but now add another:
Within each neighborhood replace the calculated measurement error with the
“actual” measurement error as shown in the upper left panels of Figs. 1,
2, and 3 for an observation.
Starting with the first observation (in time) within a neighborhood for
Footprint #1, sample a value for the observation from the distribution of
“actual” measurement errors. Label this the “modeled” observation.
For all subsequent observations in time for Footprint #1, sample each
“modeled” observation from a distribution that is correlated with the
modeled observation at the previous time step and has a variance
corresponding to the “actual” measurement error.
For observations in Footprints #2–8, sample each modeled observation
from a distribution correlated with the modeled observation at the same time
step in the previous (adjacent) footprint, again with a variance
corresponding to the “actual” error.
Adjust each modeled observation with a linear function where the slope of
the linear function is randomly chosen from the fitted Laplace distribution
to the slopes (e.g., the Laplace function shown in Fig. 7).
Calculate variance of this “modeled” set of observations, for each
neighborhood.
Figures 1, 2, and 3 (lower right panels) show the best overall agreement
between modeled distributions of XCO2 relative to the mean and the
expected distributions based on observations, demonstrating that a slowly
varying bias is needed to best explain the observed distributions within a
grid of approximately 100 km × 10 km.
Each typical observation has a random error related to noise and a
systematic error that is in principle bounded by the calculated interference
error (e.g., Boxe et al., 2010) and is approximately 0.2 ppm. Within a typical
grid box an OCO-2 observed measurement over land is within 1.28/2, or
∼ 0.65 ppm of the mean XCO2 value. For these reasons, we
expect that a typical observation over land has a systematic error of at
least 0.65 ppm, about 2 to 3 times larger than the calculated interference
error.
In contrast, the observed distributions of slopes for the ocean data is 0.48 ppm 100 km-1, or a mean error of 0.24 ppm. This error is 70 %
larger than the mean calculated interference error of 0.14 ppm. Because the
distribution of ocean data within “bins” (Fig. 3, upper left panel) is
also well described by the calculated random error, we conclude that the
ocean-glint data are reasonably well characterized by their calculated
uncertainties for this size of a grid box, except for calculated noise (or
precision) uncertainties that are less than ∼ 0.25 ppm.
We find no relationship between the distribution of slopes for a
neighborhood and the corresponding mean of the calculated interference error,
suggesting that the calculated interference error does not explain the
observed slope within a neighborhood, in contrast to the measurement error.
However, there is a correlation between the slope and the estimated
magnitude of interferences, such as aerosol optical depth, surface albedo,
and surface pressure. For example, the correlation between the slopes of
land-glint data with the mean uncertainty in the interferences is 0.06
whereas the correlation between the observed slopes in XCO2 and
similarly calculated observed slopes in aerosol optical depth is 0.37. This
correlation suggests that the observed slow variations in XCO2 across a
neighborhood could be related to how interferences affect the XCO2
estimate as OCO-2 takes observations across a neighborhood.