Other Test Method 33A (OTM 33A) is a near-source flux measurement method
developed by the Environmental Protection Agency (EPA) primarily used to
locate and estimate emission fluxes of methane from oil and gas (O&G)
production facilities without requiring site access. A recent national
estimate of methane emissions from O&G production included a large number of
flux measurements of upstream O&G facilities made using OTM 33A and
concluded the EPA National Emission Inventory underestimates this sector by a
factor of ∼2.1. The study presented here
investigates the accuracy of OTM 33A through a series of test releases
performed at the Methane Emissions Technology Evaluation Center (METEC), a
facility designed to allow quantified amounts of natural gas to be released
from decommissioned O&G equipment to simulate emissions from real facilities
(Fig. ). This study includes test releases from single and
multiple points, from equipment locations at different heights, and spanned
methane release rates ranging from 0.16 to 2.15 kg h-1. Approximately
95 % of individual measurements (N=45) fell within ±70 % of
the known release rate. A simple linear regression of OTM 33A versus known
release rates at the METEC site gives an average slope of 0.96 with 95 %
CI (0.66,1.28), suggesting that an ensemble of OTM 33A measurements may
have a small but statistically insignificant low bias.
Introduction
Methane is a potent greenhouse gas, and emissions from the oil
and gas (O&G) sector are thought to account for roughly 30 % of total
methane emissions in the United States . “Upstream” O&G
activities (extraction, production, etc.) are thought to contribute the bulk
of emissions within the O&G sector . However, attempts to quantify O&G
methane emissions are hindered by inaccurate emission inventories, a lack of
measurements, and variability between basins
. For example, basin-wide aircraft measurements of methane
emissions from different O&G basins find emissions are generally higher than
official inventories published by the U.S. Environmental Protection Agency
(EPA; e.g.,), but the scale of aircraft
measurements give little insight into the exact source of emissions on the
ground. Site- and component-level measurements are therefore necessary for
improving emission estimates of the O&G production sector
. Existing near-source studies of O&G basins suggest
the majority of large, uncontrolled emissions are the result of faulty
equipment that may not be noticed for some time
, emphasizing the need for permanent or
semi-permanent monitoring technologies instead of infrequent manual
inspections . However, more permanent
approaches are still under development and must be approved as equivalent
monitoring technologies before they can replace existing EPA-approved Leak
Detection and Repair (LDAR) methods like optical gas imaging (OGI). Annual or
semi-annual LDAR programs already in place rarely quantify total emissions
from a site, and the efficacy of these programs depends on many factors
including employee experience, leak size, and meteorological variables like
wind speed and temperature . This
makes LDAR programs an important tool for finding leaks and reducing
emissions, but they often do not explicitly quantify or provide data of the
actual emission rate from production sites, and this limits usefulness for
improving emission inventories. In the absence of OGI-equivalent continuous
monitoring approaches, both basin- and site-level emission estimates have
been gathered using a number of different techniques, all with strengths and
weaknesses.
METEC facility with nine of the 11 release points circled. Release
points include (clockwise from top of tank) tank candy cane, tank thief
hatch, tank front flange, wellhead Kimray packing, wellhead hand valve
packing, separator burner fuel supply, separator Kimray vent, separator pressure relief valve (PRV),
and separator house PRV. Not pictured: wellhead lubricator flange and
wellhead pressure gauge. The University of Wyoming (UW) mobile laboratory is
in the background.
One approach is to measure emissions at an O&G production facility. On-site
measurement teams typically detect emissions from malfunctioning components
via OGI, which can be quantified using high-volume samplers. The drawbacks of
on-site measurements include difficulty measuring emission sources rich in
volatile organic compounds (VOCs; ), the inability
to reach all emission sources (such as the tops of free-standing tanks),
difficulty measuring intermittent sources, and the time required for each
inspection . Site access
requirements also introduce the possibility of changes in operation when
measurement teams are on-site .
The tracer flux ratio (TFR) technique estimates methane emissions by
multiplying the observed concentration ratio of methane to a tracer by the
known emission rate of the tracer. TFR has been used in both ground-based
and airborne applications
, though only ground-based approaches have been used for
O&G facilities. TFR can quantify all emissions at an O&G facility, and can
often differentiate between emissions from relatively close facilities
without the need for site access, though access can improve flux estimates
. TFR does not require an atmospheric transport model
and is therefore insensitive to uncertainties in atmospheric stability and
turbulence. Limitations of TFR include the reliance on downwind roadways of
sufficient distance (∼0.5–2 km) and reliable wind direction
. Drawbacks of using TFR to estimate
methane emissions include the amount of time required to estimate emissions
from one site (2.5–2.8 sites d-1; ), and
the need to transport and release compressed tracer gases (some of which are
flammable such as acetylene) near O&G facilities.
As mentioned previously, airborne mass flux measurements have been used to
estimate methane emissions from multiple O&G basins
eg.,.
Meteorological requirements (like a fully developed planetary boundary layer
and consistent wind direction) make these measurements difficult, especially
for expansive O&G basins such as the Permian basin in Texas and New Mexico
. Emission estimates of individual production sites via
aircraft measurements are also possible, but measurement sites typically need
to have relatively large emissions and are limited by aircraft range, turning
radius, and favorable meteorological conditions . Additionally, airborne sampling
must occur during the day to meet meteorological requirements, and diurnal
variability of emissions associated with on-site maintenance could impact
aircraft-based emission estimates in some basins
.
A final type of measurement technique used to estimate emissions from O&G
production facilities – and the focus of this study – are downwind
measurements that estimate emissions by using the methane mixing ratio and
wind measurements to derive the source flux. Downwind emission flux estimates
are made using parameters measured in the field combined with additional
parameters found with Gaussian or atmospheric dispersion models
. Downwind measurements do not require site
access, but may not be able to identify or capture all sources on-site,
especially buoyant ones. Similar to TFR, these techniques require downwind
roadways (50–200 m away) and consistent wind direction. Operator-approved
site access can improve OTM 33A measurement success in regions with limited
downwind roadway infrastructure or complex topography. Though sampling time
can be considerably faster than TFR or on-site techniques, it is hard to
measure enough sites to get a representative sample (and therefore a flux) of
an entire O&G basin . As a whole, all of the emission
measurement techniques mentioned here are only representative of a timescale
between seconds and hours, and therefore it is challenging to use them to
capture emissions sources with large temporal variability .
This study focuses on a ground-based mobile emissions measurement approach,
Other Test Method 33A (OTM 33A). OTM 33A is among the most common downwind
methods, along with TFR, used to measure methane and VOC fluxes from O&G
sources . A recent
study by compared on-site, OTM 33A, and TFR measurement
techniques in the Fayetteville Shale. The results of the
study suggest OTM 33A only captured ∼40 %–60 % of emissions
measured or estimated by on-site teams in the Fayetteville Shale when the
dominant emission source was an on-site direct measurement rather than a
simulated emission source. OTM 33A had a larger low bias when manual or
automated unloadings were measured. Manual or automated unloadings occur when
the well pressure is not great enough to move liquids from the geologic
formation, preventing gas flow to the pressurized sales line. To maximize the
pressure differential, the well is vented directly to the atmosphere in order
to remove accumulated liquids. This process can be performed manually or
automatically, and may use a plunger to assist with liquid removal. This
creates an emissions plume with high vertical velocity. It is likely the
majority of this plume would pass over the mobile laboratory unless perfect
conditions and road access generate a downwind measurement site 200 m or
less from the source. The results of the Bell et al. study add uncertainty to
recent national methane emission estimates, which relied heavily on OTM 33A
measurements in five O&G basins . However, the
study also found that basin-wide emission estimates
based on OTM 33A facility measurements agreed with airborne basin-wide flux
estimates to within measurement uncertainty. Additionally, no significant
low bias (>10 %) was detected in numerous (>100) OTM 33A test
releases, conducted by multiple groups
. These test releases were all
single point-source releases conducted in open terrain without obstacles,
which may not be a reliable comparison to the types of emission sources
experienced in O&G fields. The discrepancy between results of
study and previous test releases, along with the
potential significant impact on national emission estimates, motivated the
suite of more realistic test releases described here.
Materials and methodsMobile laboratory
The University of Wyoming (UW) mobile laboratory is a customized Freightliner
Sprinter van. The front of the van is equipped with a horizontal mast that
projects instrumentation and the inlet at a fixed height of 4 m above the
ground slightly beyond the vehicle's front bumper. Meteorological instruments
on the mast include a 3-D sonic anemometer and an all-in-one compact weather
station. The mast also includes a camera, an AirMar differential GPS, and a
Teflon inlet (1/4 in. o.d.) for gas-phase species. Ambient air is pulled
through the Teflon inlet at a rate of 6.5 L min-1. For the test
releases described here, the laboratory was instrumented with a G2204 Picarro
Cavity ring-down spectrometer (CRDS) which has been modified to measure water
vapor and dry methane concentrations at a frequency of 2 Hz. The Picarro has
an additional meter of 1/8 in. OD Teflon tubing that branches from the
main inlet line, resulting in a total sample transit time through the inlet
to the instrument of 1 s. This lag is accounted for during data processing.
Additionally, the van contains a battery bank which allows the
instrumentation and data acquisitions system to be used while the vehicle
engine is turned off.
Instrument calibration
The Picarro response was tested using two methane–zero air mixtures
certified by the National Institute of Standards and Technology (NIST;
2.538±0.05 ppm, 101±5 ppm), and ultra-high-purity zero air (UHPA)
at intervals throughout the campaign to confirm stability and accuracy. The
instrument was always within ±0.01 ppm of the lower NIST standard, ±1 ppm of the higher standard, and ±0.003 ppm of zero when tested with
UHPA. The 5 s instrument precision is ±0.002 ppm. Due to the observed
instrument stability and accuracy, no calibration adjustments were made to
methane concentrations during data processing.
OTM 33A measurement method
OTM 33A is one of the EPA Geospatial Measurement of Air Pollution Remote
Emission Quantification (GMAP-REQ) techniques that was designed to observe,
characterize, and/or quantify emissions from a variety of sources, though
OTM 33A has been used most often to measure emissions from O&G operations
. While several quantification approaches are possible with
OTM 33A, the one most commonly employed is an inverse Gaussian approach,
which is the focus of this manuscript. OTM 33A has three operational parts:
concentration mapping, source characterization, and emission rate
quantification. Detection of emissions occurs by driving downwind of possible
emission sources in an attempt to transect an emissions plume, measure the
ambient background trace gas mixing ratio, and, if possible, to rule out any
emissions from upwind sources. Source characterization includes observations
of temporal variability and emissions composition. If enhancements of methane
or other trace gases are detected during downwind transects of a possible
source, the laboratory is parked 20–200 m directly downwind within the
emission plume to quantify emissions. Care is taken to position the mast
directly into the dominant wind direction to minimize impact from turbulent
eddies around the vehicle. Once the laboratory is safely positioned, the
vehicle is turned off and an OTM 33A flux measurement begins. During the
∼20 min measurement, 2 Hz measurements of wind direction (in x,
y, and z), wind speed, temperature, and the methane mixing ratio are
collected and time-stamped with a universal data system time. Meanwhile,
distance to the possible emission sources relative to the mast of the
laboratory are measured using a TruePulse laser range finder (Model 200). If
possible, the most likely emission source is identified using an infrared
camera (FLIR GF300). Site photos and observations are also collected.
The OTM 33A analysis program, written in , estimates an
emission mass flux, Q [g s-1], by using the Gaussian dispersion
equation (Eq. ). The terms of this equation are found as
follows. First, the lowest 5 % of measured mixing ratios during the ∼20 min measurement are averaged and considered ambient background, which
was around 1.9 ppm (±0.15 ppm) of methane for this study. The
background value is subtracted from the data to yield methane enhancement.
The analysis program bins observed methane enhancements by wind direction
into 10∘ bins (Fig. a), and then calculates the
average methane enhancement observed in that wind bin. A plot of methane
enhancement vs. wind direction is then generated and fit to a Gaussian
distribution (Fig. b). The Gaussian fit's apex is
Cpeak [g m-3]. To determine the expected spreading of the
emission plume, the program calculates atmospheric stability indicator
(ASI) values. The ASI values are based on the standard deviation of the two-dimensional
wind direction (horizontal spreading), and the standard deviation in vertical
wind speed (vertical spreading), also known as the turbulent intensity. The
horizontal and vertical ASI values are averaged together into a point Gaussian
indicator (PGI) value, which parameterizes the vertical and horizontal plume
spread experienced during the OTM measurement. There are seven PGI values
which correspond to Pasquill stability classes A–D
. The PGI and measured source distance are
used as inputs to a lookup table that gives the plume dispersion in two
dimensions, σy [m] and σz [m]. The average wind speed
U‾ [m s-1] is also calculated for the same time periods
that the methane enhancements are observed.
Q=2×π×σy×σz×U‾×Cpeak
Equation () does not include any terms for ground
reflection of the plume, plume buoyancy/velocity, or differences in height of
the emission source and measurement inlet. OTM 33A assumes a single emission
point. For this reason, OTM 33A is best suited for measuring O&G facilities
with equipment concentrated in one area that have downwind roadways. OTM 33A
struggles to quantify plumes with a particularly high vertical velocity or
buoyancy (such as manual unloadings, lit or unlit flares, or very hot
emissions). In this scenario, the calculated Cpeak will not
represent the center of the emission plume, leading to underestimations of
these sources . The estimated lower detection limit of the
method is 0.01 g s-1 (0.036 kg h-1;
).
Summed methane enhancement and total number of data points
in each 10∘ wind bin (a). Average methane enhancement per
10∘ wind bin and Gaussian fit (b). Goodness of fit
parameter R is calculated following Eq. (S1) in the
Supplement.
A series of built-in data quality indicators (DQI) will flag an OTM 33A flux
estimate for a variety of reasons, including poor Gaussian fit, inadequate
sampling time within the emission plume, too-variable wind speed or
direction, or a maximum methane enhancement that is too small. Flags are then
added up, and measurements are broken into categories that represent the
probability that an OTM measurement is a good flux estimate. For the current
study, the same approach as and
was used where most of the Category 1 and a few Category 2 measurements that
were only flagged for low methane concentrations (max enhancement less than
100 ppb above background) were considered. Occasionally, measurements with
very few DQI flags (Category 1 measurements) will be thrown out after review
of the Gaussian fit or if IR camera images suggest we are missing most of the
emission plume. Full descriptions of the DQI can be found in Sect. S1.2 in
the Supplement, , , and in
the EPA's documentation .
Test releases
The University of Wyoming performed two sets of test releases to assess the
ability of OTM 33A to quantify methane emissions. The first set of tests, the
Christman Field Test Releases (CF-TR), were conducted in conjunction with
Colorado State University in July and August of 2014 at the abandoned
Christman Airfield in Fort Collins, CO. These releases consisted of two
configurations, a simple point source (an opened gas cylinder) and manifold
(an elevated ∼2 m length of PVC pipe with many perforations). Neither source of methane gas was
obstructed, and they were, in essence, single point sources, one slightly
broader than the other. Release rates were set using calibrated mass flow
controllers and are correct to within 5 %. These tests spanned a variety
of release rates (0.2 to 2 kg h-1) and were staged in an open field
with no obstructions (clear line of site) between the single methane source
and mobile lab. Winds ranged from 2 to 8 m s-1 from the S/SE. The
calculated PGI ranged from 2 to 6, which roughly correspond to
Pasquill–Gifford stability classes A–D. Mean measurement distance was 78 m,
with a range of 34–174 m. Details of these results are reported in
and .
The more-recent set of tests were performed at the Methane Emissions
Technology Evaluation Center (METEC) in Fort Collins, CO in June of 2017.
METEC contains multiple faux O&G facilities ranging in size and complexity
with decommissioned O&G equipment that has been plumbed to release a known
amount of natural gas (>94 % methane) from a multitude of points. For
this study, we used one METEC site representative of a small O&G facility
that included a condensate storage tank, separator, and wellhead, all of
which were plumbed to be possible emission sources, 11 of which were used in
this study (Fig. ). This resulted in 15 release configurations
that had from one to three release points at different heights (0.33–4 m),
up to 6 m apart from one another. The relative complexity of the site also
introduced obstructions (the methane release would have to flow around a
large tank or other piece of equipment to reach the mobile lab) which could
potentially impact release quantification. Releases spanned 0.17 to
2.15 kg h-1 and were controlled by combining flows from a number of
critical orifices, resulting in a four σ release error less than
5 %. Meteorological conditions ranged from sunny to partly cloudy, with
average winds from 2 to 9 m s-1 from the E/SE. The calculated PGI
ranged from 3 to 6, which roughly correspond to Pasquill–Gifford stability
class B–D. Mean measurement distance was 114 m with a range of 53–195 m.
One to two duplicate OTM 33A measurements were attempted at different
distances for each of the 15 unique METEC test releases configurations.
Results
23 OTM 33A test releases were measured during the CF-TR; 21 passed the data
quality indicators (DQI; ) and were included in this
analysis. Thirty-four test releases were measured during the METEC-TR, of
which 24 passed the DQI and were included in this study. A similar
success rate, ∼70 %, has been observed in the majority of the
basins measured by the University of Wyoming . Of the
24 successful measurements during the METEC test releases, there were 10
replicate measurements (same release configuration but different OTM 33A
measurement distances). The following analysis explores different statistical
approaches to constrain the error associated with individual OTM 33A
measurements and to assess the accuracy and precision of an ensemble of
OTM 33A measurements. The latter analysis is especially important given this
is how OTM 33A measurements are often scaled up to estimate basin-wide
emissions from O&G.
Evaluating the accuracy of OTM 33APercent error analysis
%Error=OTM 33A flux-known releaseknown release×100
Percent error (Eq. ) was calculated for each individual
measurement made during the test releases. A histogram of percent error for
both the CF-TR and METEC-TR indicate a large range of over- and
underestimations are possible using OTM 33A (Fig. ).
Percent error ranges from -75 % to 50 % and -60 % to
170 % for CF-TR and METEC-TR, respectively. Figures
and show that the larger percent errors correlate
with smaller release rates, with OTM 33A generally overestimating smaller
releases. Of the CF-TR data, 68 % fall within ±28 % of the known
release, which is the 1σ error used by and
similar to the error reported by the EPA of 72 % of measurements within
±30 % of the known release . Of the METEC-TR
data, 68 % are within ±38 % of the known release, perhaps
suggesting that a slightly higher 1σ error is appropriate, especially
if measuring emissions fluxes less than 0.5 kg h-1. For the combined
set of test releases, greater than 85 % of the data are within ±50 % of the known value, and 95 % of the data are within ±73 %. If a Gaussian curve is fit to all of the test release data
(N=45), the 95 % confidence interval is found to be +54 % to
-84 %, suggesting a low bias of -15 % and a 2σ error of
±69 % (Fig. S5 in the Supplement). The rounded 2σ confidence
interval for test releases of ±70 % would become 0.58q and 3.33q
when q is an OTM 33A estimate made of an unknown emission source in an O&G
basin. The number of replicate measurements of METEC release configurations
were too small to perform a similar statistical analysis (N=10), but
multiple measurements did not decreased the mean OTM 33A measurement error
(14.7 % for replicate measurements, 13.1 % for all measurements).
Replicate measurements have been shown to improve flux estimates but at the
expense of measuring a number of unique sites .
Histogram of percent error of the OTM 33A flux estimate for both
Christman and METEC test releases. Data are binned in 10 % error bins.
Positive percent error corresponds to OTM 33A overestimating the known
release rate.
Scatterplot of test release error and release rate. Positive percent
error corresponds to OTM 33A overestimating the known release
rate.
Box plot of METEC and Christman OTM 33A release errors binned by
release rate. The rectangle contains the median value, while the edges
represent the 25th and 75th percentiles. Box whiskers include the rest of the
data (100 % coverage). Positive percent error corresponds to OTM 33A
overestimating the known release rate.
Ordinary least-squares regression
Another approach to assess the performance of OTM 33A is using an ordinary
least-squares (OLS) regression applied to a correlation plot of the OTM 33A
flux estimate versus the known release rate. Assuming the OTM-measured flux
and known release rate converge at (0,0) yields OLS slopes of 0.91 for
CF-TR and 0.92 for METEC-TR (Fig. ). This suggests
OTM 33A may have a ∼-10 % negative bias when an ensemble of
measurements are considered. Notably, the increased complexity of the
METEC-TR did not yield a more significant bias like that reported by
. Statistical analysis of the residuals to assess point
leverage and possible outliers supports the validity of an OLS approach for
the test release data. Removing the largest outlier found with the Cook's
test improves both OLS fits slightly to 0.97 for METEC-TR and 1.1 for CF-TR.
Residual plots are included in Sect. S2.
Correlation plot of OTM 33A-measured flux versus known release rates. Intercept is set to (0,0).
95 % confidence intervals (CI) for the OLS fit were calculated through
bootstrapping following the method detailed in ,
whereby the linear regressions of bootstrapped data sets are calculated to
assess the range of possible regressions. Bootstrapping was used because it
does not require an assumption of normally distributed data (unlike the
Gaussian fit approach used in Sect. ). Using this
method, the OLS correlation slopes have a mean (and 95 % CI) for the
CF-TR of 0.96(0.56,1.47) and 0.96(0.66,1.28) for the METEC-TR.
Bland–Altman analysis
Because the rate of methane releases for both the METEC and Christman tests
are known to within a small margin of error (<5 %), OLS regression,
which assumes no error in the independent variable, is a reasonable approach.
However, OLS analysis is weighted by larger release rates and may not give an
accurate representation of OTM 33A performance at all methane emission rates.
Bland–Altman (BA) analysis removes this bias by considering the difference
between the test release and OTM measurements (known release – OTM flux) as
a function of a known release rate (Fig. ;
). Bland–Altman analysis also assumes that the
method difference (y axis) comes from a normal distribution.
Kolmogorov–Smirnov statistical tests supporting the normality of the method
difference can be found in Sect. S3. For BA analysis, if the 2σ range
of method difference includes zero, the methods are considered to be
statistically equivalent, i.e., without bias . The BA
plot also illustrates the amount by which OTM can overestimate (negative numbers) or
underestimate (positive numbers) the known release. On average, the CF-TR and
METEC-TR both underestimate the known releases, with mean differences of
0.028 and 0.025 kg h-1, respectively. However, since the 2σ
interval includes zero, the BA analysis identifies no statistical difference
between the OTM 33A flux estimate and the known release rate.
Bland–Altman analysis of Christman and METEC test releases. Bold
black lines represent the mean difference between the test release and the
OTM measurement, while the dashed yellow and red lines indicate 1 and 2
standard deviations of the method difference.
Orthogonal distance and variance-weighted least-squares regression
Other approaches for minimizing the influence of larger release rates on the
OLS fit include orthogonal distance regression (ODR) and variance-weighted
least-squares regression (VWLS). These methods take into account error in
both the x and y variables, and require that each measurement has an
independent uncertainty estimate on both axes. Since uncertainty of OTM 33A
flux estimates is taken as a fixed percentage of the estimated value, these
methods tend to perceive a higher confidence (smaller absolute uncertainty)
in smaller estimates, and a lower confidence (higher absolute uncertainty) in
larger estimates. This results in a fit with a low bias since the estimates
with smaller absolute uncertainty are strongly weighted, and less weight is
given to estimates with larger uncertainties. To examine these approaches,
the METEC-TR are used as an example below.
Applying a measurement uncertainty of ±50 % (representing the
percent error that roughly 85 % of the data points are within) for each
OTM measurement and the metered uncertainty for each METEC-TR in kg h-1
yields an ODR slope of 0.79±0.09 when the intercept is set to (0,0)
(Fig. ). A lower slope of 0.67±0.1 is found
using the VWLS method. In this case the ODR and VWLS regressions suggest the
OTM flux estimates are 20 %–33 % lower than the known releases,
where an OLS regression indicates the method is only 8 %
low. Total emissions estimated by OTM 33A
(23.074 kg h-1) are 2.5 % lower than the total known emission
rates (23.67 kg h-1), suggesting OLS regression is a better fit for
this data set.
Correlation plot of known release and OTM 33A flux estimate for the METEC test releases. Plot includes OLS (slope=0.92), ODR (slope=0.79), and VWLS (slope=0.67) regressions.
VWLS and ODR should be used with caution where the measurement uncertainty is
not independent of the measurement (i.e., a constant fractional error),
because this may discriminate against data points of a larger magnitude
depending on uncertainty in the other (x or y) variable.
used a VWLS regression to compare OTM 33A measurements to
on-site measurements of O&G production facilities in the Fayetteville Shale,
which yielded a correlation of 0.41(+0.51,-0.17). The 95 % CI of the
VWLS regression are calculated through bootstrapping the regression and
considering the uncertainties in both the study on-site estimate and the
OTM 33A estimate for each data point. Repeating this analysis using an OLS
regression without an intercept results in a correlation of 0.39(+0.39,-0.15) (Fig. S6). If an OLS regression were chosen instead of the VWLS, the
conclusion by that OTM 33A produced lower emissions
estimates than the on-site measurement results would not have been effected.
The total mass flux measured by OTM 33A (13(+5.3,-2.1) kg h-1) and
on-site teams (19(+7.7,-3) kg h-1) also supports the conclusion
that the OTM 33A flux estimate was biased low relative to the on-site
measurements at these paired facilities.
Sensitivity analysisOTM 33A sensitivity to source distance
Because OTM 33A assumes a point source, the distance to the release point has
a large influence, as this impacts the modeled plume spread and therefore the
final calculated flux. The importance of an accurate source distance in
Gaussian plume modeling has been noted in previous studies
. During the METEC-TR, the University of
Wyoming measurement team had site access and we were able to determine the
exact emission point(s) using an IR camera. With this knowledge, we were able
to calculate the exact source distance, or the average distance in the case
of multiple emission sources. In the field, site access is often not
available and it is often not possible to detect the most likely emission
point(s). For this reason, the average distance of possible emission points
is used when calculating source distance.
OTM 33A sensitivity to source distance was tested two ways for the METEC test
releases. The following test was performed during the data analysis stage,
and compared the flux estimated using the average distance of all the
components that could be sighted with the range finder from the van (e.g.,
wellhead, separator, tank) to the flux estimated using the distance to the
known release point or point distance (identified using the FLIR camera).
Although the well pad measured at the METEC facility was quite small (∼6 m by 6 m), the average source distance was larger than the specific
source distance ∼60 % of the time. The change in the OTM 33A flux
(ΔFlux) as a result of changing the measurement distance
(ΔDistance) was found using Eqs. ()
and ().
3%ΔDistance=Average Distance-Point DistanceAverage Distance×1004%ΔFlux=Average OTM-Point OTMAverage OTM×100
A correlation plot of %ΔDistance and %ΔFlux
suggests that for a 5 % change in source distance, the OTM 33A flux
estimate would increase by almost 10 % (Fig. a).
In terms of mass error, the OTM flux estimated by the average or specific
source distance has very little impact on the over- or underestimation of the
METEC known release (Fig. b). Allowing this fit to
have an intercept changes the linear fit to y=0.978x-0.03, a negligible
difference. Source distance-related error is small in the context of the ±70 % measurement error, but this analysis underscores how determination
of the exact emission point can further reduce errors in the field.
(a) Correlation between percent change in distance ((Avg. distance-point)/point⋅100), and resulting percent change in OTM flux ((OTM avg-OTM)/OTM⋅100). (b) OTM 33A flux mass error compared to the METEC known release when using average versus known source distance.
OTM 33A sensitivity to distance was also tested in the field during the METEC
test releases. For configurations that had both a “closer”
(generally<70 m) and “farther” (generally>100 m)
measurement distance for replicate measurements, the closer measurement had a
flux estimate closer to the known release 78 % of the time (Sect. S1.4).
The average distance of the closer replicate measurements (78 m) is
comparable to the average measurement distances for the CF-TR of 78 m,
smaller than the mean METEC-TR distance of 114 m, and larger than the
measurement distances during the Arkansas campaign of 46 m (20–113 m;
). For both the CF-TR and the
METEC-TR, there is no obvious increase in percent error as measurement
distance increases (Fig. a), suggesting the
underestimation reported by cannot be blamed solely on
closer measurement distances.
OTM 33A error – sensitivity to wind speed
One hypothesized reason for the underestimation of OTM 33A compared to
on-site methods reported in the study is the lower wind
speeds (<2 m s-1) experienced in that study. The CF-TR and METEC-TR
both had wind speeds higher than 2 m s-1, making an absolute
conclusion impossible, but for the wind speeds measured there is no obvious
trend between the mean measurement wind speed and OTM 33A error
(Fig. b).
OTM 33A percent error compared to the measurement
distance (a). OTM 33A percent error compared to the mean measurement
wind speed (b). Positive percent error is OTM overestimating the
known release.
OTM 33A error – sensitivity to number of sources and source height
The METEC-TR included multiple emission points, both slightly above and below
the sampling inlet height. There is no obvious trend between the number of
release points and percent error, though the sample size for two or more
sources is relatively small (N=6). The height of the sources tested also
show no obvious influence on OTM 33A accuracy (Fig. ).
Percent error of the METEC-TR depending on source height or
average height (for multiple sources). Positive error is OTM 33A
overestimating the known release rate. Icons indicate the size bin of the
known release rate.
Ensemble mass flux
OTM 33A measurements are often used to find an average emission rate per well
or per facility in an O&G basin . To
assess the accuracy of the mean of a number of OTM 33A measurements, the mean
mass flux measured by OTM 33A is compared to the mean mass flux of the known
release through bootstrapping. The bootstrapping approach is used to generate
more statistically robust results without the need for assuming Gaussian
distributions. The OTM 33A flux estimates and known releases (including their
respective measurement uncertainties) are sampled with replacement, summed,
and compared following . This approach suggests that
the addition of complexity in the METEC-TR did not significantly impact the
accuracy of OTM 33A (Fig. ), and for both sets of test
releases there is a large amount of overlap between the OTM 33A and known
release distributions (Fig. S6). These results also indicate OTM 33A does not
drastically underestimate the total emissions for an ensemble or group of
measurements, and that scaling-up mean emissions measured with OTM 33A to an
entire basin is a valid approach.
Probability density function (PDF) of bootstrapped mean mass
emission flux for OTM measurements and the known releases. Mean and 95 %
CI in kg h-1 are as follows. Christman: known release – (0.54(0.37,0.75)), OTM – (0.51(0.34,0.73)). METEC: known release – (0.85(0.58,1.13)), OTM – (0.84(0.60,1.11)).
Figure summarizes measurement conditions experienced
during four University of Wyoming field campaigns
and the test releases. Data from the four O&G basins represented a
significant fraction of data, along with other field campaigns used by
to generate an estimate of national methane emissions.
Measurement conditions in Fayetteville, Arkansas (AR), were notable for
closer source distance, lower wind speeds, and generally more unstable
atmospheric conditions. All of these variables could have influenced the low
bias reported by and are conditions that should be
replicated (if possible) in future test releases.
Summary of accepted OTM 33A measurements from field deployments and
test releases (right of vertical line). Basins from .
Upper Green River Basin, Wyoming (UGRB), Uintah Basin, UT (UB),
Denver–Julesburg Basin, CO (DJ), Fayetteville, Arkansas (AR). Mean
statistics (from left to right) are as follows. Distance (m): 98, 114, 83,
51, 114, 78. Flux (kg h-1): 2.41, 6.99, 1.51, 1.27, 0.51, 0.96. Mean
wind speed (m s-1): 5.3, 4.2, 3.1, 2.9, 4.1, 4.9. Stability class: 5.0,
4.9, 5.3, 3.5, 5.0, 5.4.
Conclusions
The more realistic test releases described in this study build
on preexisting test releases and suggest a single OTM 33A measurement can
have a 2σ error of ±70 %. Analysis of both the simple CF-TR
and more complex METEC-TR indicate that under these measurement conditions
and release rates, an ensemble of OTM 33A may have a slight negative bias
(∼5 %) when compared to a known release rate through an OLS model.
The mean and 95 % CI found through bootstrapping are 0.96(0.56,1.47)
and 0.96(0.66,1.28) for the CF-TR and METEC-TR, respectively. The
40 %–60 % underestimation reported in the study
was not replicated during either test release experiment.
OTM 33A flux estimates are sensitive to the assumed source distance, with a
+5 % change in source distance corresponding to a ∼+10 %
change in the OTM flux. However, the error caused by uncertainty in source
distance is small compared to the measurement method error determined through
these test releases. During field measurements, uncertainty in source
distance can be mitigated by having site access and using an IR camera to
detect the emission source(s). Uncertainty did not correspond to wind speeds
observed during the test releases, but was relatively higher for smaller
release rates. Sensitivity of OTM 33A to the number or height of emission
sources was inconclusive.
For both test release experiments, the maximum release rates
(2–2.15 kg h-1) were constrained by available resources and facility
throughput and, while they represent a large fraction of emission rates
observed in the field, they do not fully encompass the dynamic range of
emissions observed in an O&G basin. The bootstrapped mean emission rates
from four O&G basins measured by the University of Wyoming range from 0.68
to 3.7 kg h-1, suggesting the range of these
test releases may not be representative of the largest emission rates
observed in the field (Fig. ).
OTM 33A has been used to estimate mean facility emissions and basin-wide
facility emissions in a number of O&G basins. The mean mass fluxes and
95 % CI for each test release experiment are not statistically different.
This analysis lends confidence to national emission estimates from the O&G
production sector using OTM 33A measurements. Despite the OTM 33A estimated
limit of detection (0.01 g s-1) and relative overestimation of smaller
release rates, the analyses reported here and the study by
suggest that OTM 33A does not overestimate an ensemble of
flux estimates.
Code and data availability
Analysis code and raw data are available on request.
The supplement related to this article is available online at: https://doi.org/10.5194/amt-13-341-2020-supplement.
Author contributions
Authors RE, ARM, DAS, JS, SMM, and RAF collected data and helped design the study. Authors DZ, CSB, TLV helped with study design, statistical methods, and study comparison. Authors RE, AMR, SMM, RAF, and CSB wrote and edited the manuscript. Authors DAS, JS, and DZ provided feedback on the manuscript.
Competing interests
The authors declare that they have no conflict of interest.
Acknowledgements
The authors acknowledge support from the Wyoming School of Energy Resources Center of Excellence in Air Quality, the Clean Air Task Force, and the Methane Emissions Technology Evaluation Center.
Review statement
This paper was edited by Thomas Röckmann and reviewed by two anonymous referees.
References
Allen, D. T.: Emissions from oil and gas operations in the United States and their air quality implications, J. Air Waste Manage., 66, 549–575, 2016.
Alvarez, R. A., Pacala, S. W., Winebrake, J. J., Chameides, W. L., and Hamburg, S. P.: Greater focus needed on methane leakage from natural gas infrastructure, P. Natl. Acad. Sci. USA, 109, 6435–6440, 2012.
Alvarez, R. A., Zavala-Araiza, D., Lyon, D. R., Allen, D. T., Barkley, Z. R., Brandt, A. R., Davis, K. J., Herndon, S. C., Jacob, D. J., Karion, A., Kort, E. A., Lamb, B. K., Lauvaux, T., Maasakkers, J. D., Marchese, A. J., Omara, M., Pacala, S. W., Peischl, J., Robinson, A. L., Shepson, P. B., Sweeney, C., Townsend-Small, A., Wofsy, S. C., and Hamburg, S. P.: Assessment of methane emissions from the U.S. oil and gas supply chain, Science, 2, 7204–9, 2018.
Bell, C., Vaughn, T., Zimmerle, D., Herndon, S., Yacovitch, T., Heath, G., Pétron, G., Edie, R., Field, R., Murphy, S., Robertson, A., and Soltis, J.: Comparison of methane emission estimates from multiple measurement techniques at natural gas production pads, Elem. Sci. Anth., 5, 1–14, 2017.
Brandt, A. R., Heath, G. A., Kort, E. A., O'Sullivan, F., Pétron, G., Jordaan, S. M., Tans, P., Wilcox, J., Gopstein, A. M., Arent, D., Wofsy, S., Brown, N. J., Bradley, R., Stucky, G. D., Eardley, D., and Harriss, R.: Methane Leaks from North American Natural Gas Systems, Science, 343, 733–735, 2014.
Brantley, H. L., Thoma, E. D., Squier, W. C., Guven, B. B., and Lyon, D.: Assessment of Methane Emissions from Oil and Gas Production Pads using Mobile Measurements, Environ. Sci. Technol., 48, 14508–14515, 2014.
Brantley, H. L., Thoma, E. D., and Eisele, A. P.: Assessment of volatile organic compound and hazardous air pollutant emissions from oil and natural gas well pads using mobile remote and on-site direct measurements, J. Air Waste Manage., 65, 1072–1082, 2015.
Caulton, D. R., Shepson, P. B., Santoro, R. L., Sparks, J. P., Howarth, R. W., Ingraffea, A. R., Cambaliza, M. O. L., Sweeney, C., Karion, A., Davis, K. J., Stirm, B. H., Montzka, S. A., and Miller, B. R.: Toward a better understanding and quantification of methane emissions from shale gas development, P. Natl. Acad. Sci. USA, 111, 6237–6242, 2014.Caulton, D. R., Li, Q., Bou-Zeid, E., Fitts, J. P., Golston, L. M., Pan, D., Lu, J., Lane, H. M., Buchholz, B., Guo, X., McSpiritt, J., Wendt, L., and Zondlo, M. A.: Quantifying uncertainties from mobile-laboratory-derived emissions of well pads using inverse Gaussian methods, Atmos. Chem. Phys., 18, 15145–15168, 10.5194/acp-18-15145-2018, 2018.
Coburn, S., Alden, C. B., Wright, R., Cossel, K., Baumann, E., Truong, G.-W., Giorgetta, F., Sweeney, C., Newbury, N. R., Prasad, K., Coddington, I., and Rieker, G. B.: Regional trace-gas source attribution using a field-deployed dual frequency comb spectrometer, Optica, 5, 320–327, 2018.Conley, S., Faloona, I., Mehrotra, S., Suard, M., Lenschow, D. H., Sweeney, C., Herndon, S., Schwietzke, S., Pétron, G., Pifer, J., Kort, E. A., and Schnell, R.: Application of Gauss's theorem to quantify localized surface emissions from airborne measurements of wind and trace gases, Atmos. Meas. Tech., 10, 3345–3358, 10.5194/amt-10-3345-2017, 2017.Daube, C., Conley, S., Faloona, I. C., Arndt, C., Yacovitch, T. I., Roscioli, J. R., and Herndon, S. C.: Using the tracer flux ratio method with flight measurements to estimate dairy farm CH4 emissions in central California, Atmos. Meas. Tech., 12, 2085–2095, 10.5194/amt-12-2085-2019, 2019.
Foster-Wittig, T. A., Thoma, E. D., and Albertson, J. D.: Estimation of point source fugitive emission rates from a single sensor time series: A conditionally-sampled Gaussian plume reconstruction, Atmos. Environ., 115, 101–109, 2015.
Giavarina, D.: Understanding Bland Altman analysis, Biochem. Medica, 25, 141–151, 2015.
Harriss, R., Alvarez, R. A., Lyon, D., Zavala-Araiza, D., Nelson, D., and Hamburg, S. P.: Using Multi-Scale Measurements to Improve Methane Emission Estimates from Oil and Gas Operations in the Barnett Shale Region, Texas, Environ. Sci. Technol., 49, 7524–7526, 2015.
Karion, A., Sweeney, C., Pétron, G., Frost, G., Michael Hardesty, R., Kofler, J., Miller, B. R., Newberger, T., Wolter, S., Banta, R., Brewer, A., Dlugokencky, E., Lang, P., Montzka, S. A., Schnell, R., Tans, P., Trainer, M., Zamora, R., and Conley, S.: Methane emissions estimate from airborne measurements over a western United States natural gas field, Geophys. Res. Lett., 40, 4393–4397, 2013.
Karion, A., Sweeney, C., Kort, E. A., Shepson, P. B., Brewer, A., Cambaliza, M., Conley, S. A., Davis, K., Deng, A., Hardesty, M., Herndon, S. C., Lauvaux, T., Lavoie, T., Lyon, D., Newberger, T., Pétron, G., Rella, C., Smith, M., Wolter, S., Yacovitch, T. I., and Tans, P.: Aircraft-Based Estimate of Total Methane Emissions from the Barnett Shale Region, Environ. Sci. Technol., 49, 8124–8131, 2015.
Lan, X., Talbot, R., Laine, P., and Torres, A.: Characterizing Fugitive Methane Emissions in the Barnett Shale Area Using a Mobile Laboratory, Environ. Sci. Technol., 49, 8139–8146, 2015.
Lavoie, T. N., Shepson, P. B., Cambaliza, M. O. L., Stirm, B. H., Karion, A., Sweeney, C., Yacovitch, T. I., Herndon, S. C., Lan, X., and Lyon, D.: Aircraft-Based Measurements of Point Source Methane Emissions in the Barnett Shale Basin, Environ. Sci. Technol., 49, 7904–7913, 2015.
Lavoie, T. N., Shepson, P. B., Gore, C. A., Stirm, B. H., Kaeser, R., Wulle, B., Lyon, D., and Rudek, J.: Assessing the Methane Emissions from Natural Gas-Fired Power Plants and Oil Refineries, Environ. Sci. Technol., 51, 3373–3381, 2017.
MATLAB: version 8.5.0 (R2015a), The MathWorks Inc., Natick, Massachusetts, 2015.
Omara, M., Zimmerman, N., Sullivan, M. R., Li, X., Ellis, A., Cesa, R., Subramanian, R., Presto, A. A., and Robinson, A. L.: Methane Emissions from Natural Gas Production Sites in the United States: Data Synthesis and National Estimate, Environ. Sci. Technol., 52, 12915–12925, 2018.
Peischl, J., Ryerson, T. B., Aikin, K. C., de Gouw, J. A., Gilman, J. B., Holloway, J. S., Lerner, B. M., Nadkarni, R., Neuman, J. A., Nowak, J. B., Trainer, M., Warneke, C., and Parrish, D. D.: Quantifying atmospheric methane emissions from the Haynesville, Fayetteville, and northeastern Marcellus shale gas production regions, J. Geophys. Res.-Atmos., 120, 2119–2139, 2015.
Peischl, J., Karion, A., Sweeney, C., Kort, E. A., Smith, M. L., Brandt, A. R., Yeskoo, T., Aikin, K. C., Conley, S. A., Gvakharia, A., Trainer, M., Wolter, S., and Ryerson, T. B.: Quantifying atmospheric methane emissions from oil and natural gas production in the Bakken shale region of North Dakota, J. Geophys. Res.-Atmos., 121, 6101–6111, 2016.Peischl, J., Eilerman, S. J., Neuman, J. A., Aikin, K. C., de Gouw, J., Gilman, J. B., Herndon, S. C., Nadkarni, R., Trainer, M., Warneke, C., and Ryerson, T. B.: Quantifying Methane and Ethane Emissions to the Atmosphere From Central and Western U.S. Oil and Natural Gas Production Regions, J. Geophys. Res.-Atmos., 123, 7725–7740, 10.1029/2018JD028622, 2018.Pétron, G., Karion, A., Sweeney, C., Miller, B. R., Montzka, S. A., Frost, G. J., Trainer, M., Tans, P., Andrews, A., Kofler, J., Helmig, D., Guenther, D., Dlugokencky, E., Lang, P., Newberger, T., Wolter, S., Hall, B., Novelli, P., Brewer, A., Conley, S., Hardesty, M., Banta, R., White, A., Noone, D., Wolfe, D., and Schnell, R.: A new look at methane and nonmethane hydrocarbon emissions from oil and natural gas operations in the Colorado Denver-Julesburg Basin, J. Geophys. Res.-Atmos., 119, 6836–6852, 10.1002/2013JD021272, 2014.
Ravikumar, A. P., Wang, J., and Brandt, A. R.: Are Optical Gas Imaging Technologies Effective For Methane Leak Detection?, Environ. Sci. Technol., 51, 718–724, 2016.
Ravikumar, A. P., Wang, J., McGuire, M., Bell, C. S., Zimmerle, D., and Brandt, A. R.: “Good versus Good Enough?” Empirical Tests of Methane Leak Detection Sensitivity of a Commercial Infrared Camera, Environ. Sci. Technol., 52, 2368–2374, 2018.
Rella, C. W., Tsai, T. R., Botkin, C. G., Crosson, E. R., and Steele, D.: Measuring Emissions from Oil and Natural Gas Well Pads Using the Mobile Flux Plane Technique, Environ. Sci. Technol., 49, 4742–4748, 2015.
Robertson, A. M., Edie, R., Snare, D., Soltis, J., Field, R. A., Burkhart, M. D., Bell, C. S., Zimmerle, D., and Murphy, S. M.: Variation in Methane Emission Rates from Well Pads in Four Oil and Gas Basins with Contrasting Production Volumes and Compositions, Environ. Sci. Technol., 51, 8832–8840, 2017.Roscioli, J. R., Yacovitch, T. I., Floerchinger, C., Mitchell, A. L., Tkacik, D. S., Subramanian, R., Martinez, D. M., Vaughn, T. L., Williams, L., Zimmerle, D., Robinson, A. L., Herndon, S. C., and Marchese, A. J.: Measurements of methane emissions from natural gas gathering facilities and processing plants: measurement methods, Atmos. Meas. Tech., 8, 2017–2035, 10.5194/amt-8-2017-2015, 2015.
Schwietzke, S., Pétron, G., Conley, S., Pickering, C., Mielke-Maday, I., Dlugokencky, E. J., Tans, P. P., Vaughn, T., Bell, C., Zimmerle, D., Wolter, S., King, C. W., White, A. B., Coleman, T., Bianco, L., and Schnell, R. C.: Improved Mechanistic Understanding of Natural Gas Methane Emissions from Spatially Resolved Aircraft Measurements, Environ. Sci. Technol., 51, 7286–7294, 2017.
Snare, D. A.: A comparison of ground-based and aircraft-based methane emission flux estimates in a western oil and natural gas production basin, MS thesis, University of Wyoming, Laramie, Wyoming, United States of America, 2015.
Thoma, E., Squier, B., Olson, D., Eisele, A., DeWees, J., Segall, R., Amin, M., and Modrak, M.: Assessment of methane and voc emissions from select upstream oil and gas production operations using remote measurements, interim report on recent survey studies, in: Proceedings of 105th Annual Conference of the Air & Waste Management Association, Durham, NC, USA, 24–26 April 2012, Control No. 2012-A-21-AWMA, 298-−312, 2012.U.S. EPA: Other Test Method (OTM) 33 and 33A Geospatial Measurement of Air Pollution-Remote Emissions Quantification-Direct Assessment (GMAP-REQ-DA), available at: https://www.epa.gov/emc/emc-other-test-methods (last access: 21 January 2020), 2014.
U.S. EPA: Inventory of U.S. Greenhouse Gas Emissions and Sinks 1990–2017, EPA, 2019.van Kessel, T. G., Ramachandran, M., Klein, L. J., Nair, D., Hinds, N., Hamann, H., and Sosa, N. E.: Methane Leak Detection and Localization Using Wireless Sensor Networks for Remote Oil and Gas Operations, 2018 IEEE SENSORS, New Delhi, India, 28–31 October 2018, IEEE, 1–4, 10.1109/ICSENS.2018.8589585, 2018.Vaughn, T. L., Bell, C. S., Pickering, C. K., Schwietzke, S., Heath, G. A., Pétron, G., Zimmerle, D. J., Schnell, R. C., and Nummedal, D.: Temporal variability largely explains top-down/bottom-up difference in methane emission estimates from a natural gas production region, P. Natl. Acad. Sci. USA, 115, 11712–11717, 10.1073/pnas.1805687115, 2018.
Yacovitch, T. I., Herndon, S. C., Pétron, G., Kofler, J., Lyon, D., Zahniser, M. S., and Kolb, C. E.: Mobile Laboratory Observations of Methane Emissions in the Barnett Shale Region, Environ. Sci. Technol., 49, 7889–7895, 2015.
Yacovitch, T. I., Daube, C., Vaughn, T. L., Bell, C. S., Roscioli, J. R.,
Knighton, W. B., Nelson, D. D., Zimmerle, D., Herndon, G. P., and C, S.: Natural gas facility methane emissions: measurements by tracer flux ratio in two US natural gas producing basins, Elem. Sci. Anth., 5, 1–12, 2017.
Zaimes, G. G., Littlefield, J. A., Augustine, D. J., Cooney, G., Schwietzke,
S., George, F. C., Lauderdale, T., and Skone, T. J.: Characterizing Regional
Methane Emissions from Natural Gas Liquid Unloading, Environ. Sci. Technol., 53, 4619–4629, 2019.
Zavala-Araiza, D., Lyon, D. R., Alvarez, R. A., Davis, K. J., Harriss, R., Herndon, S. C., Karion, A., Kort, E. A., Lamb, B. K., Lan, X., Marchese, A. J., Pacala, S. W., Robinson, A. L., Shepson, P. B., Sweeney, C., Talbot, R., Townsend-Small, A., Yacovitch, T. I., Zimmerle, D. J., and Hamburg, S. P.: Reconciling divergent estimates of oil and gas methane emissions,
P. Natl. Acad. Sci. USA, 112, 15597–15602, 2015.Zavala-Araiza, D., Alvarez, R. A., Lyon, D. R., Allen, D. T., Marchese, A. J., Zimmerle, D. J., and Hamburg, S. P.: Super-emitters in natural gas infrastructure are caused by abnormal process conditions, Nat. Commun., 8, 14012, 10.1038/ncomms14012, 2017.