A quantitative comparison of methods used to measure smaller methane emissions typically observed from superannuated oil and gas infrastructure

Recent interest in measuring methane (CH₄) emissions from abandoned oil and gas infrastructure has resulted in several methods being continually used to quantify point source emissions less than 200 g CH₄ h⁻¹. The choice of measurement approach depends on how close observers can come to the source, the instruments available, and the meteorological/micrometeorological conditions. As such, static chambers, dynamic chambers, Bacharach Hi Flow Sampler (BHFS) measurements, Gaussian plume (GP) modeling, and backward Lagrangian stochastic (bLs) models have all been used, but there is no clear understanding of the accuracy or precision of each method. To address this, we copy the experimental design for each of the measurement methods to make single field measurements of a known source, to simulate single measurement field protocol, and then make repeat measurements to generate an understanding of the accuracy and precision of each method. Here, we present estimates for the average percentage difference between the measured emission and the known emission for three repeat measurements, A_r, for emissions of 40 to 200 g CH₄ h⁻¹. The static chamber data were not presented because of safety concerns during the experiments. Both the dynamic chamber (A_r = −10 %, −8 %, and −10 % at emission rates of 40, 100, and 200 g CH₄ h⁻¹, respectively) and BHFS (A_r = −18 %, −16 %, and −18 %) repeatedly underestimate the emissions, but the dynamic chamber had better accuracy. The standard deviation of emissions from these direct measurement methods remained relatively constant for emissions between 40 and 200 g CH₄ h⁻¹. For the far-field methods, the bLs method generally underestimated emissions (A_r = +6 %, −6 %, and −7 %) while the GP method significantly overestimated the emissions (A_r = +86 %, +57 %, and +29 %) despite using the same meteorological and concentration data as input. Variability in wind speed, wind direction, and atmospheric stability over the 20 min averaging period are likely to propagate through to large variability in the emission estimate, making these methods less precise than the direct measurement methods. To our knowledge, this is the first time that methods for measuring CH₄ emissions from point sources between 40 and 200 g CH₄ h⁻¹ have been quantitatively assessed against a known reference source and against each other.

1 Introduction

Methane (CH₄) gas is a powerful greenhouse gas with a greenhouse warming potential 86 times larger than carbon dioxide over 100 years (IPCC, 2022). Quantification of CH₄ emissions from abandoned wells has recently become an area of interest, as studies suggest over 200 Gg CH₄ yr⁻¹ is emitted from 2.2×10⁶ abandoned wells in the U.S. alone (US EPA, 2022). Quantifying and then plugging these wells makes them an attractive target for achieving goals set out in the Paris Agreement (Nisbet et al., 2020). Additionally, private companies are beginning initiatives to generate revenue through carbon credits gained by plugging wells, and accurate quantification is essential for realizing the capital.

As there are millions of abandoned wells globally, there is a growing need to measure as many wells as quickly as possible to identify the most emissive wells. Typically, an emission from an abandoned well can be considered to be an aboveground point source that is relatively small in emission size, up to 180 g CH₄ h⁻¹ (Riddick et al., 2019a; Pekney et al., 2018; Townsend-Small et al., 2016; Boothroyd et al., 2016; El Hachem and Kang, 2022; Saint-Vincent et al., 2020; Townsend-Small and Hoschouer, 2021). Other emission sources, such as emissions from pipeline leakage, are fundamentally different in behavior, where gas travels through the soil and forms an area emission at the surface, and these sources require different methods for estimating the emission, e.g., mass balance or eddy covariance. Area emissions could form if a plugged well leaks from corrosion of the borehole casing, but this will not be discussed in this study.

Several methods are being used to measure emissions from these smaller point sources (less than 200 g CH₄ h⁻¹) from abandoned oil and gas infrastructure. The chosen measurement approach depends on how close an observer can come to the source, instrumentation availability, and the meteorological/micrometeorological conditions at the measurement site. Measurement methods can be classed as direct, i.e., touching/enclosing the source, and downwind measurements where access is not possible. Direct methods include static chambers (Livingston and Hutchinson, 1995), dynamic flux chambers (Riddick et al., 2019a, 2020b; Aneja et al., 2006), and BHFS sampling (Pekney et al., 2018; Allen et al., 2013; Brantley et al., 2015). Downwind methods include Gaussian-based plume models (Baillie et al., 2019; Caulton et al., 2014; Riddick et al., 2019b, 2020a; Edie et al., 2020; Bell et al., 2017) and Lagrangian dispersion models (Riddick et al., 2019b, 2017; Denmead, 2008; Flesch et al., 1995). Emissions calculated using the majority of these methods have not been comprehensively compared to controlled emission source rates.

Other quantification methods are generally unsuitable for measuring emissions from abandoned wells. While optical gas imaging (OGI) cameras can be used for detecting emissions greater than 20 g CH₄ h⁻¹ (Ravikumar et al., 2018; Stovern et al., 2020; Zimmerle et al., 2020), using this method for quantification remains in development, with few studies published to date investigating the accuracy of emission rate estimates from OGI (Kang et al., 2022). Mass balance approaches are unlikely to detect the small and narrow plume from the abandoned well. Tracer release is technically demanding, takes a long time to make a single measurement, and requires road access for measurement, although it has been used to measure nonproducing wells in Hungary (Delre et al., 2022). Remote sensing has typical detection limits of 10+ kg CH₄ h⁻¹ for aircraft (Duren et al., 2019), 100+ kg CH₄ h⁻¹ for satellites (Cooper et al., 2022), and is unsuitable for these types of emission source. As such, these other quantification methods will not be investigated in this study.

In general, as access becomes more restricted, emission rates larger, or safety concerns increase (such as the co-emission of harmful gases), the method used to estimate the CH₄ emission rate of a source must be carefully considered. From experience and the response of a four-gas monitor, working close enough to measure emissions greater than 200 g CH₄ h⁻¹ for many of these methods (especially the chambers and BHFS) can be unsafe; therefore, this study is limited to quantifying CH₄ emissions between the lowest flow the Methane Emissions Technology Evaluation Center (METEC) can produce (40 g CH₄ h⁻¹) and the highest flow we feel comfortable measuring with these methods (200 g CH₄ h⁻¹). Putting these emission ranges into a real-world context, the maximum emission from unplugged and abandoned wells was measured at 177 g CH₄ h⁻¹ in West Virginia (Riddick et al., 2019a), 175 g CH₄ h⁻¹ in Pennsylvania (Pekney et al., 2018), 146 g CH₄ h⁻¹ across the U.S. (Townsend-Small et al., 2016), and 35 g CH₄ h⁻¹ in the UK (Boothroyd et al., 2016). As most of the methods presented here require access to the source, we considered 200 g CH₄ h⁻¹ to be a sensible limit to the emission rate, and it is larger than the emissions observed by many previous studies. Therefore, the scope of this study is limited to estimating CH₄ emissions from a single point source that we would realistically be able to approach and measure, i.e., between 40 and 200 g CH₄ h⁻¹.

The study compares each method's accuracy against known emission rates. Explicitly, our objectives are to (1) reproduce the experimental design for each of the measurement methods, (2) conduct repeat measurements, as a researcher would do in the field, by taking measurements to generate an emission estimate from a point source and compare this to known emission rate, and (3) make recommendations on the suitability of each method for measuring emissions from relatively small point sources. We add the caveat that we will only present data from measurement methodologies conducted safely when wearing personal protective equipment (PPE), as regulated at the Colorado State University Methane Emissions Technology Evaluation Center (METEC) facility in Fort Collins, CO, USA (steel-toe boots, flame resistant (FR) overalls, a hard hat, safety glasses, and four-gas monitor). To our knowledge, this is the first time that methods for measuring CH₄ emissions from point sources between 40 and 200 g CH₄ h⁻¹ have been quantitatively assessed against a known reference source and against each other.

2 Methods

Each of the methods, namely static chambers, dynamic chambers, BHFS, backward Lagrangian stochastic (bLs), and Gaussian plume (GP), are tested at METEC in Fort Collins, CO, USA. METEC can reproduce the range of CH₄ emissions typically seen from individual point sources at oil and gas operations, i.e., between 20 g CH₄ h⁻¹ and 40 kg CH₄ h⁻¹, from realistic locations with oil and gas equipment. At the METEC site, compressed natural gas, with methane compositions ranging from 85 % vol to 95 % vol, is supplied from two 145 L cylinders, and flow rates are controlled using a pressure regulator and precision orifices. At METEC, the methane content of the natural gas in each release is measured by gas chromatography and accounted for in the known emission rate. For the purposes of this study, where we are comparing the ability of each method to estimate the emission from a point source, we will constrain the known emission rates to those that can be measured safely, i.e., between 40 and 200 g CH₄ h⁻¹. To accomplish this, CH₄ emission rates will be set from a point source (diameter 6 mm) at 20 cm above the ground at 40, 100 and 200 g CH₄ h⁻¹.

Two instruments are used to report CH₄ mixing ratios, i.e., the Picarro (http://www.picarro.com, last access: 26 October 2022) GasScouter G4301 mobile gas concentration analyzer and the Agilent (https://www.aglient.com​​​​​​​, last access: 26 October 2022) 7890B gas chromatograph flame ionization detector (GC-FID). The Picarro GasScouter reports CO₂, H₂O, and CH₄ mixing ratios every 3 s, with a precision (300 s; 1σ) for CH₄ of 300 ppb (parts per billion) over an operating range of 0 to 800 ppm (parts per million). The Agilent 7890B GC-FID, as used here, has a detection limit of 1.5 ppb and linear dynamic range from 1 ppm to 100 % CH₄. The instrument was calibrated every 10 samples using a 5000 ppm CH₄ gas standard (accuracy of standard ±5 %). The GC-FID was checked for linearity before and after each set of measurements using zero-air, 5000 ppm, 2.5 %, and 100 % CH₄.

2.1 Static chamber

For the static chamber method, a container of a known volume (V; m³) is placed over the emission source, and the change in concentration (C; g m⁻³) inside the container over time (t; s) can be used to calculate the emission (Q; g s⁻¹; Eq. 1). The static chamber method requires no power and is very portable. The major shortcoming of this method is that large emission sources can result in the concentration inside the chamber exceeding the CH₄ lower explosive limit (LEL).

$$\begin{array}{}\text{(1)}& Q={\displaystyle \frac{\mathrm{d}C}{\mathrm{d}t}}V.\end{array}$$

Following method descriptions presented in Kang et al. (2014), the static chamber is made by enclosing air within a fixed volume over the emission source (Fig. 1a). The chamber was constructed of two parts, with a smaller lower part that was secured 4 cm into the soil and a larger upper part that was fixed to the lower part at the start of the experiment. A fan was secured inside the chamber and used to circulate the air to ensure the air inside the chamber was fully mixed (Kang et al., 2014, 2016). As the experiment was conducted at METEC, 120 V mains power was used; however, in remote locations, power can be supplied by anything capable of delivering a stable 12 V output (e.g., battery). When the chamber is sealed with the ground, following Kang et al. (2014, 2016), an air sample is drawn using a gas syringe. During the experiment, air samples are taken at regular intervals, with the time interval pre-calculated, depending on the emission rate, to ensure the increase in concentration was linear. The emission is then calculated from the linear increase in concentration over time.

Figure 1 Schematics of the (a) static chamber and (b) dynamic flux chamber.

Two sizes of static chambers were used in this experiment (0.12 and 0.5 m³; Fig. 1), and the chambers were made from rigid plastic cylindrical chambers, with heights approximately 1.5 times the chamber's diameter. The chamber sizes were based on a measurable concentration change over time for given release rates; however, it is unlikely that the larger size is practical for field deployment. During any wind, the chamber acted as a sail, and the larger (0.5 m³) chamber lifted from the ground. Therefore, smaller chambers are better in the windy conditions but quickly fill with gas, making quantification difficult as the change in CH₄ concentration inside the chamber quickly becomes nonlinear. In each case, the chamber was placed over a point source 20 cm above the ground, emitting gas at approximately 40, 100, and 200 g CH₄ h⁻¹. During the experiment, four samples of 25 mL of air were drawn from the chamber using a 50 mL gas syringe at equal time intervals (Pihlatie et al., 2013; Collier et al., 2014). The air samples were injected into glass vials containing 30 mL of nitrogen and then stored in a fridge before the CH₄ concentrations were measured using the GC-FID. All samples were measured within 2 h of collection. All experiments were repeated three times.

The minimum time between air sampling was set at 1 min to ensure that the correct vial could be found, and the sample outlet was purged of gas. When sampling times were less, the experiment became too rushed and errors occurred. Additionally, as a health and safety precaution, a handheld CH₄ sensor, HXG-2d (SENSIT Technologies, USA; http://www.gasleaksensors.com, last access: 26 October 2022; detection limit of 10 ppm and range from 0 to 40 000 ppm), was placed in the chamber, and if the CH₄ concentration exceeded the lower explosive limit before three samples were taken, then the test was abandoned.

2.2 Dynamic chamber

To address the LEL issues inside the chamber, a dynamic flux method has also been used to measure CH₄ leakage from abandoned and active oil and gas wells (Riddick et al., 2019a). Like the static chamber, the dynamic chamber comprises of a container (0.12 m³) enclosing the source, and a propeller is used to circulate the air. Additionally, a flow of air is passed through the chamber, which reduces the likelihood of exceeding LEL inside the chamber. Unlike the static chamber, the CH₄ concentration becomes stable after a period of time, depending on the source emission rate. When the chamber reached steady state, three air samples were taken from inside the chamber. A background air sample was taken outside the chamber as the chamber approached steady state. The methane concentration in all air samples was measured using a gas chromatograph. The CH₄ flux (Q; g s⁻¹) is calculated (Eq. 2) from the CH₄ concentration at steady state (C_eq; g m⁻³), the background CH₄ concentration (C_b; g m⁻³) in the air used to flush the chamber, the height of the chamber (h; m), the flow of air through the chamber (q; m³ s⁻¹), the footprint of the chamber (a; m²), and the volume of the chamber (V; m³; Aneja et al., 2006; Riddick et al., 2019a). As well as improving the safety, the dynamic chamber reduces the theoretical uncertainty in emission rate to ±7 % (Riddick et al., 2019a); however, the added power requirement of a pump means the dynamic chamber is less portable than the static chamber. Methane emissions from abandoned wells have been quantified using this method between 4 and 100 g CH₄ h⁻¹ (Riddick et al., 2019a).

$$\begin{array}{}\text{(2)}& Q={\displaystyle \frac{\left(C_{\mathrm{eq}}-C_{\mathrm{b}}\right)\cdot h\cdot q\cdot a}{V}}.\end{array}$$

A single chamber of 0.12 m³ was used for testing the dynamic chamber method. The plastic chamber, open at one end, was placed over known leaks of approximately 40, 100, and 200 g CH₄ h⁻¹, and air was passed through the chamber at a constant rate of 67 L min⁻¹, following the method of Riddick et al. (2019a). As the experiment was conducted at METEC, 120 V mains power was used; however, in a remote location power, can be supplied by anything capable of delivering a stable 12 V output. The chamber was left until the CH₄ concentration inside had become constant, as measured by a SENSIT HXG-2d sensor (SENSIT Technologies, Valparaiso, IN, USA). When steady state was reached, three samples of 25 mL of air were drawn from the chamber using a 50 mL gas syringe injected into glass vials containing 30 mL of nitrogen. As with the samples from the static chamber, the vials were measured within 2 h of collection. All experiments were repeated three times. Following the methods of Aneja et al. (2006) and Riddick et al. (2019a, 2020b), the emission rate is calculated from the steady-state gas concentration using Eq. (2).

2.3 Bacharach Hi Flow Sampler (BHFS)

Another way of addressing the issue of enclosing methane at concentrations approaching LEL is to use a BHFS. A BHFS draws high volumes of air into a measurement chamber at a fixed rate (F; m³ s⁻¹), and the background CH₄ concentration (X_b; g m⁻³) and the concentration of CH₄ in the air are measured (X_s, g m⁻³) and used to calculate the emission rate (Q, g s⁻¹; Eq. 3). The Bacharach Hi Flow Sampler (Bacharach, Inc., Pittsburgh, USA; http://www.mybacharach.com, last access: 26 October 2022) is the only currently available BHFS and was used in this study. It draws air at between 226 and 297 L min⁻¹ and can measure CH₄ emissions between 50 g CH₄ h⁻¹ and 9 kg CH₄ h⁻¹ to an accuracy of ±10 % (Connolly et al., 2019). A recent study commissioned by the California Air Resources Board developed open-source architecture for a new BHFS unit which is capable of replacing the current Bacharach Hi Flow Sampler (Vaughn et al., 2022).

$$\begin{array}{}\text{(3)}& Q=F\left(X_{\mathrm{s}}-X_{\mathrm{b}}\right).\end{array}$$

As the BHFS method is relatively simple, no data are required, other than the direct measurements made by the instrument. Following the methods of Pekney et al. (2018), the bag containing the hose end of the BHFS was placed over the point source, and the instrument was turned on. This was repeated three times, and the average emission calculated. The BHFS used in this study was calibrated monthly as recommended by the manufacturer.

2.4 Gaussian plume

In some circumstances, access and safety restrictions mean that direct measurements are impossible, and an observer must use a far-field method to measure the emissions remotely. The most widely used of these far-field approaches is the Gaussian plume (GP) model. First used in the 1940s, a GP model describes the concentration of a gas as a function of distance downwind from a point source (Seinfeld and Pandis, 2016). When a gas is emitted from the source, it is entrained in the prevailing ambient airflow and disperses laterally and vertically with time, forming a dispersed concentration cone. The concentration enhancement of the gas (X; µg m⁻³), at any point x m downwind of the source, y m laterally from the center line of the plume, and z m above ground level can be calculated (Eq. 4) using the emission rate (Q; g s⁻¹), the height of the source (h_s; m), and the Pasquill–Gifford stability class (PGSC) as a measure of air stability. The standard deviation of the lateral (σ_y; m) and vertical (σ_z; m) mixing ratio distributions are calculated from the PGSC of the air (Pasquill, 1962; Busse and Zimmerman, 1973; US EPA, 1995). The GP model assumes that the vertical eddy diffusivity and wind speed are constant, and there is total reflection of CH₄ at the surface, where gas reflected from the surface of the Earth is accounted for in the downwind plume. The enhancement is defined as the difference between the downwind concentration and the background concentration measured upwind. The GP is the simplest of the far-field methods considered here and assumes that the emissions are well-defined plumes injected above the near-surface turbulent layer from point sources and not affected by aerodynamic obstructions that cause mechanical turbulence at the surface. However, in most situations there are aerodynamic obstacles, and plumes are rarely perfectly Gaussian in shape. Another shortcoming of the GP model is the parameterization of the PGSC, which leads to discrete values, and incorrectly assigning these values can lead to significant uncertainty. Generally speaking, the GP is rarely used for emissions less than 100 g CH₄ h⁻¹. However, an example of using a GP model is its use in estimating CH₄ emissions from oil production platforms in the North Sea, where emissions ranged from 10 to 80 kg CH₄ h⁻¹ with an uncertainty of ±45 % (Riddick et al., 2019b).

$$\begin{array}{}\text{(4)}& X\left(x,y,z\right)={\displaystyle \frac{Q}{\mathrm{2}\mathit{\pi }u{\mathit{\sigma }}_y{\mathit{\sigma }}_z}}{e}^{-\frac{y^{\mathrm{2}}}{{\left(\mathrm{2}{\mathit{\sigma }}_y\right)}^{\mathrm{2}}}}\left(e^{-\frac{{\left(z-h_{\mathrm{s}}\right)}^{\mathrm{2}}}{{\left(\mathrm{2}{\mathit{\sigma }}_z\right)}^{\mathrm{2}}}}+e^{-\frac{{\left(z+h_{\mathrm{s}}\right)}^{\mathrm{2}}}{{\left(\mathrm{2}{\mathit{\sigma }}_z\right)}^{\mathrm{2}}}}\right).\end{array}$$

The GP model uses a downwind measurement coupled with meteorology to estimate the emission rate of a source using Eq. (4). Explicitly, the data used are wind speed (u; m s⁻¹), wind direction (WD; ^∘), temperature (T; ^∘C), CH₄ concentration downwind of the source (X; µg m⁻³), location and height of the CH₄ detector, background CH₄ concentration (X_b; µg m⁻³), and the PGSC. The PGSC can either be calculated using the wind speed and a measure of the solar irradiance (Sect. S1 and Table S1 in the Supplement) or using a sonic anemometer. Due to power requirements, sonic anemometers are unlikely to be used in the field, and as such, a more basic approach is adopted, and the PGSC calculated from the wind speed (u; m s⁻¹) measured at 1.2 m and irradiance measured at the emission point (G; kW m⁻²). Pasquill and Smith (1983) originally defined strong irradiance as a sunny midday in midsummer in England and slight insolation to similar conditions in midwinter. Here we classify strong irradiance as > 1 kW m⁻², moderate irradiance as 0.5 to 1 kW m⁻², and light irradiance as > 0.5 kW m⁻² (Riddick et al., 2022).

Methane emissions are calculated using CH₄ concentrations measured 1.5 m above ground level, 5 m downwind, and background CH₄ concentrations 5 m upwind of the source by the Picarro GasScouter. Here, it assumed that the experiments are conducted as close as possible to the source (between 1 and 10 m) without direct access to the emission point. Wind speed and wind direction were measured every 10 s using a Kestrel 5500 weather meter (http://www.kestrelmeters.com, last access: 26 October 2022) on a mast 2 m above the ground. To reduce any impact of mechanical turbulence while maintaining real changes to CH₄ emission caused by changing environmental or atmospheric factors, both CH₄ concentrations and meteorological data are averaged over 15 min (Laubach et al., 2008; Flesch et al., 2009). The PGSC was calculated from the meteorological data using the method of Seinfeld and Pandis (2016). The lookup table, Table S1, is presented in Sect. S1. Complex topography, such as building and trees, are not parameterized or accounted for by the GP model.

2.5 bLs dispersion model point measurements

As an alternative to the GP model, Lagrangian dispersion models can be used to calculate the emission of a source. In a backward Lagrangian stochastic (bLs) model, the measurement position, gas concentration, meteorology, and micrometeorology are known inputs, and the model works iteratively backwards to simulate the motion of the air parcel. This is then used to infer the rate of emission from the source (Flesch et al., 1995). For given meteorological conditions, the model calculates the ratio of downwind concentration to emission, $(C/Q{)}_{\mathrm{sim}}$, depending on the size and location of the source. The emission rate (Q; g m⁻² s⁻¹) is then inferred from the measured gas concentration at 1.2 m above ground level (X_m; g m⁻³) and the background gas concentration (X_b; g m⁻³; Eq. 5). The bLs models can be used to calculate the emissions from point or area sources in a range of micrometeorological conditions. However, a major shortcoming of the model is its inability to adequately model emissions from sources with complex topography or near large objects, such as buildings. This can be mitigated by measuring far away from the source over a relatively flat fetch, but an accurate measurement of the micrometeorology is required. As an example, CH₄ emissions from individual point sources on oil and gas infrastructure have been estimated using a bLs model between 4 µg CH₄ h⁻¹ and 3 kg CH₄ h⁻¹, with an uncertainty of ±38 % (Riddick et al., 2019a).

$$\begin{array}{}\text{(5)}& Q={\displaystyle \frac{X_{\mathrm{m}}-X_{\mathrm{b}}}{{\left(\frac{C}{Q}\right)}_{\mathrm{sim}}}}.\end{array}$$

WindTrax (http://www.thunderbeachscientific.com, last access: 26 October 2022), a commercial software program, uses a bLs dispersion model to calculate the rate of gas emission from a point, area, or line source. In this application, the inversion function of the WindTrax inverse dispersion model version 2.0 was used (Flesch et al., 1995). Data used as input are wind speed (u; m s⁻¹), wind direction (WD; ^∘), temperature (T; ^∘C), downwind CH₄ concentration (X; µg m⁻³), location and height of the CH₄ detector, background CH₄ concentration (X_b; µg m⁻³), the roughness length (z₀; m), and the Pasquill–Gifford stability class. The ideal terrain for WindTrax modeling is an obstruction-free surface (Sommer et al., 2005; Laubach et al., 2008), with the maximum distance between the source and the detector of 1 km (Flesch et al., 2005, 2009). The roughness length was set at 2.3 cm to represent the short grass of the fetch. Again, it assumed that the experiments are conducted as close as possible to the source without direct access to the emission point. Data for downwind average CH₄ concentration, background CH₄ concentration, meteorological, and micrometeorological data used in WindTrax will be the same as that described in Sect. 2.4.

2.6 Measures of accuracy and precision

In each individual experiment, the difference between the known emission rate and the calculated emission rate will be presented as a percentage (Eq. 6), where A is the accuracy, Q_c is the calculated emission, and Q_k is the known emission. The average accuracy of the three experiments (A_r; %) will be presented as a measure of the accuracy, and the standard deviation (A_SD) of the individual uncertainties will be used as a comparative measure of the precision.

$$\begin{array}{}\text{(6)}& A={\displaystyle \frac{\left(Q_{\mathrm{c}}-Q_{\mathrm{k}}\right)}{Q_{\mathrm{k}}}}\times \mathrm{100}.\end{array}$$

3 Results

3.1 Method narrative – qualitative observations of methods

The static chamber is fixed around an emission source and extracts air samples at known time intervals. These vials can be stored for up to a month before analysis on a gas chromatograph. As such, the samples can be analyzed by a third party, and the researcher only requires access to the flux chamber, LEL sensor, and the required gas sampling equipment. We found the main shortcomings of the static chamber method are that (1) it was difficult to take samples fast enough during the linear change in concentration, and (2) the method is inherently dangerous, as we were unable to remove the chamber without the four-gas monitor, worn on the observer's collar, detecting CH₄ concentrations that exceeded the lower explosive limit, i.e., it triggered the monitor's alarm.

To address the first shortcoming, a trace gas analyzer could be used to measure the concentrations inside the chamber. As trace gas analyzers use a pump to draw air into the measurement cavity, the analyzer could be arranged in one of two ways. Both introduce additional uncertainty into the quantification. If the gas is removed from the chamber (i.e., the air from the analyzer exhaust is actively pushed outside the chamber), then the static chamber becomes a dynamic chamber, and the analyzer flow rate must be accounted for in the quantification. If the measured gas is reintroduced to the chamber (i.e., the analyzer outlet is vented back to the chamber), then a gas of lower concentration is being continually added to the closed system, and it is therefore unclear how much uncertainty is caused by this cycling. Furthermore, the linear response of a portable trace gas analyzer, e.g., the ABB GLA131-GGA Greenhouse Gas Analyzer (https://new.abb.com/, last access: 26 October 2022), is 100 ppm. Using the lowest emission rate in the study, 40 g CH₄ h⁻¹, and the largest chamber, 0.5 m³, the concentration inside the chamber will exceed the linear range within 7 s. Another alternative could be using a lower precision sensor with a larger detection range, such as the SGX INIR-ME100% (https://sgx.cdistore.com/, last access: 26 October 2022), that can measure from 200 ppm to 100 % methane by volume (bv), but safety issues remain.

We were aware throughout the experiment that the chamber will become explosive and pre-calculated the time between sample measurement based on the emission rate. During the 200 g CH₄ h⁻¹ experiment, the lower explosive limit of CH₄ was reached after 3 min of the chamber being sealed. As such, we have not presented the measurement data collected during the static chamber experiments and strongly encourage the use of an alternative method. The static chamber could be automated to release gas when the CH₄ concentration inside the chamber approaches LEL to prevent chamber becoming explosive. The major shortcoming of this strategy is that the automation of a chamber takes away the operator's control of when the gas is released, which could happen at an inconvenient time during measurement. If an automated system is used for collecting gas of unknown composition, then self-contained breathing apparatus should be worn.

The dynamic chamber is logistically one step more advanced than the static chamber and requires a pump to draw air through the chamber at a known rate, and, ideally, a flow meter to measure the airflow. This reduces the potential for CH₄ concentration inside the chamber becoming explosive. This means the main advantages of the static chamber are preserved, i.e., cost and ease of analysis but mitigates the health and safety concerns. Again, the major shortcoming of the dynamic chamber method is that it requires direct access to the emission source and a 12 V power source for the pump. Another factor that could affect accuracy of measurement is the air being pumped into the chamber, care should be taken to ensure the inlet is apart from other CH₄ sources and far away from the chamber outlet.

The BHFS is an off-the-shelf method/instrument and, as an integrated solution, is easier than the dynamic chamber. Once calibrated, the BHFS bag is loosely cinched around the emission source and turned on. The instrument displays the methane emission, in liters per minute (L min⁻¹), within a minute, at a precision of one significant figure. The data are stored in the instrument and can be downloaded later. The advantages of the BHFS are the ease of use and amount of time needed to measure a source, which is typically 5 min per emission source. The main shortcomings are that the researcher needs to have a BHFS instrument (costs USD 35 000), direct access to the source, calibration gas, and a means of charging batteries and/or powering the instrument.

Measurement data required for the GP and bLs methods were the same. After CH₄ is emitted from a source, it quickly disperses, and in order to measure the concentration downwind, access to a sub-parts-per-million CH₄ analyzer is required. In 2020, the least expensive, suitable, instrument on the market cost around USD 32 000. In addition to near-ambient CH₄ concentration measurements, meteorological data are required to populate the models. Despite the cost and time required to make the measurements, the practical advantages of these methods are that access is not required, and emissions can be calculated from remote sources. However, ensuring that the measurement location is in the plume for long enough to detect an enhancement large enough for the instrument to measure accurately can be challenging. In light winds, the plume can move laterally, and the sensor becomes offset.

3.2 Accuracy and precision of repeat measurements

Static chamber results are not presented, as we were unable to remove the chamber without exposing the observer to an explosive environment. Our results show that the most accurate method for generating emissions after repeat measurements from a 200 g CH₄ h⁻¹ source was the bLs method (−7 %), then the dynamic chamber (−10 %), and then the BHFS (−18 %; Table 1). The least accurate method after repeat measurements was the GP model (29 %). Repeating the experiments improved the accuracy of the emission estimate by 4 % for the GP model. Data are all presented in Sect. S3. For the 40 g CH₄ h⁻¹ source, repeating the experiments generally improved the accuracy of the emission estimate, except for the GP model which became 20 % less accurate (Fig. 2a). Like the accuracy, the precision of the methods became better, i.e., the standard deviation (SD) of the individual uncertainties became smaller, as the emission rate of the source increased (Fig. 2b). Methods that made measurements while being attached to the source – chamber and BHFS methods – were more precise than those that measured remotely – bLs and GP methods.

Table 1 Condensed description of logistical needs and results of each experiment. Access describes if physical access to the emission source is required (Y denotes having permission to touch/enclose the emission point and N denotes experiments are conducted as close as possible to the source without direct access), Inst describes if a dedicated instrument is required, and Cost is the approximate cost of the lowest price instrument capable of the measurements. Met describes if meteorological data are required. T_meas and T_analysis are the times it takes to conduct and analyze one measurement, respectively. A is the accuracy of one measurement of a 200 g CH₄ h⁻¹ source (as defined above in Sect. 2.6), A_r is the average accuracy when repeating the measurement of a 200 g CH₄ h⁻¹ source three times, A_SD is the standard deviation of the accuracy of the three repeated experiments, and U is the theoretical uncertainty as presented in previous studies.

^a Riddick et al. (2019a), ^b Pekney et al. (2018), ^c Riddick et al. (2020a), ^d Riddick et al. (2016). ^e Cost of sample analysis by GC will vary by laboratory. The static chamber data are not presented, as the method was found to be inherently dangerous.

Figure 2 (a) Average accuracy (percent difference from a known emission rate) of emission estimates from three repeat measurements using each of the measurement methodologies at different known emission rates (∼ 40, 100, and 200 g CH₄ h⁻¹). (b) The standard deviation of the uncertainties in repeated measurements against the emission rate of the experiment. Abbreviations are as follows: DC is the dynamic chamber, HF is the hi flow, GP is the Gaussian plume, and bLs is the backwards Lagrangian stochastic method.

4 Discussion

This study investigates the utility, accuracy, and precision of five methods that have recently been used to estimate smaller, < 200 g CH₄ h⁻¹, CH₄ emissions from oil and gas infrastructure and include the static chamber, dynamic chamber, the Bacharach Hi Flow Sampler, Gaussian plume modeling, and backward Lagrangian stochastic models. When the method has been shown to be of no danger to the observer, we generate CH₄ emission estimates from a known CH₄ source emitting approximately 40, 100, and 200 g CH₄ h⁻¹. Experiments simulating published methods are carried out once to generate a single visit estimate and are then repeated twice more to better understand how repeat experiments can improve the accuracy and precision of the emission estimate.

The static chamber method was found to be inherently dangerous, as the observer was unable to remove the chamber without being exposed to an explosive environment. As a result, the data from the static chamber experiments have not been presented in this study. Furthermore, the experiment conducted at METEC used processed natural gas where heavier/aromatic hydrocarbons and toxic gases have been removed. Gas emitted from abandoned oil and gas wells is unrefined, and we advise that the static chamber method should not be used to quantify emissions of an unknown composition of natural gas, as this could expose the observer to high concentrations of toxic gas. Therefore, we recommend that one of the other methods presented here should be used to quantify emissions from abandoned oil and gas wells.

Both the dynamic chamber (A_r = −10 %, −8 %, and −10 % at emission rates of 40, 100, and 200 g CH₄ h⁻¹, respectively) and BHFS (A_r = −18 %, −16 %, and −18 %) repeatedly underestimate the emission, but the dynamic chamber is more accurate. For the far-field methods, the bLs method underestimated emissions (A_r = +6 %, −6 %, and −7 %) while the GP method significantly overestimated the emissions (A_r = +86 %, +57 %, and +29 %), despite using the same meteorological and concentration data as input. These findings are consistent with another study (Bonifacio et al., 2013); however, this is the first study that has compared both to a known emission rate. In all cases, the accuracy in the emission estimate increased with emission rate apart from the BHFS. The Bacharach Hi Flow Sampler is designed to measure emissions from 50 g CH₄ h⁻¹ to 9 kg CH₄ h⁻¹ to an accuracy of ±10 %. All flow rates presented here are at the lowest range that the BHFS can measure, and it is likely that the uncertainty in the system sensors that measure between 40 and 400 g CH₄ h⁻¹ is of negligible difference.

The method that improves the most as the emission rate increases is the GP method, where accuracy increases from +87 % to +29 % as the emission rate increased from 40 to 200 g CH₄ h⁻¹. This improvement in emission is likely caused by the increased size of the plume and the ability of the GP model to parameterize the concentration at distances from the centerline of the plume. Although not explicitly stated, the parameterization of the lateral dispersion in the GP model is the same at 100 m as at 5 m, which is unlikely. Other controlled release experiments using the GP approach show similar uncertainties, and one experiment reported average emissions calculated using a GP model less than 20 % (release rates were not published), with the uncertainty mainly driven by atmospheric variability (Caulton et al., 2019). Another showed uncertainties of ±50 % for triplicate measurements of emissions between 90 and 970 g CH₄ h⁻¹ (Caulton et al., 2018).

Figure 3 (a) Individual uncertainty in Gaussian plume measurements at 40, 100, and 200 g CH₄ h⁻¹ and (b) individual uncertainties plotted against the horizontal distance from the plume center (m).

Data do not exist on controlled release experiments using a dynamic chamber. One study suggested a theoretical emissions uncertainty in the dynamic chamber approach of ±7 % (Riddick et al., 2019a), with the largest source of uncertainty caused by the measurement of the flow rate of air through the chamber. Other sources of uncertainty for the dynamic chamber methods are relatively negligible as the methane quantification of the background gas and the gas at steady state (assuming complete mixing of the gas in the chamber) using the GC is highly accurate over a large concentration range, and the volume of the chamber is fixed by a plastic structure.

A controlled release has been conducted for the bLs model but only for an emission from an area source (Ro et al., 2011) at the surface and is not analogous to the emissions of this study. Ro et al. (2011) estimated the bLs uncertainty at ±25 % for a gas emitted at an unspecified rate from a 27 m² emission area. As with the GP approach, the bLs model's main source uncertainty is the parameterization of the atmospheric stability (Riddick et al., 2012; Flesch et al., 1995; Ro et al., 2011). The main advantage of the bLs model over the GP at these short distances is that it calculates the lateral dispersion of gas for individual particles, while the GP uses an averaged dispersion parameter.

The emission estimates quantified using direct methods, i.e., the dynamic chamber and BHFS, have a lower SD than the far-field methods (Fig. 2b). The standard deviation of direct measurement methods remain relatively constant for emissions between 40 and 200 g CH₄ h⁻¹ and reflect the relative simplicity of the methods. Assuming all other parameters are measured correctly, for direct methods the variability in the emission estimate is a function of how well the CH₄ is mixed into the air in the chamber during the measurement.

Variability in the far-field emission estimates is much larger and reflects the relative complexity of inferring emissions. Variability in wind speed, wind direction, and atmospheric stability over the 20 min averaging period is likely to propagate through to large variability in the emission estimate. It may be reasonable to suggest that the variability in bLs calculated emissions is less than for the GP method because of the added parametrization available (roughness length and gas species). In summary, the penalty of downwind measurement is a higher uncertainty in individual measurements, but this appears to be corrected for by the bLs model through repeated measurements, where uncertainty is corrected for by the stochastic nature of particle movement modeling.

Regardless of accuracy and precision, this study shows that all methods can be used to estimate emissions from a source between 40 and 200 g CH₄ h⁻¹ to an accuracy of at least 40 %. It is reasonable to assume that this level of uncertainty is acceptable in some studies where the research is only aiming to determine relative sizes of emission (e.g., Riddick et al., 2019b), while other studies require time-resolved emission estimates to compare against modeled output (e.g., Riddick et al., 2017).

It is, however, concerning that many of the methods show a bias in measurement results, and this is particularly true of the GP model (Fig. 3). In most studies, it is assumed that, in taking multiple measurements, the average uncertainty will be reduced to an aggregate, unbiased emission estimate. Taking the GP emission estimates as an example, the individual calculated emissions are all overestimates of the true emission, therefore suggesting a fundamental shortcoming in the method (Fig. 3). These measurements were taken 4 d apart in similar environmental conditions (all PGSC C), with wind direction being the only difference between measurements, which can be seen from the correlation between the uncertainty and horizontal distance from plume center (Fig. 3b). As mentioned above, it is likely that this is due to the lateral dispersion in the GP approach being parameterized incorrectly, i.e., using values that were defined for distances of 100 m. This suggests that, when using the GP approach with a single measurement in the plume for distances less than 100 m, it is not correct to assume that repeat measurements will remove the bias in the calculated average emission. It is currently unclear if mobile, in situ measurements in and across the plume, even at distances shorter than 100 m, would give much better results.

It is also important to note that the study performed here did not simulate or account for issues which increase error in field conditions. For example, when using downwind methods (GP or bLs), the scientist may not know the exact location of the emission point, and it may be further downwind of the emission location. These knowledge errors may result in uncertainties or bias in excess of what is presented here; our study should be viewed as a best-case bound on the accuracy of the methods.

5 Conclusions

We find that both the dynamic chamber (A_r = −10 %, −8 %, and −10 % at emission rates of 40, 100, and 200 g CH₄ h⁻¹, respectively) and BHFS (A_r = −18 %, −16 %, and −18 %) repeatedly underestimate the emissions, but the dynamic chamber had better accuracy. The standard deviation of emissions from these direct measurement methods remained relatively constant for emissions between 40 and 200 g CH₄ h⁻¹. The static chamber data were not presented because of safety concerns during the experiments. For the far-field methods, the bLs method generally underestimated emissions (A_r = +6 %, −6 %, and −7 %), while the GP method significantly overestimated the emissions (A_r = +86 %, +57 %, and +29 %), despite using the same meteorological and concentration data as input. Variability in wind speed, wind direction, and atmospheric stability over the 20 min averaging period are likely to propagate through to large variability in the emission estimate, making these methods less precise than the direct measurement methods. Our results provide evidence to justify the selection of methods used to quantify emissions from abandoned oil and gas infrastructure on the basis of accuracy and precision and practical and economic considerations.