Review of “Development of a Forced Advection Sampling Technique (FAST) for Quantification of Methane Emissions from Orphaned Wells”

October 22^nd 2024

This paper describes a novel approach to a current hot topic in science.  The authors describe a new method to quantify methane emissions from orphaned oil and gas wells.   Controlled release experiments (~1 g CH₄ h^-1 to 40 g CH₄ h^-1) were used to tune the model and then were compared against another instrument’s data.  The FAST system then is deployed in the field to measure emissions from four orphaned wells in a real-world setting.  The paper is well written but overlong in many places and could benefit from some text being removed completely and extraneous figures moved to the SI.  One major concern I have is that the conclusion reads like a commercial advertisement for the FAST method and I would urge caution about overselling this method’s ability and making claims about the efficacy of other technologies/approaches that have been validated.

This study describes an in interesting idea, but I have many concerns about the methodology and the main conclusion that this method is a “low-cost, portable, fast and safe alternative to existing methods” is more subjective rather than what is shown by the data.  I do think that the FAST method could be used in special circumstances that other methods cannot, however, I am not encouraged to adopt the FAST method for abandoned well measurement.  My general concerns on the methods, results and conclusions are as follows with specific comments on the manuscript thereafter.

General comments

1. The main conclusion that the FAST method is lower-cost, more portable, faster and safer than existing methods appears to come from arbitrary comparison presented in Table 1. The data presented in Table 1 is at best misleading and in some cases incorrect. This table should be improved by using quantitative variables instead of the undefined “Accuracy”, “Size”, “Labor” and “Safety”.  For “Accuracy” you could use the “Uncertainty”; “Size” present the volume of the measurement equipment with appropriate units; “Labor” you could use time taken for 1 measurement (including set up and taking the measurement); and “Safety” a binomial of whether an operator is likely to be exposed to unprocessed natural gas.  Three key variables missing here are “Will it work in a complex aerodynamic environment (i.e. wells in woodland)” and “Does it account for undetected sources”, and “Does environmental variability affect precision/accuracy?”. 

Please also review the “Hardware” cost data as these are incorrect.  You can build a reasonably good static or dynamic chamber for $400 and the FAST system comprising of a Picarro GasScouter, Sonic anemometer and multiple laptops would cost >$50k (if the Picarro GasScouter was still available to buy).  The static/dynamic chambers were initially used to measure emissions from abandoned wells as the researchers did not have access to trace gas analyzers and were encouraged to develop lower cost/tech methods of quantification.

My expectations are that with a correct Table 1, the FAST method (as described in the methods section) would be relatively expensive, difficult to set up, difficult to transport but could be considered safer than many approaches, will work in complex aerodynamic environments, and could be used to quantify emissions from a larger piece of infrastructure that has many leaks.  This method could be used for measuring emissions from abandoned pump jack wells that won’t fit in a chamber and are surrounded by trees and suggest the authors focus on realistically evaluating the FAST systems performance and utility in the real world.

2. The assumptions about abandoned wells are very simplified. In your study, the controlled releases assume that all emissions from abandoned wells are single, continuous point sources of between 0.9 g CH₄ h^-1 and 40 g CH₄ h^-1 in the middle of a grass field. This is not the case.  I would have expected the controlled release experiments to simulate many more emission scenarios across the full range of emission rates.  The largest reported emission from and abandoned well was 70 kg CH₄ h^-1.  What is the lower limit of quantification of the FAST method, what is the highest?  What were the meteorological/micrometeorological conditions and did these affect quantification?  How is the experimental set up in Figure 8 different from what was encountered in the field (Figure 12)?  Many things are missing from if this is to be considered a validated quantification approach.  It would be acceptable to say that “The FAST method has been tuned using single, continuous point sources between 0.9 and 37 g CH₄ h^-1 at 1 m above the ground, in a simple aerodynamic landscape (grass field), in X, Y and Z meteorological conditions, and in low cross-wind conditions (< X m/s).”

3. The controlled experiments do not validate the approach, but instead are used to generate the fan specific value for K_FAST. The way data are presented is a little misleading as the emission rates calculated (in Table 3) using the FAST approach are presented before the calculation of K_FAST (Figure 11). The emission rates in Table 3 presumably use the calculated value for K_FAST.  This (K_FAST), and all other data used to calculate the FAST emission rates, should be clearly stated in Table 3 and include values for all variables in Equations 3 and 4.  As the controlled release experiments were used to generate one of the key variables for the FAST approach, it cannot be claimed that the method has been validated against controlled release experiments.  This would have added much more to credibility to the FAST approach and give more confidence in the emission rates calculated in real world settings, especially given the claims in the conclusion.

4. The results of the “Hooper #41” well are very telling. This is a well that looks like it has been capped and cut near the surface and is leaking, there are many of these across the country. From Figure 12, it is very likely that the fan is moving the air above the well head towards the sensor inlet and it is unlikely that all emitted gas will be entrained in the air flow.  I suspect this could result in lower concentrations being detected and lead to an underestimation of emission (as is shown in Section 2.3.2.2 and Figure 16).  This is strongly suggested by the data from the SEMTECH.

This suggests that there could be a major shortcoming in the methodology, i.e. when the fan does not blow gas from the point of emission to the analyzer inlet.  This makes measuring abandoned wells that are effectively holes in ground very difficult.  The confirmation of this could be proven using controlled release experiments at different heights (down to 0 m AGL), I do not expect you to do this, but this key shortcoming should be discussed in the paper.

5. The assumption that σ₀, as defined in Halloran et al (2014), can be used in this context is another key assumption of the approach. Halloran et al. (2014) used smoking oil to generate their results while this study is modelling the dispersion of methane emitted at pressure through a small aperture. Some comment of this is warranted in the discussion.

Specific comments

Abstract

L12: This is the definition for abandoned wells.  Not sure how an environmental challenge can be both “significant” and “poorly understood”.

L14: As stated above the FAST method doesn’t really improve on current methods, it provides an alternative measurement method.

L15: Is the flow of air a “jet”? 

L20:  Conclusion isn’t well supported by the rest of the paper.  Caveats to the FAST method’s performance should be stated here.

Introduction

L24: Repeat of L12.  Same issues.

L26:  EPA estimates there are ~3.5 million abandoned wells in the US.

L27: “imagined” sounds like the wrong word.

L30: Can you explain methane is an issue.

L31-33: Lines starting “Furthermore…”, who is underestimating emission and by how much.  Is this total national emission estimates?  If so, how does this relate to the total emissions from the country?  If there is a large underestimation from a large source, this is a problem.  If the underestimation is from a small source, it is likely to be in the noise of the total national emission and not a significant national problem.  Need to be careful here about potentially inflating the size of the problem caused by orphaned wells.  Are there any papers out there that suggest total regional emissions from abandoned wells are a representative proportion of total regional emissions?

L33: “measuring” instead of “plugging”?

L36: Could add “measuring and plugging orphaned wells to achieve climate goals.”

L41: Why such the large range?  Did this change over time or is it regional?

L51: What proportion of wells are > 1 kg/h?

L52: Double “]]”

L53-55: Not strictly true, Bridger Photonics claim a lower quantification limit of 1 kg/h.  If all the wells emitting > 1 kg/h were plugged, what fraction of total emissions would be removed.  This is the argument for all the remote sensing companies, get rid of the long-tail and the problem will be solved. 

L59-60” “Unmanned…” what vehicles and why are they promising?

L60: As mentioned above, the hardware costs in the table are incorrect.  This is a key selling point of the FAST method and it is based on flawed data.  Please revise the cost in the table and revisit the text.

L64-70: As detailed in general comment 1 above, the table is too qualitative and does not contain metrics that can be reasonably compared. This is key to why a reader would choose the FAST method over the others, currently the reasoning is flawed.

L83: Again, not sure “jet” is the correct word.

L83: The FAST method requires a sonic anemometer, therefore, it is measuring the atmospheric stability.  Why can this not be used in the GPM?

L86: Concluding line sentence should contain the caveats described in general comment 2.

L90-95:  This is incorrect.  There are many studies that have used a GPM to calculate emission from less than 1 km.  Some studies have even validated the use of the model using controlled releases and generated uncertainty bounds.  It would be useful to include these studies in your paper.

Methods

L145: Add location of manufacturer.

L146: Add detail of the duct blaster.  Why was this specific fan chosen?  Was it the same type as in Halloran et al?

L155-205:  These are all results and should not be in a methods section.  This section is too long and should be condensed to important outcomes:  High setting should be used; measurements should be taken along the plume center line; and measurements should be taken less than 2 m from the fan.  Could put all the details and extra figures in the SI?

L209:  Why was the SEMTECH Hi Flow used?

L234: Figure 7 should go in the results section, but it is actually unnecessary and should be removed or put in SI.

L242: Which MFC was used?  Add manufacturer and specs.

L243: Add manufacturer details.

L244: How far?

L255: Why test the FAST approach without a fan?

L260: A schematic would be better than a photograph.

L308-341:  This is all results, not the method.  Please move to the Results section.

Table 3: The uncertainties for the FAST method are very large +150%, -100% in some cases.  Are these correct?  Why present data with no fan, it’s not part of the method for the FAST approach and doesn’t add anything here.  Also, don’t need both Table 3 and Figure 10 – they are showing the same data.

L348-359: These are also results.  Please move to the correct section.

L377-384:  Superfluous detail, should be removed.

Figure 13: Not necessary in the main body of the paper.  Should be moved to SI.

L396: How was the leak detected?

L427-430: Superfluous detail, should be removed.

L435-339: Superfluous detail, should be removed.

Figure 14: These are results and should be in the correct section.  Also, why are the error bars much smaller than those presented in the controlled release experiments?

L451-466:  This measurement suggests fundamental flaws in the methodology.  See general comment 3 above.

Results

L480: A very doubtful assumption.

L505: The sentence starting “This shows…” is deeply flawed and should be rewritten.

L509-511: Please provide details of cost.  I am not convinced by the qualitative sentence.

L513-518: Please provide more details of which sensors could replace the Picarro at a lower cost.  The accuracy of the emission quantified by the FAST system are directly linked to the precision and accuracy of the analyzer taking the measurement, therefore, if the analyzer is replaced then the results presented in this paper will not be relevant to that system.

L522-525: This sentence is incorrect.

L528:  What I would like to see added here is the time taken to set up the FAST system and take one measurement as compared to the SEMTECH. 

Summary and Conclusion:  This section makes very bold statements about the FAST method and the limitations of other instrumentation, some of which has not been borne out by the results.  Care should be taken here on the strength of the claims and the section should be rewritten from a more balanced point of view.

L565: What I would like to see is an analysis done using GPM using the data from the “no fan” scenario.  How does this compare to the FAST method emission estimates?

Dubey et al. make measurements of methane emissions from orphaned wells using a new Forced Advection Sampling Technique (FAST).  This technique places a fan upwind of a suspected point source, which blows the emissions toward a measurement location.  Using measurements of the wind speed and some plume shape assumptions, they calculate an emissions rate.  Their technique is compared with results from a SEMTECH HI-FLOW backpack in a controlled release experiment, and then used to determine emission rates from 4 real-world orphaned wells.

This paper investigated a new technique for determining natural gas emissions from orphaned wells. As such, it is worthy of publication.  However, this reviewer would like to see some extra information in the discussion and results before I think it’s ready for publication.

Some discussion of the filter angle needs to be included in the methods section of the text.  It is hinted at in Figure 5, but never really discussed.  In Figure 4, the caption says data were filtered to remove points coming from the negative x direction (180 degrees).  How many points were these relative to the total?  Later, a filter of 300 degrees is used, but I don’t know what that means.  If 180 degrees is the negative x direction, I am guessing a crosswind be either 90 or 270 degrees?  In line 352, the authors state the crosswind is filtered as the filter angle approaches 360.  Is that the same as saying it approaches 0?  Or in line 356, when the authors state a filter angle of 300 filters out all crosswind interference, does this mean they are only looking at 60 to -60 degree wind directions? Or from 120 to -120?  Since so much of the discussion (Fig. 16, Table 4, etc.) depends on understanding this filtering, I don’t think I can give a full final review of this manuscript.

It seems like one of the bigger uncertainties is what Figure 6 would look like for concentration in the y-z plane, especially for cases like Hooper #41.  In fact, concentration or mixing ratio data are missing throughout the manuscript. Could the authors show concentration data for the reader to get an idea of how much variability and plume enhancement are involved in these calculations?

Other comments:

I’m suspicious that the Williams et al. (2021) conclusion that a 20% uncertainty in orphaned well emissions is most uncertain emission in the U.S.  Alvarez et al. (2018) estimated a 60% under-reporting from oil and natural gas production, for example.

Table 1, why is the safety so low for the SEMTECH compared to FAST?  I assume this is partly due to FAST diluting the plume quicker.

For Figures 2 and 3, is U defined in the x direction, or is it aligned to E-W? 

Figure 3, why is w so high?  Why wouldn’t one expect that to be 0? Add discussion

Figure 7, perhaps add the standard deviation of the emission rate to show “accuracy and precision … decrease” as stated in the caption.

Line 402, How do the authors account for changing upwind background in an oil field?  Is there variability in the background?  This is one instance where concentration data would be helpful to get a sense for what is being measured. For example, what is the signal to background variability? Can the authors show a time series of measurements?

Fig. 15, would the SEMTECH estimate really be negative? Or is this a problem with an improper Gaussian distribution assumption?

For Figure 1A, what is the blue dot?  Also, this graph seems to be limited by the resolution of the plume tests of Figure 6.  Wouldn’t one still expect to have a Gaussian plume within the resolution of 0.3 meters?

Lines 58–59, please spell out all acronyms.