Review of egusphere-2024-3161 “Evaluating the accuracy of downwind methods for quantifying point source emissions”, Mbua et al. (Reviewer Comments by E. Thoma and S. Ludwig)

The authors compare three approaches to quantify point source emissions (Gaussian, eddy covariance, and aerodynamic flux gradient methods). This manuscript is not recommended for publication in current form. Primary concerns center on the method application, transparency of the experiment conditions and data corrections, and quality assurance (QA). In addition to the comments provided here, the authors are encouraged to consider CC1.

(1)    It is assumed that this work was opportunistically executed as part of other controlled release activities performed by METEC to test, for example, fenceline sensor network leak detection capability. If this is true, this point should be made clear to the reader as this may represent a study limitation. The controlled release profiles are complex and a primary issue with this work resides in the QA of the individual release trails in the context of this comparison. Even when the mean wind direction during a 20-minute or 30-minute observation is acceptable, a short duration release event (as low as 10 seconds, line 148) producing a spatially underdeveloped plume may yield zero methane signal enhancement on the instruments, as the release could have occurred while winds were off axis. If there is insufficient sampling of the plume for a given trial, the emission calculation performance is irrelevant. A similar argument holds for the 2m and 4 m vertical positions. The work would be improved by adding additional detail on the controlled release trials along with methane concentration statistics (including background corrections) by trial for each instrument. Ideally this could occur on a uniform 20-minute time base so direct comparisons across the techniques are possible.

Overarching comments (2)-(10) focus on the non-standard application of the eddy covariance method, (11) relates to the application of aerodynamic flux gradient method, followed by specific comments related to all methods.

(2)    As it is currently written, there are issues with the eddy covariance data collection, pre-processing, flux calculations, post-processing, and footprint modeling that will cause erroneous results. These are detailed below: The authors use a non-standard system and instrument configuration for eddy covariance. These choices have significant consequences for flux quantification that need to be addressed and justified. 

(3)    The gas analyzer (MGGA) used here and 3D sonic anemometer are not designed to be logged together with clocks synced. Properly synced clocks at 10 Hz is an essential step and possible large source of error in data acquisition, and the authors need to address how they achieved this with their non-standard system. For example, LICOR and Campbell Scientific instrument systems are designed to log all fast data (10 Hz) simultaneously on the same data logger or microcomputer with PIP LAN networking to ensure clocks remain synced. 

(4)    The authors use a closed-path sampling system with a long (3 meter) tube. This choice creates significant errors that need to be addressed and corrected, if possible. Eddy covariance measurements require at least 10 Hz system response time in order to properly characterize turbulent eddies. Sampling at 10 Hz is not the same as a system response of 10 Hz. Closed path systems introduce a time-lag, laminar flow issues, and lack the response time of open-path systems. Slower system response is one of the main reasons the vast majority of methane eddy covariance is done using open-path sensors, and when closed-path sensors are used the system performance needs to be optimized and carefully justified. A well designed closed-path system with sufficient pump speed might effectively have a system response of 5 Hz while sampling at 10 Hz. This leads to an under estimation of fluxes, and in the best case scenarios can be corrected using transfer functions. At a minimum, the authors need to investigate the cospectra of their results to demonstrate they have a sufficient overall system response and properly apply transfer functions to correct their data. Related to this issue, when using a closed-path system the pump speed needs to be reported. With a longer tube such as this, a very fast pump speed, possibly also with a vacuum induced, is required in order to have a fast enough system response. Without a fast enough pump speed, the data cannot be used for eddy covariance methods. Pump speeds for closed path systems range from 8 l/m at normal pressure for short tubes, to over 100 l/m with partial vacuum for longer tubes.

(5)    The authors have not implemented numerous steps that are standard in the pre-processing of eddy covariance data. There are community standards for the QA/QC of the 10-20 Hz fast data before it is used for flux calculations. These include but are not limited to: Spike removal, absolute limits thresholding, corrections for skewness and kurtosis, dropouts removal, amplitude resolution, time lag corrections (absolutely critical for closed path systems such as this), steady-state tests and the mean removal through block averaging or detrending, and coordinate rotation or planar fit.

(6)    There are additional steps during the flux processing the authors do not implement. These include: high-pass and low-pass corrections, transfer functions (absolutely required given their system configuration), and WPL corrections. 

WPL corrections might not be necessary for a closed path system if the instrument is measuring methane, water, cell temperature, and cell pressure at synced 10 Hz rate. If this is the case, the reported dry mixing ratio of methane can be used without WPL corrections. Since it can be the case that methane and water are measured at 10 Hz but cell temperature and pressure are not, the authors need to report how their instrumentation sampling was configured in this regard. Without synced 10 Hz cell temperature and pressure, the dry mixing ratio of methane is not truly at 10 Hz, and instead the authors should use the non-dry gas measurement and apply WPL corrections.

(7)    There are post-processed QA/QC steps the authors have not implemented. These include:

•    Quality tests for developed turbulence and stationarity. These are standard across the eddy covariance scientific community.

•    Friction velocity thresholding. This eliminates low turbulence times and is assessed for each site/system.

•    Energy balance closure. A good energy balance closure does not necessarily mean good fluxes, but a bad energy balance closure can immediately identify issues with the sonic anemometer or instrument configuration.

(8)    Given the experimental design, any controlled releases with durations shorter than the averaging period used in eddy covariance flux calculations should not be used because they will by definition be non-stationary. The authors should consider using a shorter averaging period. 30 minutes is standard for ecosystem fluxes which are spatially homogeneous and change relatively slowly in time. Eddy covariance controlled release experiments commonly use shorter averaging periods in order to minimize non-stationarity issues. The authors should process their fluxes using 5 minutes, 10 minutes, and 15 minutes for example. They should then investigate the ogives to determine how much if any of the fluxes are underestimated by using a shorter averaging window. Previous controlled release experiments with eddy covariance have shown that the under sampling of large eddies by using a shorter averaging window affected flux estimates by <10% and considered this a worthwhile trade off to ensure stationarity in the observations.

(9)    The cospectra should be investigated and included in the SI. This is a standard QA/QC step that can illuminate issues such as slow system response times, aliasing, and interference near the sonic anemometer. A good cospectra will not necessarily mean good fluxes, but a bad cospectra will certainly mean there are issues that need to be corrected. 

(10)     Footprint modeling: This is an essential step in interpreting eddy covariance data in a system with point sources. However, the authors incorrectly parameterized the model and used the footprint area to normalize the fluxes to emissions incorrectly. Firstly, the choice of a roughness length of 1 is physically improbable for their site, and given it is a key parameter in the model, the footprints are likely incorrect. The roughness length should not be estimated or chosen randomly. It can be derived from the authors’ 3D sonic anemometer data under neutral conditions. In a scenario with perfectly homogeneous fluxes surrounding the eddy covariance tower, then the footprint area can be multiplied by the flux as the authors have done here. If there is any heterogeneity present (such as a point source) then this is incorrect. The footprint area is weighted with a small area of peak influence that asymptotically declines to zero influence in all directions away from it. The correct approach, given that the location of the point source is known, is to multiply the weight of the pixel representing the point source by the measured flux, and by the area of the pixel containing the point source. See Rey-Sanchez et al. 2022 for a description of how footprints should be used to calculate point source eddy covariance emissions and compared to controlled releases. Given that the authors can likely assume a zero flux for all non-point source pixels within the footprint, this is a straightforward calculation. Lastly, the authors chose to use a footprint model which previously studies have shown to perform the worst in controlled release experiments. Rey-Sanchez et al. 2022 compared three commonly used models and at a minimum, the authors here should do so as well. The exact footprint peak influence and location matters much more for correctly calculating emissions from point sources, so the variation in this result between footprint models is a source of uncertainty that should be quantified for a methods comparison paper such as this. 

(11)     It is unclear with the way the methods are written how the footprint normalization for the aerodynamic flux gradient method was done. It seems as though the area of the eddy covariance footprint was used to convert the AFG emissions into kg/hr. This is incorrect. The sensors are in a different location at different heights and will not have the same footprint as the eddy covariance measurements and will additionally have the same issues of normalizing to the whole area for a point source that was described above. In general, the AFG method should not be used to measure point source emissions. There will be a different footprint for each height along the profile. This means each sensor will see (or maybe not see at all) the point source differently. This will obviously lead to issues when calculating a flux from the profile gradient. The AFG method only works in homogeneous environments where the differing footprints at different heights don’t matter. 

------- General comments

(12)     Line 147 “The gas release rates ranged between 0.005 kg h-1 and 8.5 kg h-1, and the release durations ranged from 10 seconds to 8 hours, simulating both fugitive and large emission events.  The releases were run both during the day and night”.  A table in SI that describes the release experiments and the meteorological conditions to make this transparent to the reader would be very helpful. Development of a QA metric that summarized the methane concentration fields observed during each trial would provide important supporting information.  As currently written, the reader cannot understand if the under performance of the techniques is related to the method or to non-representative concentration fields or instrument factors.  

(13)     Extending comment 12, because the measurements were simultaneous and almost colocated, a direct comparison of the concentrations measured would be very useful. For example, at 2 m and 4 m on the same mast, what this the ratio of concentrations before and after background correction?  For proximate releases at ground level, there may be little signal at 4 m potentially invalidating method assumptions for those cases. At the eddy covariance unit position at 3 m mast height there should be some agreement in measured concentrations (for like time periods) with the other mast (perhaps the average of 2 and 4 m values). Concentration correlation plots by trial would inform both the degree of colocation of the slightly separated masts and the factors of instrument performance, apart from inverse modeling complexity. 

(14)     Line 125: Out of curiosity, why were the two masts not located together to facilitate direct comparison?  By the photos, this seems that it would have been an easy thing to do. 

(15)     Line 145: Basic information on the release trials should be included in SI.  For example, how many releases of each type were conducted. At what levels, duration, time of day/night, etc. 

(16)     Line 159-165: It is unclear why uniform 20-minute event tables were not used for al measurements to facilitate direct comparisons between the techniques for discrete experiments.  The authors make the argument that for two of the approaches, 15 min is not long enough, and 30 min may capture excess atmospheric variability but then just choose the default 30 min homogenous source eddy covariance default period without explanation. For this custom use of EC, 20-minute averages temporally aligned with the other approaches would be the preferred starting position.

(17)     Line 172: Why was this form of atmospheric stability estimate utilized for the inverse Gaussian approach instead of more robust and comparative estimates derived from high frequency sonic anemometry?  This stability estimate was developed to support a specific mobile source assessment approach designed to provide approximate emissions estimates with the derived stability index values limited to daytime measurement. How were nighttime data treated?

(18)     Line 192: The background subtraction procedure may represent a significant issue, or at minimum requires further clarification and supporting detail.  The authors reference a background correction procedure utilized for a specific mobile source assessment approach designed to provide approximate emissions estimates.  That background correction procedure is based on the lowest 5th percentile of a single 15-minute observation. The authors state “CH4 background was calculated as the average of the lowest 5th percentile of all continuous concentration readings (US EPA, 2013).”  If the authors combined all data for the series of experiments for this calculation, there are many issues with that approach related to drift and other factors. If the background for each 20/30 minute trial was determined, this needs to be more clearly stated. These data (the corrections) should ideally be provided as SI so the reader can better understand the fidelity of the concentration measurement itself. Currently, the concentration measurements are assumed to be accurate and precise with operation in challenging field conditions.

(19)     Related to comment (18), even if the background correction technique utilized was applied to each trial, it may be inappropriate for the inverse forms utilized.  This approach can be useful for single point measures of short duration that employ high precision instrumentation (e.g. CRDS).  However, is the ambient precision envelope is somewhat broad and if the calibration offset exhibits trend (drift), the background compensation for the 2m vs 4m units for the Aerodynamic Flux Gradient, for example, can have a large impact on the calculation. Currently the reader has no sense for the levels of these corrections or the basic stability and comparability of these instruments over time.

(20)     The introduction and discussion lack context and comparison to previous research. There are numerous controlled release experiments and other methods comparisons for these three approaches that the authors’ results should be put in context with. 

This manuscript compares three methods of point source emissions quantification (eddy covariance-EC, aerodynamic flux gradient method-AFGM, and the inverse Gaussian plume method-IGPM), investigating the performance of these methods in quantifying emissions for known gas release rates and evaluating uncertainties involved in the estimates. Although this is a timely topic, I recommend a revision to the current form of the manuscript before its publication. Here are my comments:

• Please provide a sufficient explanation of the three employed approaches. For instance, it is unclear how the inferred rates are estimated in Figures 4-6. Providing sufficient details about the different modeling choices can make significant improvements to the paper. For instance, the equation for the inverse Gaussian plume model (equation 7) only applies to a single source. It will be helpful to explain how this approach is used to model multiple release scenarios. Another example is the definition of the downwind distance ($x$) under such conditions.
• As part of this study, critical choices are made related to the physics embedded in the covariance w’c’ (equation 5) under different scenarios that an eddy covariance (EC) tower provides. More in-depth explanations of those choices will be helpful.
• Equations 1-4 and their associated text can be moved to Supplemental Information, as they provide more introductory content. I also suggest moving the Gaussian Plume Model (equation 7) along with detailing the dispersion coefficients 𝝈 and other model parameters in the text to the SI.
• While a detailed criticism of the assumptions embedded in the EC and other approaches is added in the discussion section by the authors, very little quantitative effort is made to confirm how several of these assumptions are being violated. For instance, when using the EC approach, no plots are provided, attempting to explain whether the local surface layer at the point of measurement is well mixed or not.
• The decision to use an averaging time window of 30 minutes remains suspect as there are many METEC experiments with durations shorter than half an hour. Reynolds averaging is a critical piece behind the use of eddy covariance concepts and a choice of 30 mins based on discussion from a prior article on agricultural emissions isn’t well justified.
• I recommend providing a reference dataset of the raw measurements along with a comprehensive validation exercise. This helps readers in the evaluation of the accuracy of the data reduction procedures and to appreciate the inverse modeling choices.
• I recommend a better flow of text and linkage between sections 4.1 and 4.2 (where authors refer to results) and section 3 to draw their conclusions.
• In dispersion modeling efforts, the modeler connects raw measurements with their understanding of the complex boundary layer physics and then uses this to judiciously fix the free parameters of the chosen model while acknowledging the underlying limitations of the constitutive equations all along. Additional explanations are appreciated to provide details on how authors tackled this challenge.
• Line 390: “Oil and gas point sources violate assumptions (1), (2) and (4)”. Assumption 2 requires the CH4 fluxes to be turbulent. If they are not turbulent at the point of measurement, are they laminar? I’d recommend that the authors confirm this by plotting the power spectrum of the CH4 concentration at the point of measurement and investigating the absence of the characteristic turbulence cascade in response. Our experience with continuous monitoring methane concentration data from METEC experiments referenced in the study unequivocally suggests that ambient concentration levels at measuring stations could be turbulent in nature.
• Assumption 4 states that measurements should be inside the boundary layer and in the constant stress layer. If the choice of RL from section 2.4.4 is to be taken at face value, then the constant stress layer extends up to 1m upward thus implying that measurements satisfy assumption 4.
• Line 395-398: “Even though current eddy covariance application assumes the vertical flux at a point is independent of atmospheric stability (Denmead, 2008), atmospheric stability has an impact on point source gas …”. While Denmaead (2008) notes that EC “is independent of atmospheric stability”, it merely implies that EC by virtue of being a more fundamental approach relies on direct measurement of turbulent fluxes like <u’w’> and <u’𝜃’>, i.e. the information on atmospheric stability is implicitly built into the EC approaches. Apriori estimates of the Obhukov length scale that divide the surface layer into multiple regimes as required by the AFGM and Gaussian models are therefore not needed. The EC does not imply that the vertical flux is independent of atmospheric stability.
• Line 384-385 & 411: “Positive fluxes represent emissions and downward fluxes represent absorptions”. This is not an assumption, but rather a statement of fact and an unnecessary one at that.
• Line 391: “Emissions are collimated plumes instead of turbulent fluxes”. It requires clarification as to what the authors mean when mentioning the phrase ‘turbulent fluxes’. Collimated plumes are simply the smoothing effect of time averaging on an underlying turbulent structure. Please see the following references
  • https://www.cambridge.org/highereducation/books/turbulent-flows/C58EFF59AF9B81AE6CFAC9ED16486B3A#overview
  • https://epubs.siam.org/doi/pdf/10.1137/10080991X
• Line 72: I wonder how are major shortcomings identified using fence-line approaches by Ilonze et al. (2024)?
• Line 219: “The roughness sublayer is set to 1” is not well justified. I suppose it means 1m? A roughness sublayer (RL) is a region immediately above the surface and below the inertial layer where horizontal in-homogeneity in flow variables persists. In other words, within the RL variables like <w’c’> (where c is the local methane concentration, w is the vertical wind speed, and ‘ represents the fluctuating field) can be assumed to be horizontally homogeneous. While there are well-established results in the literature on how to estimate the RL height, having access to direct point measurements at multiple heights offers a direct route to estimate the RL height. Please see the following reference.
  • https://journals.aps.org/prfluids/abstract/10.1103/PhysRevFluids.3.114603
• Lines 430-431: “The main assumption of the Gaussian plume model is that CH4 emitted from a point source enters the airflow, disperses vertically and laterally, forming a conical plume (Riddick et al., 2022b; US EPA, 2013)”. This is not an assumption but instead the standard response of the tracer being advected horizontally and vertically by the air.
• Line 431-433: “The formation of a conical plume is hindered at oil and gas facilities by obstacles (equipment) and is affected by atmospheric stability.” While strictly speaking sizable obstacles may hinder the plume, whether this is indeed the case for the METEC site is unclear. The manuscript does not offer proper evidence to support this argument. Given the sparsity of the few obstacles on the METEC site and the focus of the current study on distant, fenceline monitoring, it seems to reason that obstacles in fact do not hinder plume features.
• Line 463-465: “Even though these modeling approaches have been reported to work elsewhere (e.g., agricultural and landfill emissions), it does not necessarily mean it could work in the intended area of application.” This blanket statement is unwarranted. Compared to many complex scenarios observed in agricultural and landfill emission cases, the current controlled-release tests present a much-simplified dataset both in terms of emission characteristics (discrete constant emissions rates from only 5 equipment groups all starting and stopping simultaneously) and site features (flat terrain with few sparsely spaced obstacles).

Mbua et al.: Evaluating the accuracy of downwind methods for quantifying point source emissions.

General Comments

This paper makes a valuable contribution towards quantifying emissions from point sources. The problem is important and the work is thoughtful.

Mbua et al. use controlled release experiments to  test three widely used methodologies and find all three methods are inaccurate and two methods seriously underestimate emissions.

In Mbua et al’s controlled release experiments, using fence line measurements, both eddy covariance and flux gradient methods were found to underestimate emissions.  Gaussian plume methods performed better but with very wide scatter.

I note the points already raised by RC1 and RC2,  particularly RC1’s point 4 on closed path measurement leading to underestimation.

More generally, this paper is tackling a major problem.  Accurate quantification of emissions (especially methane) from point sources such as natural gas production facilities is a tough nut to crack, more so if the “point: source is somewhat disseminated  (e.g. a gas production facility with pipes, pumps and valves scattered over areas up to a hectare, or similarly, a farm complex over a similar area with manure lagoons, biodigesters and animals. Very large sources (eg >100 kg/hr methane)  can be studied from space, and small sources (e.g. single cows) can be isolated, but quantifying complex aggregated sources in the 1-50 kg/hr range (like gas production facilities) is not easy.

Use of methods similar to Mbua et al’s systems is widespread, and thus these experimetns are potentially important in devising better protocols and better regulation and mitigation of methane emissions from fossil fuel installations (gas production facilities, venting oil wells, coal mine vents, etc). The work also has wide applicability for tracking biogenic emissions from landfill and waste, around biodigesters, and on farms with large animal populations.

The scientific methodology  and experimental procedures are clearly explained and the work is well presented. Overall the paper is well written. Controlled release experiemnts like this are likely to be influential in designing better quantification methods and in devising regulation protocols. Thus the paper should be accepted. That said, I note the comments made by other referees and have some additional minor comments.

Minor comments:

Abstract. Lines 1-11 are really introduction and might be omitted here, and covered in Section 1 instead. Line 20-21 ‘input to……..method’ – minor English problem? Maybe methodS?

Lines 1-32 could be rewritten somewhat, maybe to mention the Global Methane Pledge and to introduce the tension between bottom-up and top-down methodologies.

Line 50 – maybe give some details about very rough emission ranges (e.g. satellite detection of point sources >100 kg/h, low flying aircraft (say 10-100 kg, SUV-mounted instruments (say 1-20 kg/hr). Also maybe mention the scale of  ‘disseminated’ ‘point’ sources – e.g. a complex gas facility on a 100mx100m pad, or a farm barn/lagoon complex of the same size.

Line 56 – ‘NG acronym may not be understood in a global journal. Better spell it out.

Line 96 – Fig 1 is good. Maybe add a Gaussian plume panel also?

Line 115 – maybe say how big the METEC facility is and give more details for non-US readers.

Line 181 – masts’ locations? Plural masts on many locations? Text as it is now has many mast on one location.

Line 194 – background. 5^th percentile - A steady time-invariant small leak could thus be included in ‘background’ Are there any measurements of upwind background?

Line 215 – footprint 80%-90% contour contrast. This is interesting: space for a slightly longer comment?

Line 235 – Line 346 ‘largely underestimated’ – see also Fig 4. I note the comment posted earlier by Brian Lamb. Maybe that question could be picked up further? Mbua et al’s finding is surprising and immediately makes the reader worry about eddy covariance quantification in wetlands also!

Lines 388- 401. This is important and perhaps needs also later to be put in the wetland context where atmospheric stability can also be a problem.

Line 434 – daytime insolation - in facilities working all night as well as all day, nocturnal factors like low inversion height and fogs may also be a factor, especially if the facility is installing automated systems.

Line 465 – maybe a paragraph here in ‘future fixes’ – how to make EC work better perhaps, and especially how to make Gaussian methods better. For example, by using drones it may be possible to map plumes much more accurately. Also perhaps a digression somewhere to mention wetlands (e.g. L386)  in more detail.

Conclusion

This is a valuable paper that should be published after minor revision.