The MBPD retrieval algorithm is an improved retrieval method with modifications based on the MBMP retrieval method from Varon et al. (2021). The new algorithm follows the same logic of retrieving the vertical column concentrations of atmospheric methane ΔΩ (kg m-2) from Sentinel-2 SWIR reflectances. The main steps are shown in the flow chart of Fig. 1. The main idea is retrieving methane column concentrations from one spectral measurement featuring methane absorption and one not, such as two observations from different passes with or without a methane plume or two adjacent spectral bands with different methane absorption properties. For a given scene, the method compares the Sentinel-2 measurements with the top-of-atmosphere (TOA) radiance simulated by a 100-layer, clear-sky radiative transfer model at 0.02 nm spectral resolution over the band 11 and 12 wavelength ranges. The specific steps are as follows: first, in a specific pass (pass 1), the methane concentration enhancements are retrieved by minimizing the difference between the fractional change of Sentinel-2 reflectance and a fractional absorption model based on the simulated TOA radiance in bands 11 and 12; the same process is then repeated in another pass (pass 2), and the difference of these two retrieved column enhancements (two MBSP retrievals) is the MBMP methane column enhancement in pass 1 (Eq. 1). Here the subtraction between two passes aims to remove systematic errors in the MBSP retrieval due to wavelength separation between bands 11 and 12. In other words, the MBSP retrieval in pass 2 is mainly used for removing artifacts of the MBSP retrieval in pass 1. Therefore, in this paper we name pass 1 as the “target date (TD)” and pass 2 as the “comparison date (CD)” for clarification. The TD in our method is the date for which the plume size is estimated. By default the target date is here assumed to be chronologically after the comparison date, although in practice this need not be the case. 1ΔΩMBMP=ΔΩMBSP,TD-ΔΩMBSP,CD(kgm-2) We make some modifications during the column retrieval process since the MBMP retrieval can still lead to false detections, especially in the MBMP subtraction step (Eq. 1). In theory, in the background with no methane plume, we expect the two MBSP retrievals to have similar values of methane column enhancements since they are at the same scene. However, this is not always true because (1) MBSP retrieval can be greatly affected by the atmospheric conditions such as cloud coverage, (2) the MBSP retrieval in one pass may have similar spatial distribution but with all the pixel values higher or lower than the MBSP retrieval in another pass due to differences in various atmospheric or earth properties (e.g., solar zenith angle, surface albedo) between different dates, and (3) other unpredictable random measurement errors can occur in a specific pass. Therefore, we add the following steps to further reduce the number of false detections (see Fig. 1 for sequence).

Choose clear-view passes. First, we only select passes with a clear view for both the target date and comparison dates since clouds can result in false detections by affecting reflectance. Here we use Sentinel-2 cloud probability, a data product created with the sentinel2-cloud-detector library, to select clear-view passes with no large cloud coverage. Specifically, we select the passes with less than 10 % cloud coverage (i.e., the area with cloud probability higher than 65 % is less than 10 % of the total area of the study region).

After column retrieval, the methane column enhancements ΔΩMBPD are further used to calculate the emission source rate Q using the integrated mass enhancement (IME) method described by Varon et al. (2021) (Eq. 2) (Frankenberg et al., 2016; Varon et al., 2018). In this equation, IME is the integrated mass enhancement (kg), Ueff is the effective wind speed (m/s), and L is the plume size (m). 2Q=3.6×IME×UeffL(th-1) To calculate IME, we first generate Boolean plume masks based on ΔΩMBPD by selecting methane columns above some percentage threshold for the scene and smooth with a 3 × 3 median filter and a 3 × 3 Gaussian filter (see Fig. 1e). Here the percentage threshold is a parameter that will be tuned using the controlled-release experimental data below. This plume mask generation step sets the location and shape of the methane plumes.

Then the IME is defined as the sum of multiplication of column enhancements and pixel-level area of all the mask pixels. Note that the column enhancements here are the original enhancements without any data transformation such as normalization or extreme value removal applied to aid detection of the plume shape. The effective wind speed Ueff is the function of the local 10 m wind speed U10 derived by Varon et al. (2021), calibrated with large-eddy simulations. We collect local wind speed data from the high-resolution rapid refresh (HRRR) atmospheric model from the US National Oceanic and Atmospheric Administration (US NOAA, 2021). The plume size L is taken in a simplified form as the square root of the plume mask area.

To validate the performance of the new algorithm, calibration is required to compare the algorithm outcome with the ground truth. The goal of calibration is to assess the algorithm performance in both detection and quantification. Accurate yes/no detection is defined as the algorithm being able to detect a methane plume when it appears and detecting nothing when no plume appears. Accurate quantification means that the emission rate estimates derived from the algorithm are consistent with the ground-truth measured release volumes.

Additionally, the algorithm performance can also be improved by parameter tuning to best match the ground truth. Here the following three parameters in the new algorithm are tuned: (1) the upper bound threshold during extreme value removal bu, (2) the number of comparison dates for each target date n, and (3) the percentage threshold during the plume mask generation p. The way each parameter affects the algorithm outcome is described as follows.

The upper-bound threshold bu. bu is a parameter that occurs during the extreme value removal, during which the retrieval values higher than it are considered to be extreme outliers and are replaced by the threshold value. Thus, a lower bu means a more strict constraint during extreme value removal. Ideally, an optimal bu helps remove false detections due to the extreme highs. However, if bu is too low, a true methane plume may also be ignored since its retrieval values could be removed.

To quantify the algorithm performance, we use two assessment factors with focus on different aspects. First, we choose F1 score to assess the performance of detection. F1 score is a function of “precision” and “recall”, measures of false positives and false negatives, respectively (Eqs. 3–5). F1 score has a range of 0 to 1, with higher values representing better algorithm performance. In addition, we choose the average absolute error (AAE) to assess the performance of quantification (Eq. 6, where xi and x^i are the emission rate estimate and ground-truth emission rate in day i, and N is the number of days). AAE has a range of 0 to ∞ with lower values suggesting better algorithm performance. Absolute error is used so that under- and over-estimates do not cancel each other out.

3F1=2×precision⋅recallprecision+recall4precision=2×#TruePositive#TruePositive+#FalsePositive5recall=2×#TruePositive#TruePositive+#FalseNegative6AAE=∑i=1Nxi-x^iN

We selected a wide value range for each algorithm parameter during the parameter tuning. For bu, we noticed that the magnitudes of the pixel-level column enhancements of a methane plume are usually from 10-3 to 10-1 kg m-2. Thus, we selected 10 values from 0.01 to 0.1 kg m-2 with increments of 0.01 kg m-2 and four other values 0.005, 0.12, 0.15, and 0.20 kg m-2. For n, for each target date 15 clear-view passes were selected with the earliest comparison date around 45 d before the target date, so n ranges from 1 to 15 with increments of 1. For p, 16 values were selected from 0.80 to 0.95 with increments of 0.01. Therefore, there are in total 3360 scenarios of different combinations of three parameters. Each of these 3360 parameter settings were run to quantify volumes from all 10 study days.

Figure 5 shows how each parameter affects the algorithm outcome. In each figure, an assessment factor (AAE or F1 score) is shown as a function of two parameters, based on a fixed value of the third parameter (i.e., a “slice” through 2 parameters keeping the third constant). Here the fixed values are from the parameter setting with the lowest AAE. Figure 5a and b show that a small bu value (0.005–0.02 kg m-2) leads to bad algorithm performance with high AAE and low F1 score (AAE > 1.3, F1 score < 0.4). This suggests that the bu constraint is too strict in this range and removes retrievals not only from the extreme highs but also from true methane plumes. Thus, the algorithm starts to generate false negatives. Particularly in Fig. 5b when bu is 0.005 kg m-2, we see NAN values of F1 score because there is no true positive detection at all. Aside from the low-value range, AAE and F1 score show less sensitivity to bu at the other values. Therefore, the conclusion from bu tuning is that one should avoid excessively low values of bu (< 0.02 kg m-2).

Figure 5a and c show a rough decreasing trend of AAE along with higher n when n < 12. This suggests that a higher n helps with quantification accuracy by providing a more stable background and lowering the possibility of high-volume plume in the comparison dates. However, AAE does not show an obvious decrease when n ≥ 12, which suggests that 12 or more comparison dates are not necessary or at least cease to improve performance. Figure 5b and d show low F1 scores when n is low (for example, F1 scores < 0.67 when n = 2). This is because some target dates have their earlier comparison dates with higher methane plume volumes, and a low value of n does not effectively reduce the average volume in the comparison dates, thus resulting in more false negatives. In real applications, this may be a more serious problem if the plume is continuous across a long time period with varying volumes. Additionally, computational cost is roughly proportional to n, so too high of a value of n can have excessive computational costs with little benefit to accuracy. Therefore, the value of n should not be too low or too high, and from the figures we can conclude that a reasonable choice of n is in the range 10–12.

Figure 5c and e show that AAE decreases with higher p at first but starts to increase when p > 0.92. The decreasing trend is due to smaller plume volumes and fewer false positives resulting from smaller plume masks during the Boolean plume mask generation. The increasing trend in high p range, however, is because p becomes sufficiently high such that no mask is generated even for the dates with real methane plumes. This also explains why in Fig. 5d and f that the F1 score is low in high p ranges. Low AAEs occur in the p range 0.91–0.93, while high F1 scores occur in the p range 0.85–0.86. This suggests a trade-off between accurate quantification and accurate yes/no detection: accurate quantification usually requires a high p value, but accurate yes/no detection needs a lower p value (though not excessively low). Therefore, when selecting the best p value, we can choose to emphasize quantification accuracy and accept the possibility of missing plumes (p > 0.90), or we can choose to detect more plumes and accept the possibility of emission rate overestimation (p ≈ 0.85).

Here two specific scenarios shown in Tables 1 and 2 further illustrate the trade-off between accurate quantification and accurate yes/no detection. The “min AAE” scenario is an example of pursuing quantification accuracy. It has the lowest AAE of all the parameter settings and the highest precision, meaning that it also has the minimum amount of false positives. However, this scenario has three false negatives that reduce the F1 score. Aside from this specific scenario, the top 1 % scenarios with low AAEs have their bu ranging widely at 0.03–0.15, n in a middle-to-high range of 7–14, and p staying high at 0.91–0.92. On the other hand, the “max F1 score” scenario has the highest F1 score. It does not have false negatives, but in order to find all plumes it becomes too aggressive, leading to one false positive. Note that multiple scenarios have the same highest F1 score, and the scenario we show here is the one with the lowest AAE among them. The top 1 % scenarios with high F1 scores have their bu ranging widely at 0.02–0.12, n in a wide range of 1–15, and p in the middle range of 0.82–0.85.

As a compromise, we developed a method to apply the MBPD algorithm in sequence to reduce the quantification error further while keeping a high F1 score. The specific steps are (1) apply a scenario with high F1 score as the base case to generate the first round of emission rate estimates, (2) raise the value of p and apply the updated scenario again to generate the second round of emission rate estimates, and (3) for the passes with non-zero emission rates in both scenarios, update the base case estimates to the new ones since they are likely to be closer to the ground-truth volumes. We name this method the “two-step application” method. Here we only change the value of p since the mask extraction step where p is applied is after the column retrieval step where bu and n are applied. So a consistent bu and n greatly reduce the computation workload as we only need to redo the mask extraction. Different from the direct application of the MBPD algorithm, this method is specifically designed to address the trade-off issue between quantification accuracy and detection accuracy. Table 1 shows an example of the two-step application (“two-step hybrid” scenario) with the “base case” scenario. Results show that the two-step hybrid scenario achieves lower AAE than the base case scenario with F1 score remaining the same. Specific locations and shapes of detected plumes in min AAE, max F1 score, and two-step hybrid scenarios are shown in Fig. 6.

We also compared the performance of MBPD algorithm with the MBMP, MBSP, and SBMP methods from Varon et al. (2021) in Fig. 7. The top row is for a true emission rate of 7.38 t CH4 h-1, while the bottom row is for a true emission rate of 0 t CH4 h-1. Results show that the MBPD algorithm performs the best with both true positive and true negative detections. Its emission rate estimates are also the closest to the ground-truth volumes. The MBMP method has true negative detection in 17 October 2021 but shows a small false positive detection in 19 October 2021. Its emission rate estimate for this date is also much lower than the ground truth. This implies that the steps of normalization and inclusion of multiple comparison dates in the MBPD method contribute to a higher sensitivity to the true plume than the MBMP method. MBSP and SBMP retrievals perform worst with multiple large-area false positive plumes. The SBMP method is likely to produce false detections if the surface albedo changes across different passes, and the MBPD method reduces the effect of changing surface albedo by including different spectral bands and multiple comparison dates. The MBSP method can produce false detections because of the wavelength separation between two spectral bands, and the MBPD method largely removes these artifacts by subtracting the MBSP retrieval between different passes.