the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Quantifying the uncertainties in thermal-optical analysis of carbonaceous aircraft engine emissions: An interlaboratory study
Abstract. Carbonaceous particles, such as soot, make up a notable fraction of atmospheric particulate matter and contribute substantially to anthropogenic climate forcing, air pollution, and human health. Thermal-optical analysis (TOA) is one of the most widespread methods used to speciate carbonaceous particles and divides total carbon (TC) into the operationally defined quantities of organic carbon (OC; carbon evolved during slow heating in an inert atmosphere) and elemental carbon (EC). While multiple studies have identified fundamental scientific reasons for uncertainty in distinguishing OC and EC, far fewer studies have reported on interlaboratory reproducibility. Moreover, existing reproducibility studies have focused on complex atmospheric samples. The real-time instruments used for regulatory measurements of aircraft engine non-volatile particulate matter (nvPM) mass emissions are required to be calibrated to the mass of EC determined by TOA of the filter-sampled emissions of a diffusion flame source. However, significant differences have been observed in the calibration factor for the same instrument based on EC content determined by different calibration laboratories. Here, we report on the reproducibility of TC, EC, and OC quantified using the same TOA protocol, instrument model (Sunset 5L), and software settings (auto split-point: Calc405) across five different laboratories and instrument operators. Six unique data sets were obtained, with one laboratory operating two instruments. Samples were collected downstream of an aircraft engine after treatment with a catalytic stripper to remove volatiles. We compared laboratory-reported uncertainties with actual variability in the data set, the difference of which (dark uncertainty) was substantial. Interlaboratory (dark) contributions increase uncertainties by a factor of 1.2 – 1.6 relative to the laboratory-reported uncertainties, even for these relatively simple samples (combustion particles downstream of a stripper), resulting in uncertainties of 26 % (k = 2) for EC. Uncertainties were a little larger for EC than for OC. These results indicate that current TOA uncertainties are underestimated and should be adjusted upwards to reflect these interlaboratory differences.
- Preprint
(1019 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on amt-2024-1', Anonymous Referee #2, 18 Mar 2024
This study comprehensively evaluates the previously poorly quantified inter-laboratory uncertainty in TOA analysis used to derive EC, OC, and TC. Aerosol exhaust from a helicopter engine was collected on 20 filters in accordance with regulatory civil aviation specification for nvPM mass instrument calibration. The comprehensive analysis presented in this study underscore the importance of including the inter-laboratory contribution to EC, OC, and TC uncertainties as it was found to be significant. I enjoyed reading your manuscript, which is well written, includes informative figures, and provides a detailed description of the methods and uncertainty analysis. However, I felt that the implications and limitations of your work were not discussed extensively. Below are a few general comments and minor suggestions for the authors to consider.
Minor comments:
- Line 19: You’ve omitted to say that regulatory aircraft nvPM mass can be directly calibrated on a gas turbine, not just a diffusion flame.
- Line 27: “Uncertainties were a little larger for EC than for OC”; I suggest merging with previous sentence and directly report OC uncertainty rather than using “a little larger”.
- Line 200-204: It is unclear to me how the 'dark inter-laboratory uncertainty', as shown in Figure 5 and Table 2, could be lower for TC than for EC, given that TC is derived from EC. Can you clarify and discussed in the main text if relevant?
- The sections 3.1 and 3.2 titles aren’t very clear, I suggest renaming them (for example 3.1 could be EC, OC and TC uncertainty and 3.2 could be EC/OC uncertainty).
- Line 287-288: ”uncertainties are poorly captured by existing estimates for these measurements”; Can you clarify that you mean by existing estimates? Do you mean lab-reported or what has been reported in the literature or both? Can you also expand on “except perhaps for measurements of OC” because I didn’t get that impression from your literature review in the introduction.
- L304: It would be useful to introduce the term “metrological” earlier in the manuscript to ensure that all readers are familiar with the term and its significance.
General comment:
- Are there any instruments other than the Sunset 5L commercially available and used by competent laboratories to make TOA measurements? Could your reported uncertainties be specific to that instrument? I suggest discussing this where relevant.
- Something that caught my attention is that your calculated cumulated uncertainties (e.g., 26% (k = 2) for EC) from your highly controlled study (i.e., nvPM, identical filters, known composition) are higher than that reported from other studies (Schauer et al., Ten Brink et al, Panteliadis et al., Brown et al.), yet you mentioned that atmospheric samples from these studies should have higher uncertainties than yours. Why do you think you estimated a higher overall uncertainty than say, Panteliadis et al.? It would be interesting to discuss these differences in more details.
- Could the way each laboratories take their punches and/or their handling of the punches be responsible for the laboratory bias you reported (could one laboratory take their punch in a way that led to a systematic bias)?
- I suggest adding more discussion on the implications and limitations of your work. By that, I mean discussing alternatives to TOA and how to reduce TOA uncertainty for regulatory aircraft nvPM mass emissions (use manual split, more thorough calibration procedures and quality controls, alternative calibration methods, etc).
Citation: https://doi.org/10.5194/amt-2024-1-RC1 -
RC2: 'Comment on amt-2024-1', Anonymous Referee #1, 19 Mar 2024
Review of “ Quantifying the uncertainties in thermal-optical analysis of carbonaceous aircraft engine emissions: An interlaboratory study”
This article presents an interesting and pertinent study on the estimation of measurement uncertainties on the total carbon (TC), elemental carbon (EC) and organic carbon (OC) contents, measured with the instrument Sunset 5L owned by five certified laboratories worldwide. Six equally shared punches from 20 samples probed from the exhaust of a helicopter engine were distributed for analysis among the chosen laboratories. Each instrument user respected the NIOSH5040 analysis protocol. The database was analyzed with statistical multilevel models to identify or predict the uncertainty bias among the samples.
The article is well-written and fits the topic of this journal. The obtained results are highly interesting for research topics such as nvPM emissions measurement protocols, atmospheric measurements, and aviation emissions, measurements, and protocols. There are a few arguably contrasting points that deserve to be put in a better light or clarified.
General remarks to clarify
- The text does not clearly state when the samples were obtained. Is it the same work as Olfert et al. (2017) or another specific study? Please clarify this aspect.
- Despite the significant experimental work in the referenced work Olfert et al., 2017, this article does not state which engine operating conditions were used for the obtained samples. It may not seem relevant to the authors, but why not have well-identified conditions in which nvPM was produced by the engine? I think this information is essential since the same operating conditions of the engine were used for three sets of samples loaded with 50, 100 and 250 µg/m3 of soot, while the last sample loaded with 500 µg/m3 of soot was obtained by increasing the RPM of the engine. It is well known that changing the engine's operating conditions will impact the structure and morphology of soot particles/nvPM. Isn’t this contradictory with what the authors state in the paragraph from the introduction containing Lines 74 to 77? The reader can find additional information about the sampled particles on the filter by identifying the operating conditions in the specified article if the work is common and even though the detailed statistical analysis did not identify any correlation between the filter loadings and uncertainties (lines 183-184).
- The filter holder from Figure 1 contains two filter holders in series. Was the second filter analyzed for some residual TC content, as presented in the work of Corbin et al. (2020)?
- It is surprising that the different loadings of the samples do not affect the uncertainty measurement of the three quantities measured by the Sunset instrument. This finding deserves a more detailed discussion since studies show that the loading of the filter impacts the uncertainty measurement of the thermo-optical analysis measurements.
- The use of the word structure (lines 183, 225, 229) and structural trends (line 227) can sometimes be misleading in the text for readers who are not specialized in multilevel statistical analysis. Please be more specific where it is the case; such as data/uncertainty structure or something that fits better in the context.
Specific comments:
Figure 2 - to which sample corresponds to the obtained data? it is worth mentioning.
Figure 6 - what represents the error bars in the bottom graphs with the Relative value [%] since it is mentioned that the error bars are excluded for clarity?
Line 37: remove on from the sentence “… mass on collected …”
Line 110: “darkness …” can be replaced with “coverage …” to differentiate from dark uncertainty
Line 136: remove the from the sentence “… and the their uncertainty …”
Line 199: The authors mentioned, “These filters coincide with cases where the overall variance is larger and represent a minority of cases.” Please be more specific when selecting a criterion for the value of the variance to eliminate the sample in question. Either be specific and justify why this selection was made or mention if you referred to data outliers.
Citation: https://doi.org/10.5194/amt-2024-1-RC2
Viewed
HTML | XML | Total | BibTeX | EndNote | |
---|---|---|---|---|---|
225 | 37 | 13 | 275 | 11 | 13 |
- HTML: 225
- PDF: 37
- XML: 13
- Total: 275
- BibTeX: 11
- EndNote: 13
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1