the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Long-term Evaluation of Commercial Air Quality Sensors: An Overview from the QUANT Study
Abstract. In times of growing concern about the impacts of air pollution across the globe, lower-cost sensor technology is giving the first steps in helping to enhance our understanding and ability to manage air quality issues. While the benefits of greater spatial coverage and real-time measurements that these systems offer are evident, challenges still need to be addressed regarding sensor reliability and data quality. Given the limitations imposed by intellectual property, commercial implementations are often "black boxes", which represents an extra challenge as it limits end-users' understanding of the data production process. In this paper we present an overview of the QUANT (Quantification of Utility of Atmospheric Network Technologies) study, a comprehensive 3-year assessment across a range of urban environments in the United Kingdom. QUANT stands out as one of the most comprehensive studies of commercial air quality sensor systems carried out to date, encompassing a wide variety of companies in a single evaluation and including two generations of sensor technologies. Integrated into an extensive data set open to the public, it was designed to provide a long-term evaluation of the precision, accuracy, and stability of commercially available sensor systems. This overview discusses the assessment methodology, and key findings showcasing the significance of the study. The results shown here highlight the significant variation between systems, the incidence of corrections made by manufacturers, the effects of relocation to different environments and the long-term behaviour of the systems. Additionally, the importance of accounting for uncertainties associated with reference instruments in sensor evaluations is emphasised. Practical considerations in the application of these sensors in real-world scenarios are also discussed, and potential solutions to end-users data challenges are presented. Offering key information about the sensor systems' capabilities the QUANT study will serve as a valuable resource for those seeking to implement commercial solutions as complementary tools to tackle air pollution.
- Preprint
(1928 KB) - Metadata XML
-
Supplement
(998 KB) - BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on amt-2023-251', Colleen Marciel Rosales, 30 Jan 2024
Please see attached PDF for the review of this paper.
-
AC1: 'Reply on RC3', Sebastian Diez, 09 Apr 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-251/amt-2023-251-AC1-supplement.pdf
-
AC1: 'Reply on RC3', Sebastian Diez, 09 Apr 2024
-
RC2: 'Comment on amt-2023-251', Anonymous Referee #2, 07 Feb 2024
General Comments
Overall this paper provides a good overview of the QUANT study and some salient results. A few clarifications are needed, as outlined below.
Section 2.3 should describe any harmonization of the data from the sensors’ reporting frequencies to a standard frequency, i.e., what was the common time frequency for which the measurements were averaged for analysis and comparison with the reference? Or was this done differently for the native reporting frequencies of each instrument? Finally, in the available QUANT dataset, are the measurements reported at the initial sampling frequency or at the down-averaged frequency (or both)?
In Section 3.1, results are only presented for the PM2.5 data. I would suggest that information on the inter-sensor precision for all measurands should be provided, maybe as part of the supplemental materials, since this is a basic feature of the different sensors which can inform all the other results presented later.
Since one of the goals of this paper is to introduce the QUANT dataset as a public resource for long-term performance assessment, it may be worth adding a section which details the dataset itself, or expanding the “Data Availability” section to do this. Some points to consider for this section would be the size of the dataset, the parameters included, what quality controls are applied (especially to the reference data), and any licensing of the dataset or policies associated with its use. Currently, the link provided in the “Data Availability” section does not seem to be working; presumably this will be active by the time of publication.
Specific Comments
Line 19: suggest clarification that this technology is providing the first steps for regions without pre-existing monitoring.
Line 34: “end-users” should be “end-user”.
Line 35: “capabilities the” should be “capabilities, the”.
Line 54: “helping mitigating” should be “helping to mitigate”.
Line 61: suggest removing “of”.
Line 90: “extensive” is repeated.
Line 118: “inlfuenced” should be “influenced”.
Line 128: “Quant” should be “QUANT”.
Line 142: “Polludrone: Poll” should be “Poll: Polludrone”.
Figure 2: Suggest using the same colors for the different sensors between the left and right panels.
Figure 3: Suggest moving this figure and associated discussion to the next section, since it is an assessment of performance against a reference rather than an assessment of inter-sensor consistency.
Figure 8: These seems to be a switch between the use of uncalibrated and calibrated data between the left and right panels as well. It is not clear what these calibrations are based on, and the application of the calibration might be a contributing factor to the difference in performance, together with the move between sites. It may be more illustrative to present a comparison at both sites with either the calibrated or the uncalibrated data only.
Lines 329-331: Sentence may be incomplete.
Lines 373-374: The meaning of this is unclear; does this mean that results from 2 systems were combined (e.g., to increase coverage)? Or were coverage and REU assessed separately for each device and then data from both devices combined to create the density plots of Figure 10?
Line 374: “systems, not” should be “systems not”.
Line 423-424: Please explain further the use of the reference data as a prior in this method.
Lines 435-436: Consider changing one of “developments” or “developing”.
Lines 460-469: I would suggest adding a sentence earlier in the document (and perhaps in the abstract) noting that further analysis will be left for future publications. I was expecting at several points a more comprehensive presentation of results across all pollutants and for all phases of the study, while only particular aspects of the results were highlighted. This is alright, but I think it needs to be more clearly stated up-front that this is not a comprehensive presentation of the study results. I would also suggest, as a topic for future work, examining the manufacturer-suppled calibrations in more detail, seeing where these improved upon the raw and where they perhaps did not, and how robust the calibrations are to environment changes and movement of the sensors to new sites. This is briefly presented in several figures, e.g., Figure 8, but a more comprehensive assessment across all sensors and pollutants could be made.
Supplemental Information, Lines 4-6: Indicate which of these channels and/or data products were considered for this study. Also report the sampling frequency for this sensor.
Supplemental Information, Line 55: “y” should be “and”.
Citation: https://doi.org/10.5194/amt-2023-251-RC2 -
AC1: 'Reply on RC3', Sebastian Diez, 09 Apr 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-251/amt-2023-251-AC1-supplement.pdf
-
AC1: 'Reply on RC3', Sebastian Diez, 09 Apr 2024
-
RC3: 'Comment on amt-2023-251', Anonymous Referee #3, 20 Feb 2024
I recommend this publication. However, as this is an important study that can be reused or used as a basis for others, I think it is important to go into more detail, especially methodologically, so that it can be continued and used, as the authors suggest at the end.
-
AC1: 'Reply on RC3', Sebastian Diez, 09 Apr 2024
The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-251/amt-2023-251-AC1-supplement.pdf
-
AC1: 'Reply on RC3', Sebastian Diez, 09 Apr 2024
Viewed
HTML | XML | Total | Supplement | BibTeX | EndNote | |
---|---|---|---|---|---|---|
395 | 144 | 19 | 558 | 47 | 15 | 13 |
- HTML: 395
- PDF: 144
- XML: 19
- Total: 558
- Supplement: 47
- BibTeX: 15
- EndNote: 13
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1