22 Feb 2022
22 Feb 2022
Status: a revised version of this preprint is currently under review for the journal AMT.

Air pollution measurement errors: Is your data fit for purpose?

Sebastian Diez1, Stuart Lacy1, Thomas Bannan2, Michael Flynn2, Tom Gardiner3, David Harrison4, Nicholas Mardsen2, Nick Martin3, Katie Read1,5, and Pete M. Edwards1 Sebastian Diez et al.
  • 1Wolfson Atmospheric Chemistry Laboratories, University of York, York, YO10 5DD, UK
  • 2Department of Earth and Environmental Science, Centre for Atmospheric Science, School of Natural Sciences, The University of Manchester, Manchester, M13 9PL, UK
  • 3National Physical Laboratory, Teddington TW11 0LW, UK
  • 4Bureau Veritas UK, London, E1 8HG, UK
  • 5National Centre for Atmospheric Science, University of York, York, YO10 5DD, UK

Abstract. When making measurements of air quality, having a reliable estimate of the measurement uncertainty is key to assessing the information content that an instrument is capable of providing, and thus its usefulness in a particular application. This is especially important given the widespread emergence of Low Cost Sensors (LCS) to measure air quality. To do this, end users need to clearly identify the data requirements a priori and design quantifiable success criteria by which to judge the data. All measurements suffer from errors, with the degree to which these impact the accuracy of the final data often determined by our ability to identify and correct for them. The advent of LCS has provided a challenge in that many error sources show high spatial and temporal variability, making laboratory derived corrections difficult. Characterizing LCS performance thus currently depends primarily on colocation studies with reference instruments, which are very expensive and do not offer a definitive solution but rather a glimpse of LCS performance in specific conditions over a limited period of time. Despite the limitations, colocation studies do provide useful information on measurement device error structure, but the results are non-trivial to interpret and often difficult to extrapolate to future device performance. A problem that obscures much of the information content of these colocation performance assessments is the exacerbated use of global performance metrics (R2, RMSE, MAE, etc.). Colocation studies are complex and time-consuming, and it is easy to fall into the temptation to only use these metrics when trying to define the most appropriate sensor technology to subsequently use. But the use of these metrics can be limited, and even misleading, restricting our understanding of the error structure and therefore the measurements’ information content. In this work, the nature of common air pollution measurement errors is investigated, and the implications these have on traditional metrics and other empirical, potentially more insightful, approaches to assess measurement performance. With this insight we demonstrate the impact these errors can have on measurements, using a selection of LCS deployed alongside reference measurements as part of the QUANT project, and discuss the implications this has on device end-use.

Sebastian Diez et al.

Status: final response (author comments only)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on amt-2022-58', Anonymous Referee #1, 24 Feb 2022
  • RC2: 'Comment on amt-2022-58', Anonymous Referee #2, 16 Mar 2022

Sebastian Diez et al.

Sebastian Diez et al.


Total article views: 677 (including HTML, PDF, and XML)
HTML PDF XML Total Supplement BibTeX EndNote
507 159 11 677 28 5 9
  • HTML: 507
  • PDF: 159
  • XML: 11
  • Total: 677
  • Supplement: 28
  • BibTeX: 5
  • EndNote: 9
Views and downloads (calculated since 22 Feb 2022)
Cumulative views and downloads (calculated since 22 Feb 2022)

Viewed (geographical distribution)

Total article views: 665 (including HTML, PDF, and XML) Thereof 665 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
Latest update: 24 May 2022
Short summary
We compare the quality of the information provided by cheaper and more expensive but accurate devices when used to measure air pollutants. As commonly used metrics can be misleading to qualify this, we propose a visual analysis. As there are no perfect devices (regardless of price), this work highlights that before judging devices' usefulness, the user must specify their needs. We also stress that the quality of information must be considered for decision-making and when designing new policies.