Calibration of PurpleAir low-cost particulate matter sensors: model development for air quality under high relative humidity conditions

Mathieu-Campbell, Martine E.; Guo, Chuqi; Grieshop, Andrew P.; Richmond-Bryant, Jennifer

doi:https://doi.org/10.5194/amt-17-6735-2024

Articles | Volume 17, issue 22

https://doi.org/10.5194/amt-17-6735-2024

© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/amt-17-6735-2024

© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 17, issue 22

Research article

|

26 Nov 2024

Research article |

| 26 Nov 2024

Calibration of PurpleAir low-cost particulate matter sensors: model development for air quality under high relative humidity conditions

Martine E. Mathieu-Campbell, Chuqi Guo, Andrew P. Grieshop, and Jennifer Richmond-Bryant

Download

Final revised paper (published on 26 Nov 2024)
Supplement to the final revised paper
Preprint (discussion started on 02 May 2024)
Supplement to the preprint

Interactive discussion

Status: closed

CC1:
'Comment on egusphere-2024-1142', JAMES OUIMETTE, 09 May 2024

Hi,
Thank you for your preprint. I have a couple suggestions that could improve your paper.
Could you please provide a table with the following information about each of the PA sensors used in this study:
PurpleAir ID number; AQS number for the regulatory monitoring site; name of regulatory PM2.5 monitor (e.g., Teledyne T640x, Met One BAM 1020, etc); distance from PurpleAir to regulatory PM2.5 monitor; name of the NOAA site used for relative humidity and temperature data; distance from PurpleAir to NOAA site.
The sites that you chose are characterized by high dew points, resulting from both high RH and high temperatures.
Your graphs comparing RH between the PurpleAir and its corresponding NOAA site is inadequate for assessing whether or not the NOAA site is representative. The best way to show if the PurpleAir and its corresponding NOAA site are sampling similar air is to compare their hourly average dew points. That's because the PurpleAir slightly heats the air sample, resulting in a higher temperature and lower RH compared to the NOAA site. However, the water content and dew point should be the same for the PurpleAir and the NOAA site.
Could you please provide graphs comparing the hourly average dew points for your 21 sites.
Thanks,
Jim Ouimette

Citation: https://doi.org/10.5194/egusphere-2024-1142-CC1
- AC3: 'Reply on CC1', Jennifer Richmond-Bryant, 05 Jul 2024
  
  Dear Dr. Ouimette,
  We greatly appreciate the insightful thoughts you raised about our manuscript! Please see the attached file with a description of how we considered those comments in our revisions.
  
  Citation: https://doi.org/10.5194/egusphere-2024-1142-AC3
RC1:
'Comment on egusphere-2024-1142', Anonymous Referee #1, 11 May 2024
This paper looks for a better PurpleAir correction for sensors in the US southeast. However, they only consider one equation from the literature when much additional work has been done on this topic in the past 4 years. This is not the first paper to look at nonlinear RH correction and the paper would be strengthened by comparing to other corrections in the literature that account for nonlinear RH. I have a number of other specific comments below that I hope the authors will address to strengthen their paper. The editor should also find someone to review that is more familiar with semi-supervised clustering.

Major
I think this paper would be strengthened by considering other common corrections from the literature especially those that consider nonlinear RH terms (e.g., Wallace https://www.mdpi.com/1424-8220/22/13/4741, Nilson https://amt.copernicus.org/articles/15/3315/2022/amt-15-3315-2022.html, Malings https://www.tandfonline.com/doi/full/10.1080/02786826.2019.1623863)

Also, can you add a plot showing the RH nonlinearity? You say that the model shows that it shows up around 50% but where does it increase visually? Something like RH on the X axis and Sensor/Monitor on the Y axis (Examples: Zheng https://amt.copernicus.org/articles/11/4823/2018/)

This paper discusses how the southeast is unique because it is high humidity but it would also be helpful to comment on how particle properties (e.g., composition, size distribution) are different in the south east and how that might impact the performance (e.g., Patel https://amt.copernicus.org/articles/17/1051/2024/, Jaffe https://amt.copernicus.org/articles/16/1311/2023/).

How does the recent release of the T640 correction impact this work? I agree with Dr. Ouimette that it would be helpful to list all the AQS monitors compared to, I assume some of them are Teledyne T640s.

Were any of these sensors the alternate PMS5003s? Sear, Kaur, Kelly, https://www.sciencedirect.com/science/article/pii/S0021850223001210 How does this impact your results?

How much data is excluded for each of the QA methods? (AB channel comparison high, low, etc.)

Figure 2 seems to show a wider range of RH with more noise over time. Is this due to seasonal differences or because the RH sensor performance is changing over time?

Did you consider whether sensor age had any impacts on your results? (e.g., deSouza https://pubs.rsc.org/en/content/articlehtml/2023/ea/d2ea00142j)

“However, DP was excluded as a predictor in our study, because collinearity was found between DP, RH, and T when testing for variance inflation factor. This collinearity is attributed to the direct physical relationship between RH and T” I don’t understand what this is saying? T and RH weren’t significantly colinear?

Random withholding is likely not a good test of your model. It would likely be fairer to withhold by site or state. I think it isn’t surprising that the model you built for your dataset is a better fit than a model built on another dataset. This is likely something to mention in the limitations.

Table 2 this is interesting basically if the RH is high add 5 ug/m3 to the concentration? This difference doesn’t seem to be reflected in Figure 6. Is there a typo?

Citations should be checked for accuracy throughout see a few specific comments below.

While the results are significantly different, they are not largely different. You might consider adding evaluation of performance by AQI category to further strengthen your findings (e.g., https://www.mdpi.com/1424-8220/22/24/9669, https://amt.copernicus.org/articles/15/3315/2022/amt-15-3315-2022.html )

Minor
A study conducted in 2016 (AQ-SPEC, 2016), evaluating about twelve low-cost PM2.5 sensors showed an overall good agreement between PM2.5 PurpleAir sensors and two reference monitors with a R2 of 78 % and 90 % (Wallace et al., 2021). - Is this citation correct? It seems like the beginning and ending of the sentence are citing 2 different things.

Lunden, M. M.; Parworth, C. L.; Barkjohn, K. K.; Holder, A. L.; Frederick, S. G.; Clements, A. L. Correction and Accuracy of PurpleAir PM 2.5 Measurements for Extreme Wildfire Smoke. 2022. https://doi.org/10.3390/s22249669. – This citation is incorrect

Line 45, 269: Why are there superscript numbers? Check for this throughout

Figure 1: Is the number of counties by state relevant to the story you are telling?

Line 270: “For all the four fitted models, average concentration of 8.80 μg m-3 , with an SD varying between 4.71- 4.84 μg m-3 were obtained, whereas Model Bj provided and a higher MAE than the four developed models with a mean of 7.67 μg m-3 and a SD of 6.08 μg m-3 .” -A little unclear if the first and second part of this sentence are comparing the same thing.

“Zheng et al. (Zheng et al., 2018) found an R2 280 value of 66 % for a 1-h averaging period after applying an MLR calibration equation to compare three PA sensors” – This is not a paper about PurpleAirs it is a paper about custom built Plantower PMS3003 sensors

I don’t think R2 is usually reported as a Percentage?

What is R in Table 1? Just the root of R2?

Figure 4: I think this plot would be easier to interpret if both plots used the same color scale.

This is a personal preference so take or leave, but I would always put the monitor on the X axis and the Sensor on the Y since the monitor is the independent variable. This is also the recommendation in the EPA performance targets.

Figure 7: Is there an assumed T and RH for the lines on this plot?
Citation: https://doi.org/10.5194/egusphere-2024-1142-RC1
- AC1: 'Reply on RC1', Jennifer Richmond-Bryant, 05 Jul 2024
  
  Thank you so much for your very thorough and thoughtful review! Please see the attached file, which explains how we incorporated your feedback into the manuscript.
  
  Citation: https://doi.org/10.5194/egusphere-2024-1142-AC1
RC2:
'Comment on egusphere-2024-1142', Anonymous Referee #2, 28 May 2024
General comments:
This paper provided the evaluation of PurpleAir correction using the warm, humid climate zones data and aimed to improve the EEPA Barkjohn model. It provides helpful information about improved performance metrics and avoids collinearity using DP, RH and T. However, the multilinear regression has been used before. There is no significant scientific insight gained with the new parameters. Several suggestions to strengthen this paper:
Consider other correction methods and explain what can provide the best insight of the Purpleair data.

Typically, the low-cost sensors measure the PM base on the optical size, and it is unclear how they can accurately predict the aerodynamic size and get the correct PM2.5. The conversion of particle aerodynamic size to optical size, or vice versa, is not straightforward because it depends on several factors, including the particle's shape, density, and refractive index. Are the FRM/FEM monitors filter-based measurements? How does the linear regression provide reliable information?

Specific comments:

Line 127-129, Please explain how to determine the detection range for PurpleAir? The reference used 1.15-2.55? This paper used 1.5? Why not 1.6? or 1.75?

Line 131, What is the difference between the two channels? Should we expect them to agree in a certain percentage at low and high concentrations?

Line 141, For each site, how much data remained? Does this data cleaning cause any bias in the data collection?

Section 2.4.2, the equations are confusing. Will the beta 2 in equation 2 be the same as the beta 2 in equation 3?

Table 1, the parameters from each model have a very high precision. Is it realistic to include such high precision?

Figure 4 the data plotted seemed to be from two groups. One follows 1:1 line, and the other one follows 2:1 line. Cluster 2 still has the 2:1 group. Is there any other reason for this 2:1 group?
Citation: https://doi.org/10.5194/egusphere-2024-1142-RC2
- AC2: 'Reply on RC2', Jennifer Richmond-Bryant, 05 Jul 2024
  
  Thank you so much for your very thorough and thoughtful review! Please see the attached file, which explains how we incorporated your feedback into the manuscript.
  
  Citation: https://doi.org/10.5194/egusphere-2024-1142-AC2

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload

AR by Jennifer Richmond-Bryant on behalf of the Authors (05 Jul 2024) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (10 Jul 2024) by Jessie Creamean

RR by Anonymous Referee #2 (19 Jul 2024)

RR by Anonymous Referee #1 (23 Jul 2024)

Suggestions for revision or reasons for rejection

While some improvements have been made to this paper, in my opinion the authors have not yet adequately shown what their work adds to past work. They use a new correction method but do not compare to the accuracy that could have been achieved with other commonly used corrections in the literature. They only compare to the Barkjohn equation. The Barkjohn equation is one of the older PurpleAir corrections and multiple other corrections showing better performance than the Barkjohn equation have been published in the past 3 years (e.g., Wallace, Nilson). This was an issue I brought up on the last round of the draft that the authors did not address. I hope that the authors will address this comment along with my other specific comments below. They seem to have added additional inaccuracies in some places and may have errors in some figures. I do not feel this paper is ready for publication but think that it could be after another round of major revisions.
Major:
• On the last round of revisions, I and the other reviewer requested that the author consider other corrections common in the literature. To me considering other equations would mean applying corrections of similar form to their dataset and then comparing how the coefficients and performance compare to past work and other corrections. Instead, they have just added a paragraph summarizing the results from the past studies.
• Met One BC-1060 – this is a black carbon monitor not a PM2.5 monitor. Why would you compare it to PM2.5? It looks like this site also runs an FRM R & P Model 2025 PM-2.5 Sequential Air Sampler w/VSCC that would have been more appropriate to compare to (https://www.epa.gov/outdoor-air-quality-data/interactive-map-air-quality-monitors). I don’t think these sites should be included in Figure 1. It is misleading when you didn’t have any PM2.5 data you compared to in GA to include it in the states that you had comparisons for. At a minimum it should be in a different color if not included in the model development.
• If you are using only hourly data, then you will never use FRM data since it is only 24-hr averages. You should clarify this throughout including “included only hourly average PurpleAir data points that had a spatial (within the calculated radius) correspondence to hourly FRM or FEM concentration”. When you do the 24-hr analysis do you only use 1-hr data averaged up to 24-hr data or do you also pull in the 24-hr FRM data?
• Figure 4: I am surprised that Model 4 performs better. It seems like the green plot shows more scatter and also has a lower slope further from the 1:1 line. I also don’t see any negative data from the Barkjohn equation even though you stated that was an issue. Comparing Figure 4 to 7, I think one of these plots is wrong in Figure 7 model Bj is closer to the y axis while in Figure 4 Model 4 is closer to the Y axis. Am I missing something or is there a mistake in your figure labeling?
Minor:
• “However, we did not include Nilson et al. (2022) since they only developed linear models using CF-1 PurpleAir data.” Their work was based on the cf_atm data based on the corrigendum they released last July https://amt.copernicus.org/articles/15/3315/2022/amt-15-3315-2022-corrigendum.pdf.
• Figure S4. It is hard to understand what this means since you haven’t normalized for the true monitor concentration (i.e., dividing each hourly PurpleAir concentration by that hours monitor concentration)
• The authors didn’t check their dataset to see if there were any alternative PMS5003s they just said that could have impacted their results when they easily could have checked the ratio between the bin data to see if they had any of the alternative PMS5003s.
• “The LOSOCV resulted in a RMSE and a MAE of 3.31 μg m-3 and 2.29 μg m-3 respectively for Model 4. These values were higher than those for the LOGOCV process, which is not surprising considering the variability between states.” 3.32 is the value in table S8.
• “To ensure data accuracy, AQS data are collected by FRM or FEM, which are typically filter-based monitors (U.S. EPA, 2023b)” I’m not sure this is helpful. While FRM are filter based and a beta attenuation monitor uses filter tape, the T640 is optical and is the most common FEM used in your study.
• “Based on the electronic effects of water uptake” Is this accurate? I thought this was due to the particles up taking water.
• Table S13 – it would be helpful to include the number of valid data points for each pair to better understand what part of the time period is represented since there is so much missing data according to Tables S1.
• Line 383 (track changes version): as I stated in my first review Zheng et al. is not about PurpleAir sensors PurpleAir should be replaced with Plantower in this sentence.
• Line 421: SD is not listed in Table 1. Is it an error metric?
• Line 489: “We compared our results with some nonlinear models that were previously tested for PurpleAir sensors” I don’t think you have done that. You haven’t even drawn the conclusion for the reader on whether their error is higher or lower than yours and you didn’t apply them to your dataset.
• Line 465: ” Click or tap here to enter text.” Remove

Hide

ED: Reconsider after major revisions (01 Aug 2024) by Jessie Creamean

AR by Jennifer Richmond-Bryant on behalf of the Authors (15 Aug 2024) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (21 Aug 2024) by Jessie Creamean

RR by Anonymous Referee #1 (21 Aug 2024)

RR by Anonymous Referee #2 (21 Aug 2024)

ED: Publish subject to minor revisions (review by editor) (05 Sep 2024) by Jessie Creamean

AR by Jennifer Richmond-Bryant on behalf of the Authors (12 Sep 2024) Author's response Author's tracked changes Manuscript

ED: Publish subject to technical corrections (17 Sep 2024) by Jessie Creamean

AR by Jennifer Richmond-Bryant on behalf of the Authors (25 Sep 2024) Author's response Manuscript

Short summary

The main source of measurement error from particulate matter PurpleAir sensors is relative humidity. Recent bias correction methods have not focused on the humid southeastern United States (US). To provide high-quality spatial and temporal data to inform community exposure in this area, our study developed and evaluated PurpleAir correction models for use in the warm–humid climate zones of the US. We found improved performance metrics, with error metrics decreasing by 16–23 % for our models.