Evaluation of optical particulate matter sensors under realistic conditions of strong and mild urban pollution

. In this paper we evaluate characteristics of three optical particulate matter sensors/sizers (OPS): high-end spectrometer 11-D (Grimm, Germany), low-cost sensor OPC-N2 (Alphasense, United Kingdom) and in-house developed MAQS which is based on another low-cost sensor – PMS5003 (Plantower, China), under realistic conditions of strong and mild urban pollution. Results were compared against a reference gravimetric system, based on Gemini (Dadolab, Italy), 2.3 m 3 /h air sampler, with two channels (simultaneously measuring PM 2 . 5 and PM 10 concentrations). The measurements were performed 5 in Sarajevo, the capital of Bosnia-Herzegovina, from December 2019 until May 2020. This interval is divided into period 1 strong pollution (December 2019 – March 2020) and period 2 - mild pollution (March 2020 – May 2020). The city of Sarajevo is one of the most polluted cities in Europe in terms of aerosols: the average concentration of PM 2 . 5 during the period 1 was 83 µ g/m 3 , with daily average values exceeding 500 µ g/m 3 . During period 2, the average concentration of PM 2 . 5 was 20 µ g/m 3 . These conditions represent a good opportunity to test optical devices against reference instrument in a wide range of ambient 10 particulate matter (PM) concentrations. The effect of an in-house developed diffusion dryer for 11-D is discussed as well. In order to analyze the mass distribution of aerosols, a scanning mobility particle sizer (SMPS), which together with the 11-D spectrometer gives the full spectrum from nanoparticles of diameter 10 nm to coarse particles of diameter 35 µ m, was used. All tested devices showed excellent correlation with the reference instrument in period 1, with R 2 values between 0.90 and 0.99 for daily average PM concentrations. However, in period 2, where the range of concentrations was much narrower, R 2 values 15 decreased signiﬁcantly, to values from 0.28 to 0.92. We have also included results of a 13.5 month long-term comparison of our MAQS sensor with a nearby beta attenuation monitor (BAM) 1020 (Met One Instruments, USA) operated by the United States Environmental Protection Agency (US EPA), which showed similar correlation and no observable change of performance over time.

aerosol optical properties are impossible to implement. Furthermore, densities appearing in equation (4) are not known a priori. That explains why it is difficult to calibrate OPS for realistic ambient PM concentration measurements: any laboratory calibration may or may not be applicable to the changing outdoor conditions (Tryner et al., 2020;Crilley et al., 2020).
For outdoor applications, there is an additional problem: hygroscopic growth of aerosols Granados-Muñoz et al., 2015;Di Antonio et al., 2018), which leads to overshoots of OPS if the ambient air humidity is (too) high. An 70 obvious solution is to dry the air. However, any proper drying system would cost more than many models of OPS and it is rarely seen in combination with low-cost sensors. Analytical corrections are often used: humidity sensors are used to measure the relative humidity of ambient air and some analytical model, like Kohler's theory (Castarède and Thomson, 2018) or Hänel equation (Hänel, 1976), is applied. Later in this paper we will make some observations on this issue.
Due to all the above-mentioned factors, it is always interesting to check how OPS perform in different realistic scenarios. 75 Numerous papers deal with laboratory calibrations and outdoor evaluations of OPS (Karagulian et al., 2019;Borghi et al., 2018;Chatzidiakou et al., 2019;Magi et al., 2020;Sousan et al., 2016b;Malings et al., 2020;Kelly et al., 2017;Sayahi et al., 2019;Crilley et al., 2018;Zheng et al., 2018;Tasic et al., 2012;Cavaliere et al., 2018;Mukherjee et al., 2017;Sousan et al., 2016a;Zhang et al., 2018;Holstius et al., 2014;Badura et al., 2018). Reported results vary depending on the composition of aerosol pollution, range of concentrations and meteorological factors. 80 The novelty of this research is a unique combination of instruments and conditions of extremely high urban pollution. The city of Sarajevo is situated in a valley and is affected by strong temperature inversions that appear typically 150 m-300 m above ground level with a very strong temperature gradient in the inversion layer, exceeding 30 K/km (Masic et al., 2019).
The inversion episodes were present during most of January 2020. As a consequence, the average monthly concentration of PM 2.5 was very high: 167.3 µg/m 3 . In contrast to that, monthly average values for March and April 2020 were 21.6 µg/m 3 and 85 19.6 µg/m 3 , respectively. This presented an excellent opportunity to test the performance of OPS in very different pollution levels. Simultaneously with OPS and reference gravimetric measurements, an SMPS was employed to detect nanoparticles.
It can detect particles with diameters from few nm up to 1 µm. While SMPS can count very small particles, 11-D can count larger particles, from 0.25 µm to 35 µm in diameter. When they work simultaneously, they can detect (almost) the full range of aerosol diameters, with a span of more than three orders of magnitude. This will give detailed insights on the mass distribution 90 of aerosols.

Methodology and experimental setup
The experimental facility was located at the Faculty of Mechanical Engineering in the central part of Sarajevo valley (N 43.85424, E 18.39607, 540 m above sea level) and represents well the overall conditions in the city. The reference instrument for measurements of PM concentrations was a Dadolab Gemini air sampler. It is a single device with two completely indepen-95 dent channels (PM 2.5 and PM 10 in this campaign). The filter preparation and gravimetric analysis are performed in separate laboratory of Faculty of Science, Department of Chemistry. The air sampler, gravimetric laboratory and all filter procedures satisfied requirements of the standard EN 12341:2014.
Grimm 11-D is a high-end optical particle sizer, with sophisticated construction and ability to count individual particles from 250 nm to 35 µm in 31 equidistant (on logarithmic scale) channels. It uses a proprietary algorithm and the manufacturer 100 does not share information about the refractive index, density or weighting factors. It was factory calibrated, and equipped with firmware version 12.50. Data was recorded in 1 minute intervals (6 seconds is also possible). Since we use the common term OPS occasionally, it should be noted that 11-D belongs to different category of devices (in comparison to low-cost sensors).
Alphasense OPC-N2 belongs to the category of low-cost sensors. The manufacturer transparently shared most specifications.
It has a much simpler construction than 11-D: instead of regulated pump, air flow is provided by 25 mm fan. The device has 105 16 channels, from 380 nm to 17 µm. Firmware version 18.2 was used. Refractive index was n = 1.50 + i0 and density was 1.65 g/cm 3 . All other parameters, including weighting factors, were used as firmware default values.
The Plantower PMS5003 could be termed a very low-cost sensor, since its price is lower by an order of magnitude than that of the OPC-N2. Limited specifications don't reveal all operating parameters. From the specification sheet we can conclude that the device uses Mie scattering theory, with detection limit of 300 nm, and has 6 channels. It is likely that PMS5003 works in 110 photometric mode, rather than counting individual particles. By disassembling one PMS5003 sensor we have discovered some components: red semiconductor laser which pulsates approximately once per second, photodetector and 32-bit processor. Air flow is provided by a 20 mm fan. The PMS5003 has two data outputs, one is called SM (standard material, CF = 1), and another AE (atmospheric environment), without further explanation from the manufacturer. The latter mode is used in our work. Based on PMS5003, we have designed MAQS (Mobile Air Quality System) smart sensor. Essentially, it is a modular platform for 115 PMS5003, with options for additional sensors (pressure, temperature, humidity, carbon dioxide, wind speed), GNSS receiver, flash memory, Wi-Fi module and 3D-printed enclosure. Eight MAQS sensors were made and tested prior to the main campaign in order to evaluate consistency between units. Figure 1 shows the results of preliminary outdoor measurements for a batch of 8 MAQS sensors. They showed very good consistency: the coefficient of determination, R 2 , between any two sensors from the batch was greater than 0.99 and average readings from all sensors are within ±10% from the average value of the batch of 120 sensors. Data was recorded every minute on a local SD card and remote cloud server simultaneously. Recording interval can be as short as 1 second, but there was no need for that. 4 https://doi.org/10.5194/amt-2020-237 Preprint. Discussion started: 8 July 2020 c Author(s) 2020. CC BY 4.0 License.
Grim 11-D and Alphasense OPC-N2 could not be used outdoors without shelter, while MAQS has a special case which provides basic protection for outdoor use. Furthermore, netbook PC was used to record data from the OPC-N2. Outdoor shelter had to be constructed to accommodate 11-D with power supply, OPC-N2 with PC and SPI adapter, and MAQS (for better 125 protection). Stevenson screen like wooden structure was designed for that purpose. Another MAQS sensor was used at a remote location, for reasons that will be explained later.
For low-cost sensors (OPC-N2 and MAQS) there was no air dryer or heater, since they are typically used in such conditions. We have designed and constructed diffusion dryer for application on 11-D, which consists of porous stainless steel tube surrounded with 1 kg of silica gel. The dryer is compact, 25 cm in length with 8 cm external diameter and does not reduce the 130 mobility of the instrument. It was installed only during the period of mild pollution.
Meteorological parameters were measured using Vantage Pro2 (Davis Instruments, USA) weather station with recording intervals of 15 minutes.
SMPS is a complex system which consists of a condensation particle counter (CPC), a differential mobility analyzer (DMA) and a charge conditioner (often inadequately called "neutralizer"). Depending on the characteristics of the DMA, the SMPS 135 can be configured for certain span of particle diameters. We have used Grimm 5.416 high-end SMPS with long DMA which is able to separate particles from 10 nm to 1000 nm in 129 channels, equidistant on a logarithmic scale. Despite the fact that aerosols with diameter below 10 nm play an important role in nucleation and growth studies (Tiszenkel et al., 2019), their contribution to the mass budget is negligible. Another (larger) in-house developed diffusion dryer was installed at the inlet of SMPS. A soft X-ray device was used as the charge conditioner. Scanning mode (alternating upscan and downscan) was used 140 for all measurements. One scan takes about 4 minutes (8 minutes for both upscan and downscan). When working parallelly, SMPS and 11-D form a powerful wide-range spectrometer, which covers a range of aerosol diameters from 10 nm to 35 µm in 160 channels. Additionally, there is an overlapping area between 250 nm and 1000 nm where we can see how well these two instruments match. The complex SMPS system was kept indoors (an unavoidable necessity, since both X-ray charger and DMA use very high operating voltages). The air was sampled from outside using a conductive tube of shortest possible length, 145 to avoid aerosol losses. It was running continuously, except for the periods of maintenance.
A rigorous data validation procedure was used. All instruments were inspected periodically and data logs were analyzed thoroughly. When calculating daily average values, complete and consistent data series were required.

Results and discussion
During this campaign 296 filters were used in the reference air sampler. After the removal of several blank filters used for 150 periodic verification and those with incomplete sampling (pneumatic system of air sampler failed to load new filters automatically couple of times), 288 filters remained: 143 PM 2.5 and 145 PM 10 samples. Figure 2 shows PM 2.5 and PM 10 daily average concentrations, together with hourly and daily values of ambient air temperature and relative humidity.
Some modifications of the shelter for 11-D and OPC-N2 were necessary, making those instruments unavailable periodically during December and January. Additionally, more frequent maintenance, such as cleaning of 11-D, was needed when working 155 5 https://doi.org/10.5194/amt-2020-237 Preprint. Discussion started: 8 July 2020 c Author(s) 2020. CC BY 4.0 License. in extreme conditions. The same stands for SMPS, which was maintained according to the recommendations of the manufacturer. Taking into account difficult operating conditions, the amount of data collected is satisfactory during the period of strong pollution and excellent during the period of mild pollution.
The lower limit of detection (LLoD) of PM 2.5 concentration for evaluated optical aerosol devices is estimated based on their actual field performance using standard methodology (3σ), and the results are: 5 µg/m 3 for PMS5003, 2 µg/m 3 for OPC-N2 160 and 1 µg/m 3 for 11-D. These values are comparable to those reported by Bulot et al. (2019).

Strong urban pollution
During the period of strong urban pollution (12/2/2019-3/12/2020), the average value of PM 2.5 concentration was 82.9 µg/m 3 , with minimum daily average value 1.3 µg/m 3 and maximum value 504.9 µg/m 3 . In the same period, the average PM 10 concentration was 95.5 µg/m 3 , with minimum value 3.6 µg/m 3 and maximum value 549.0 µg/m 3 . The ratio of average values of 165 concentrations PM 2.5 /PM 10 was 0.87. Very good correlations were observed for all three OPS against the reference instrument.
Such a range of ambient PM concentrations was favorable for achievement of high R 2 values, but non-linear effects of low-cost sensors were observed too.
Grimm 11-D produced results with R 2 values 0.988 and 0.985 for PM 2.5 and PM 10 concentrations, respectively. Absolute values were larger than the reference, on average 17.6% for PM 2.5 and 25.5% for PM 10 . The average ratio PM 2.5 /PM 10 170 measured by 11-D was 0.93.
Alphasense OPC-N2 undershoots with respect to the reference values, on average 31.0% for PM 2.5 and 36.8% for PM 10 , but R 2 coefficients are relatively high: 0.903 and 0.920 for PM 2.5 and PM 10 respectively. The OPC-N2 measured the ratio PM 2.5 /PM 10 to be 0.97.
MAQS sensor produced surprisingly good R 2 values of 0.975 for PM 2.5 and 0.950 for PM 10 . In terms of absolute values, 175 it overshoots by 31.9% for PM 2.5 and 49.3% for PM 10 (on average). The calculated ratio PM 2.5 /PM 10 was 0.76. It seems that the Plantower PMS5003 can not accurately determine the PM 10 fraction. One possible explanation is provided by a laboratory test of PMS5003, where it was found that its size bin [2.5 µm-10 µm] is noisy and inaccurate (Kuula et al., 2020). Further investigation of this behavior would be useful.
None of the tested OPS were equipped with an air dryer, and this certainly contributes to overprediction. Yet, Alphasense 180 OPC-N2 with default firmware settings underpredicts values, despite the aerosol hygroscopic growth effect. Figure 3 shows comparisons of all three OPS against reference for PM 2.5 and PM 10 during the period of strong pollution.
The much narrower range of particulate matter concentrations plays an important role, and even the reference method is less 185 accurate, since the mass difference of loaded and blank filters becomes very small (smaller than 1 mg for 24 h sampling period if PM concentration is below 18 µg/m 3 ). The average concentration of PM 2.5 was 19.7 µg/m 3 with a minimum daily average value of 7.1 µg/m 3 and a maximum value of 39.3 µg/m 3 . During this period, the average value of PM 10 concentration was 24.2 µg/m 3 , with minimum and maximum values of 7.6 µg/m 3 and 48.8 µg/m 3 , respectively. The ratio PM 2.5 /PM 10 was 0.81 on average.

190
This time Grimm 11-D was equipped with a dryer, whose effects will be discussed in the next subsection. The device produced relatively high R 2 values 0.868 for PM 2.5 and 0.917 for PM 10 . The absolute readings underestimated concentrations of PM 2.5 by 16.3% on average, while PM 10 were underestimated by 10.9% on average. The PM 2.5 /PM 10 ratio was 0.87. This test clearly shows that 11-D is a completely different class of instrument (in comparison to low-cost sensors). When equipped with dryer, 11-D shows level of performance comparable to BAM, at least those reported by Hafkenscheid and Vonk (2014). It would be interesting to test low-cost sensors with a proper dryer as well, but that combination is rarely seen in practice.

Humidity influence
One of the important factors in ambient measurements of PM concentrations is humidity, since the particles reflect more light (i.e. appear larger) during measurements due to hygroscopic growth. This can be described using Hänel equation: where f ζ is enhancement factor for aerosol property ζ. Here RH represents the relative humidity and RH ref is a reference relative humidity: It is important to note that the coefficient γ, which is an indicator of the hygroscopicity of aerosol, depends on the type of 210 aerosol (and changes whenever composition of ambient aerosols is changed).
If we compare results produced by 11-D relative to the reference, during period 1 (without dryer) and period 2 (with dryer), we can see that readings of the 11-D were reduced by more than 30%. However, we can not conclude whether it was the effect of the dryer or the consequence of significantly different ambient conditions. Unfortunately, we have only one 11-D, so we couldn't measure simultaneously with and without dryer (that's the reason why we used the instrument with the dryer 215 only in one of the two periods). If we take into account two intervals with similar ambient conditions, with and without dryer we get following values: from 2/27/2020 to 3/12/2020 average ambient concentration of PM 2.5 was 21.1 µg/m 3 while 11-D (without dryer) measured 21.5% more. In the second interval, from 3/13/2020 to 4/1/2020, ambient concentration was similar, 21.0 µg/m 3 while 11-D (with dryer) measured 1.4% smaller value. This comparison indicates that the effect of the dryer could be around 23%. A similar analysis for PM 10 concentrations gives an estimate of about 20% for the dryer effect. Grim 11-D has a very useful feature: internal temperature and humidity sensor. Figure 5 shows self-heating and diffusion dryer effect on 11-D, by comparing internal and external measurements of temperature and humidity. The average ambient air temperature from 2/27/2020 to 4/1/2020 was 7.02 0 C while the average 11-D internal temperature was 14.27 0 C, which shows a significant difference of 7.25 0 C. This self-heating effect reduces internal humidity significantly, and we can see that it rarely goes beyond 50%. Once the dryer is installed, internal relative humidity is further reduced: the average value of internal 225 humidity without dryer (2/27/2020-3/12/2020) was 36.2% and with dryer (3/13/2020-4/1/2020) was 21.8% (the ambient air humidity also dropped in the later period, but nevertheless the effect of the dryer is evident).
After roughly a month, the dryer's performance degraded and the silica gel needed a regeneration (but it wasn't performed since we didn't want to interrupt measurements when the end of the campaign was near).

SMPS data and wide-range spectrometer 230
The wide-range spectrometer (SMPS+11D) produced very valuable results. Figure 6 shows the continuous concentration and mass distributions. It is created from hourly average measurements from SMPS and 11-D. A relative density of 1.65 was applied in SMPS software (based on LabVIEW) for the mass calculation. No other corrections were performed and all settings were factory defaults. Selected histograms (hourly average values) are shown in Figure 7. What we can see from figures 6 and 7 is that in the period of strong pollution the dominant mass contribution comes from particles with diameters around 300 nm.

235
In terms of concentrations, particles around 100 nm appear in greatest numbers, with occasional secondary peaks coming from even smaller particles.
In the period of mild pollution, however, we can see that particles larger than 2.5 µm often appear on histograms (usually about 3 µm in diameter). Number concentrations still have peaks about 100 nm, but sometimes the distribution is different in favor of even smaller particles, as Figure 7 shows. Again, the largest mass contribution comes from particles around 300 nm.

240
In the overlapping area, SMPS and 11-D matched very well, almost perfectly for concentrations. Their match was not as good for mass calculations, but that is understandable, taking into account all the factors explained in section 1. Overall, the combination of SMPS and 11-D worked very well and gave the full spectrum of aerosols, both for number concentrations and mass distribution.
The obtained mass distribution of particles, especially during the period of strong pollution, rises the question on suitability 245 of OPS for measurements of mass concentrations, and resolving different fractions, since they cannot detect small particles that significantly contribute to the total mass. For example, the Alphasense OPC-N2 has a detection limit of 380 nm, and is likely to miss the particles around 300 nm which form the dominant contribution to the mass budget. The Grimm 11-D, with a detection limit of 250 nm, has a far better potential to resolve mass fractions.

Long-term performance 250
Another question about OPS, especially low-cost types, is the drift of performance over time. The PMS5003 sensor uses a semiconductor laser (diode laser) which has a limited lifetime. We have some long-term comparisons of the MAQS sensor with MetOne BAM-1020 operated at a nearby location by the US EPA. Strictly speaking, their station is not collocated with our equipment, but for the distance of only 300 m it is reasonable to assume that the air composition is very similar at these two points, since they are located in the same neighborhood. In order to verify that assumption, we have installed another MAQS 255 sensor at the location of Faculty of Electrical Engineering, University of Sarajevo, which is in immediate vicinity of the US EPA site, and at the same distance from us (about 300 m). Figure 8 shows long-term comparisons of MAQS sensor and BAM-1020, and additional verification of correlation between readings of two MAQS sensors, which was very high (R 2 = 0.970 for hourly average values and R 2 = 0.994 for daily average values) confirming our assumption that these two locations share the same air, in terms of PM concentrations and properties. learning (Si et al., 2020). An excellent viewpoint of this issue is given by Hagler et al. (2018). The calibration of a larger number of low-cost sensors can be simplified if they show similar relative performance (to each other) in laboratory and field (Sousan et al., 2018). Floating corrections, even physically justifiable interventions, such as the instantaneous correction for humidity growth of aerosols, insert a lot of noise, and the benefit is questionable. Depending on ambient conditions, selfheating of the sensor and some other factors, relative humidity may not be accurately determined. Even if we have a very 270 accurate humidity measurement, the hygroscopic growth coefficient will change whenever the composition of aerosol changes, inevitably injecting noise into the results.
We can also see strong non-linear effects at very high concentrations, above 500 µg/m 3 . In that case a quadratic regression fit will be more suitable.
During this period of 13.5 months of continuous outdoor operation, the MAQS sensor worked flawlessly without perfor-MAQS sensor overshoots PM 2.5 concentrations by approximately 30% on average, which is likely to be partially caused by hygroscopic growth.
The wide-range spectrometer, which consists of SMPS and 11-D, produced valuable information about distribution of aerosols, both in number and mass concentrations. Particles with diameters around 100 nm (and sometimes below) represent the dominant fraction in pure numbers, while particles with diameter of around 300 nm give the highest contribution