Depending on the measurement location, we deployed one set or several sets of air pollution sensors, and we refer to each set (provided by London-based AirPublic Ltd) as a multi-sensor “node”. Each of these nodes consists of multiple electrochemical and metal oxide sensors for PM and NO2, as well as sensors for environmental quantities and other chemical species known for potential interference with their sensor signals (required for calibration). Each node thus allows for simultaneous measurement of multiple air pollutants, but we will focus on individual calibrations for NO2 and PM10 here, because these species were of particular interest to our own measurement campaigns. We note that other species such as PM2.5 are also included in the measured set of variables. We will make our low-cost sensor data available (see “Data availability” section), which will allow other users to test similar calibration procedures for other variables of interest (e.g. PM2.5, ozone). A caveat is that appropriate co-location data from higher-cost reference measurements might not always be available.

For NO2, we incorporated three different types of sensors in our set-up for which purchasing prices differed by an order of magnitude. One aspect of our study will therefore be to evaluate the performance gained by using the more expensive (but still relatively low-cost) sensor types. Of course, our results will only be validated in the context of our specific calibration method so that more general conclusions have to be drawn with care.

Each multi-sensor node contained the following (i.e. all nodes consist of the same types of individual sensors):

Two MiCS-2714 NO2 sensors produced by SGX Sensortech. These are the cheapest measurement devices deployed in our set with market costs of approximately GBP 5 per sensor.

We conducted measurements at three sites in the Greater London area during distinct multi-week periods (Table ). Two of the sites are located in the London Borough of Croydon, which we label CR7 and CR9 according to their UK postcodes. The third site is located in the car park of the company AlphaSense in Essex, hereafter referred to as site “CarPark” (Fig. a). At CR7, the sensor nodes were located kerbside on a medium busy street with two lanes in either direction. At CR9, the nodes were located on a traffic island in the middle of a very busy road with three lanes in either direction (Fig. b). For the two Croydon sites, reference measurements were obtained from co-located, publicly available measurements of London's Air Quality Network (LAQN; https://www.londonair.org.uk, last access: 30 April 2020​​​​​​​). In the two locations in question, the LAQN used chemiluminescence detectors for NO2 and Thermo Scientific tapered element oscillating microbalance (TEOM) continuous ambient particulate monitors, with the Volatile Correction Model for PM10 measurements at CR9. For the CarPark site, PM10 measurements were conducted with a Palas Fidas optical particle counter AFL-W07A. These CarPark reference measurements were provided at 15 min intervals. For consistency, these measurements were averaged to hourly values to match the measurement frequency of publicly available data at the other two sites.

In total, we co-located up to 30 nodes, labelled by identifiers (IDs) 1 to 30. For our NO2 measurements, we considered the following 15 sensor signals per node to be important for the calibration process: the NO sensor (plus its baseline signal to remove noise), the NO2 + O3 sensor (plus baseline), the two intermediate cost NO2 sensors (NO2-A43F, NO2-B43F) plus their respective baselines, the two cheaper MiCS sensors, three different temperature sensors, and two relative humidity sensors. All 15 signals can be used for calibration against the reference measurements obtained with the co-located higher-cost measurement devices. We discuss the relative importance of the different signals, e.g. the relative importance of the different NO2 sensors or the influence of temperature and humidity in Sect. . For the PM10 calibrations, we used two devices of the same type of low-cost PM sensor, resulting in 2×10 different particle measures used in the PM10 calibrations. In addition, we included the respective sensor signals for temperature and relative humidity, providing us with in total 24 calibration signals for PM10.

We evaluate four regression calibration strategies for low-cost NO2 and PM10 devices, by means of co-location of the devices with the aforementioned air quality measurement reference stations. The four different regression methods – which are multiple linear regression (MLR), ridge regression, random forest regression (RFR) and Gaussian process regression (GPR) – are introduced in detail in the following subsections. As we will show in Sect. , the relative skill of the calibration methods depends on the chemical species to be measured, sample size available for calibration, and certain user preferences. We will additionally consider the issue of site transferability for sensor node 19, including its dependence on the calibration algorithm used. We note that we do not include the manufacturer calibration of the low-cost sensors in our comparison here mainly because we found that this method, which is a simple linear regression based on certain laboratory measurement relationships, provided us with negative R2 scores when compared with reference sensors in the field. This result is in line with other studies that reported differences between sensor performances in the field and under laboratory calibrations see e.g..

For all regression models, we performed fivefold cross-validation where the data are first split into training and test sets, keeping samples ordered by time. The training data are afterwards divided into five consecutive subsets (folds) of equal length. If the training data are not divisible by five, with a residual number of samples n, then the first n folds will contain one surplus sample compared to the remaining folds. Each fold is used once as a validation set, while the remaining four folds are used for training. The best set of model hyperparameters or kernel functions is found according to the average generalization error on these validation sets. After the best cross-validated hyperparameters are found, we refit the regression models on the entire training data using these hyperparameter settings (e.g. the α value for which we found the best out-of-sample performance for ridge regression).

The skill of a sensor calibration function is expected to increase with sample size, i.e. the number of measurements used in the calibration process, but will also depend on aspects of the sampling environment. For co-location measurements, there will be time-dependent fluctuations in the value ranges encountered for the predictors (e.g. low-cost sensor signals, humidity, temperature) and predictands (reference NO2, PM10). The calibration range in turn affects the performance of the calibration function: if faced with values outside its training range, the function effectively has to perform an extrapolation rather than interpolation, i.e. the function is not well constrained outside its training domain. This limitation is particularly critical for non-linear machine learning functions . Calibration performance will further vary for each device, even for sensors of the same make, due to unavoidable randomness in the sensor production process . To characterize these various influences, we here test the dependence of three machine learning calibration methods, as well as of MLR, on sample size and co-location period for a number of NO2 sensors.

The NO2 co-location data at CR7 is ideally suited for this purpose. Twenty-one sensor nodes of the same make were co-located with a LAQN reference during the period October to December 2018 (Table 1). We actually co-located 30 sensor sets at the site, but we excluded any sensors with less than 820 h (samples) after outlier removal from our evaluation. The remaining sensors measure sometimes overlapping but still distinct time periods, because each sensor measurement varied in its precise co-location start and end time and was also subject to sensor-specific periods of malfunction. To detect these malfunctions, and to exclude the corresponding samples, we removed outliers (evidenced by unrealistically large measurement signals) at the original time resolution of our measurements, i.e. <1 min and prior to hourly averaging. To detect outliers for removal, we used the median absolute deviation (MAD) method, also known as “robust Z-Score method”, which identifies outliers for each variable based on their univariate deviation from their training data median. Since the median is a robust statistic to outliers itself, it is a typically a better measure to identify outliers than, for example, a deviation from the mean. Accordingly, we excluded any samples t from the training and test data where the quantity 7Mj,t=0.6745|xj,t-x̃j|median|xj,t-x̃j| takes on values >7 for any of the predictors, where x̃j is the training data median value of each predictor. To train and cross-validate our calibration models, we took the first 820 h measured by each sensor set and split it into 600 h for training and cross-validation, leaving 220 h to measure the final skill on an out-of-sample test set. We highlight again that the test set will cover different time intervals for different sensors, meaning that further randomness is introduced in how we measure calibration skill. However, the relationships for each of the four calibration methods are learnt from exactly the same data and their predictions are also evaluated on the same data, meaning that their robustness and performance can still be directly compared. To measure calibration skill, we used two standard metrics in the form of the R2 score (coefficient of determination), defined by 8R2(y,y^)=1-∑i=1n(yi-y^i)2∑i=1n(yi-y‾)2, and the RMSE between the reference measurements y and our calibrated signals y^ on the test sets. For particularly poor calibration functions, the R2 score can take on infinitely negative values, whereas a value of 1 implies a perfect prediction. An R2 score of 0 is equivalent to a function that predicts the correct long-term time average of the data but no fluctuations therein.

As discussed in Sect. , each of AirPublic's co-location nodes measures 15 signals (the predictors or inputs) that we consider relevant for the NO2 sensor calibration against the LAQN reference signal for NO2 (the predictand or output). Each of the 15 inputs will potentially be systematically linearly or non-linearly correlated with the output, which allows us to learn a calibration function from the measurement data. Once we know this function, we should be able to make accurate predictions given new inputs to reproduce the LAQN reference. As we fit two linear and two non-linear algorithms, certain transformations of the inputs can be useful to facilitate the learning process. For example, a relationship between an input and the output might be an exponential dependence in the original time series so that applying a logarithmic transformation could lead to an approximately linear relationship that might be easier to learn for a linear regression function. We therefore compared three set-ups with different sets of predictors:

using the 15 input time series as provided (label I15);

In the same way as for NO2, we tested several calibration settings for the PM10 sensors. For this purpose, we consider the measurements for the location CarPark, where we co-located three sensors (IDs 19, 25 and 26) with a higher-cost device (Table 1). However, after data cleaning, we have only 509 and 439 samples (hours) for sensors 19 and 25 available, respectively, which our NO2 analysis above indicates is too short to obtain robust statistics for training and testing the sensors. Instead we focus our analysis on sensor 26 for which there are 1314 h of measurements available. We split these data into 400 samples for training and cross-validation, leaving 914 samples for testing the sensor calibration. Below we discuss results for various calibration configurations, using the 24 predictors for PM10 (Sect. ) and the same four regression methods as for NO2. The baseline case with just 24 predictors is named I24, following the same nomenclature as for NO2. I48 and I72 refer to the cases with additional logarithmic and exponential transformations of the predictors according to Eqs. () and (). In addition, we test the effects of environmental conditions, as expressed through relative humidity and temperature, by excluding these two variables from the calibration procedure, while using the I48 set-up with the additional log-transformed predictors.

The results of these tests are summarized in Table . For the baseline case of 24 non-transformed predictors, RFR (R2=0.70) is outperformed by ridge regression (R2=0.79) and GPR (R2=0.79). This is mainly the result of the fact that some of the pollution values measured with sensor 26 during the test period lie outside the range of values encountered at training stage. RFR cannot predict values beyond its training range (i.e. it cannot extrapolate to higher values) and can therefore not predict those values accurately see also. Instead, RFR constantly predicts the maximum value encountered during training in those cases.

However, this problem is not entirely exclusive to RFR but is inherited by all methods, with RFR only being the most prominent case. We illustrate the more general issue, which will occur in any co-location calibration setting, in Fig. . In the training data, there are not any pollution values beyond ca. 40 µgm-3, so the RFR predictions simply level off at that value. This is a serious constraint in actual field measurements where one would be particularly interested in episodes of highest pollution. We note that this effect is somewhat alleviated by using GPR and even more so by ridge regression. For the latter, this behaviour is intuitive as the linear relationships learnt by ridge will hold to a good approximation even under extrapolation to previously unseen values. However, even for ridge regression the predictions eventually deviate from the 1:1 line for the highest pollution levels. This aspect will be crucial to consider for any co-location calibration approach, as is also evident from the poor MLR performance, despite being another linear method. In addition, MLR sometimes predicts substantially negative values, producing an overall R2 score of below 0.3, whereas the machine learning methods appear to avoid the problem of negative predictions almost entirely. In conclusion, we highlight the necessity for co-location studies to ensure that maximum pollution values encountered during training and testing/deployment are as similar as possible. Extrapolations beyond 10–20 µgm-3 appear to be unreliable even if ridge regression is used as calibration algorithm, which is the best among our four methods to combat extrapolation issues.

A test with additional log-transformations (I48) of the predictors led to test score improvements for the two linear methods (Table 3), in particular for MLR (R2=0.7) but also for ridge regression (R2=0.8). This implies that the log-transformations have helped linearize certain predictor–predictand relationships. Further exponential transformations (I72) and thus also further increasing the predictor dimensionality did not lead to an improvement in calibration skill. We therefore ran one final test using the I48 set-up but without relative humidity and temperature included as predictors. This test confirmed that the sensor signals indeed experience a slight interference from humidity and temperature, at least considering the machine learning regressions. Notably, this loss of skill is not observed for MLR for which the R2 score actually improves. A likely explanation for this behaviour is the curse of dimensionality that affects MLR more significantly than the three machine learning methods, so the reduction in collinear dimensions (given the sample size constraint) is more beneficial than the information gained by including temperature and humidity in the MLR regression.

In summary, we have found that ridge regression and GPR are the two most reliable and high-performing calibration methods for the PM10 sensor. We are able to attain very good R2 scores >0.7 for all four regression methods though. An important point to highlight is the characteristics of the training domain, in particular of the pollution levels encountered during the training data measurements. If the value range is not sufficient to cover the range of interest for future measurement campaigns, then ridge regression might be the most robust choice to alleviate the underprediction of the most extreme pollution values. However, the power of extrapolation of any method is limited, so we underline the need to carefully check every training data set to see if it fulfils such crucial criteria; see also similar discussions in other calibration contexts .

Finally, we aim to address the question of site transferability, i.e. how reliably a sensor calibrated through co-location can be used to measure air pollution at a different location. One of the sensor nodes (ID 19) was used for NO2 measurements at both locations, CR7 and CR9, and was also used to measure PM10 at CR9 and CarPark, allowing us to address this question for our methodology. Note that these tests also include a shift in the time of year (Table 1), which has been hypothesized to be one potentially limiting factor in site transferability. The results of these transferability tests for PM10 (from CR9 to CarPark and vice versa) and NO2 (from CR7 to CR9 and vice versa) are shown in Figs.  and , respectively.

For PM10, we trained the regressions, using the I24 predictor set-up, on 400 h of data at either location. This emulates a situation in which, according to our results above, we limit the co-location period to a minimum number of samples required to achieve reasonable performances across all four regression methods. To mitigate issues related to extrapolation (Fig. ), we selected the last 400 h of the time series for location CarPark (Fig. a) and hours 600 to 1000 of the time series for location CR9 (Fig. b). This way we still emulate a possible minimal scenario of 400 consecutive hours of co-location, while also including near maximum and minimum pollution values within our training data (given the available measurement data). We note that alternative sampling approaches, such as random sampling with shuffling of the data, could lead to artificial effects at validation and testing stages because of autocorrelation effects that could inflate apparent calibration skill. The maximum pollution found within the two time slices differ only by ca. ±10 µgm-3, for which at least ridge regression should provide reasonable extrapolation performances. For the resulting predictions at location CarPark, using models trained on the CR9 data, we achieve generally very good R2 scores ranging between 0.67 for RFR and 0.78 for ridge regression. The site-transferred measurement performance of sensor 19 is therefore almost as good as the one for sensor 26 at the co-location site itself (Table 3); i.e. we cannot detect any significant loss in measurement performance due to the site transfer. A surprising element is that MLR performs almost as good as ridge regression in this case, whereas it performed poorly for sensor 26, where it only achieved an R2 score of 0.28 (Table 3). This underlines our previous observation that the performance of MLR is more sensitive to the specific sensor hardware, with sometimes low performance for relatively small sample sizes (Fig. ). However, our results also show again that linear methods appear to generally perform well for our PM10 sensors, with ridge regression being the most reliable and high-performing choice overall. These results are further supported by calculations concerning the detection of only the most extreme pollution events in the time series; see also the definition of such events described in the caption of Fig. . We characterize such events in the form of their statistical recall in our sensor measurements (the fraction of extreme pollution events in the reference time series that are also identified by our calibrated sensors) and precision (the fraction of extreme events identified by our sensors that are indeed also extreme events in the reference time series) and show the results as inset numbers in Figs.  and . Overall, these site-transfer PM10 results from CR9 to CarPark imply that sensors calibrated through co-location can achieve high measurement performance distant from the co-location site.

However, we do find that site transferability is not always as straightforward as found for this particular case. For example, for the inverse transfer using models trained at CarPark and predicting PM10 pollution levels at CR9, we find lower R2 scores overall. There is consistency in the sense that MLR and ridge remain the best performing methods for sensor 19 with R2 scores of around 0.5, but the sensors now miss several significant pollution events in the time series. We note, however, that many of the most extreme pollution events are still detected, which is evident from the still relatively high precision and recall scores for all methods. These results underline that, in general at least, a good performance can be achieved with co-location calibrations but that there are also significant challenges posed by site transfers. In particular, the problem is not necessarily symmetric among sites, i.e. the skill of the method can depend on the direction of the transfer, even if the pollution levels at both sites are similar. We therefore hope that our insights and results will motivate further work in this direction, with the aim to identify possible causes of such effects. We discuss some of the possible reasons for this behaviour in Sect. 4.

Similarly, we find promising results for the NO2 sensor site transfer using the I30 predictor set-up (Fig. ). The key challenge for the sensor transfer from CR7 to CR9 is that the maximal pollution levels at the two locations differ substantially, with peak concentration being around 100 µgm-3 greater at CR9. To allow for the best possible learning opportunities for the regression algorithms, we therefore used all available samples for training, which are 1482 samples at CR9 and 829 samples at CR7. This leads to overall good performance of the non-linear RFR and GPR methods at location CR7 using models trained at CR9. As no extrapolation is necessary, these methods achieve a good performance of R2 scores >0.6 and also a good balance of precision and recall. The results are, however, slightly worse than for the same site calibrations (Table 2). Ridge regression has a tendency to overpredict NO2 pollution levels in this particular case, likely because it cannot capture some non-linear effects that would have limited the prediction values. As a result, it also reproduces almost all extreme pollution events where the concentration of NO2 exceeds 45 µgm-3 (recall =0.98) but also predicts many false pollution events (precision =0.49).

Despite the large sample size, MLR performs poorly for both site transfers (R2=0.07 and -0.31). In particular, MLR underpredicts, sometimes providing even impossible negative pollution estimates at CR7, whereas it provides several runaway positive values at CR9 (Fig. ). However, at CR9 all methods struggle with the impossible challenge of extrapolation far outside their training domain, which effectively is an extreme demonstration of the effects of an ill-considered training range (cf. Fig. ). Among the machine learning methods, the effect is as expected most prominent for RFR (R2=0.36), which cannot predict any pollution values beyond those encountered at training stage. This is a serious limitation and means that the method scores nil on precision and recall of any extreme pollution events at CR9 where NO2 levels exceeded 90 µgm-3. GPR is slightly better at extrapolating beyond its training domain (compare also Fig. ) but still not good enough to reproduce any of the extreme pollution events, giving rise to equally low precision and recall. Ridge regression, as a regularized linear method, performs best in the sense that it is able to reproduce at least a few of the extreme events (recall =0.05) and predicting no false extreme events (precision =1.0), while still achieving an R2 score of 0.53. Nonetheless, it is clear from the time series in Fig.  that none of the regression methods works for this site transfer, simply because of the too large extrapolation range.