Transferability of machine-learning-based global calibration models for NO₂ and NO low-cost sensors

It is essential to accurately assess and verify the effects of air pollution on human health and the environment in order to develop effective mitigation strategies. More accurate analysis of air pollution can be achieved by utilizing a higher-density sensor network. In recent studies, the implementation of low-cost sensors has demonstrated their capability to quantify air pollution at a high spatial resolution, alleviating the problem of coarse spatial measurements associated with conventional monitoring stations. However, the reliability of such sensors is in question due to concerns about the quality and accuracy of their data. In response to these concerns, active research efforts have focused on leveraging machine learning (ML) techniques in the calibration process of low-cost sensors. These efforts demonstrate promising results for automatic calibration, which would significantly reduce the efforts and costs of traditional calibration methods and boost the low-cost sensors' performance.

As a contribution to this promising research field, this study aims to investigate the calibration transferability between identical low-cost sensor units (SUs) for NO₂ and NO using ML-based global models. Global models would further reduce calibration efforts and costs by eliminating the need for individual calibrations, especially when utilizing networks of tens or hundreds of low-cost sensors. This study employed a dataset acquired from four SUs that were located across three distinct locations within Switzerland. We also propose utilizing O₃ measurements obtained from available nearby reference stations to address the cross-sensitivity effect. This strategy aims to enhance model accuracy as most electrochemical NO₂ and NO sensors are extremely cross-sensitive to O₃. The results of this study show excellent calibration transferability between SUs located at the same site (Case A), with the average model performance being R² = 0.90 ± 0.05 and root mean square error (RMSE) = 3.4 ± 0.9 ppb for NO₂ and R² = 0.97 ± 0.02 and RMSE = 3.1 ± 0.8 ppb for NO. There is also relatively good transferability between SUs deployed at different sites (Case B), with the average performance being R² = 0.65 ± 0.08 and RMSE = 5.5 ± 0.4 ppb for NO₂ and R² = 0.82 ± 0.05 and RMSE = 5.8 ± 0.8 ppb for NO. Interestingly, the results illustrate a substantial improvement in the calibration models when integrating O₃ measurements, which is more pronounced when SUs are situated in regions characterized by elevated O₃ concentrations. Although the findings of this study are based on a specific type of sensor and sensor model, the methodology is flexible and can be applied to other low-cost sensors with different target pollutants and sensing technologies. Furthermore, this study highlights the significance of leveraging publicly available data sources to promote the reliability of low-cost air quality sensors.

1 Introduction

Interest in air quality (AQ) has increased significantly over the past decades as a result of the severe impact of air pollution on the environment and public health (WHO, 2004). Major air pollutants such as carbon monoxide (CO), nitric oxide and nitrogen dioxide (NO+NO₂ = NO_x), particulate matter (PM), and anthropogenic volatile organic compounds (VOCs) originate mainly by anthropogenic activities that directly and indirectly affect the AQ and public health (Kelly and Fussell, 2015). Consequently, monitoring and mitigating air pollution are of utmost importance in support of sustainable development. To date, the official regulatory monitoring stations use high-precision instruments based on optical measurement principles (e.g., in the chemiluminescence method in the case of NO₂) that are highly cost-intensive. The unit price for a fully equipped regulatory monitoring station varies from EUR 50 000 to 100 000, in addition to maintenance and operating costs (Mead et al., 2013; Ionascu et al., 2021). According to the current European Ambient Air Quality Directive (2008/50/EC), implemented by EU member states, the microscale siting of a monitoring station for atmospheric pollutants subject to regulatory limits has several requirements. Among these, the site should be within 10 m of the road edge and at least 25 m from high-traffic intersections. These high costs and space requirements constrain their spatial distribution to few areas. Moreover, as shown by several studies (e.g., Zhu et al., 2020; Beckwith et al., 2019; Baruah et al., 2023), NO₂ hotspots at urban sites are not fully represented by their corresponding monitoring station. In order to bridge the gap, it is crucial to increase the spatial coverage of air quality monitoring. A possible way to do this is using networks of low-cost sensors along with modeling.

The drive to promote spatial coverage of air quality monitoring, combined with advancements in sensor technology, has paved the way for the utilization of low-cost sensors in air quality monitoring (Ionascu et al., 2021). Due to their affordability, portability, and simple deployment, utilization of low-cost sensors has been widely acknowledged (Karagulian et al., 2019; Suriano and Penza, 2022; Snyder et al., 2013; Bigi et al., 2018). However, concerns about the stability of their performance and the quality of the data have significantly reduced their implementation on a large scale. Low-cost sensors for gas detection are mostly metal-oxide and electrochemical sensors (Spinelle et al., 2015; Borrego et al., 2016; Mijling et al., 2018), and when deployed in environmental conditions, they suffer from drift, cross-sensitivity, and induced bias dependent on relative humidity or temperature (Masson et al., 2015; Mueller et al., 2017; Maag et al., 2018; Tagle et al., 2020; Papaconstantinou et al., 2023). This type of sensor is generally subject to two main sources of error: internal errors arising from the sensor's working principle and external errors resulting from environmental factors. Internal errors include variable detection limits, drift, and non-linear response. Identical sensors can introduce bias even when deployed at the same site, mainly due to manufacturing tolerances. External errors are mainly attributed to environmental factors such as temperature and relative humidity, as well as cross-sensitivity to interference gases (Ionascu et al., 2021; Giordano et al., 2021). In response to certain temperatures or relative humidity levels or changes in their values, low-cost sensors can exhibit significant biases. For example, both Masson et al. (2015) and Tagle et al. (2020) reported such high biases for NO₂ electrochemical cells during periods of high relative humidity (above 75 %). Widely used NO₂ electrochemical cells have been shown to have significant cross-sensitivity to O₃ (Miech et al., 2021; Spinelle et al., 2017; Alphasense Ltd, 2022). As a solution, an O₃ scrubber was added (Hossain et al., 2016; Alphasense Ltd, 2022), and it was shown that the filter material was successful at removing O₃ without affecting the signal due to the target NO₂. Notwithstanding this scrubber, NO₂ cells still show some O₃ interference. For example, according to Miech et al. (2021), Alphasense NO₂-B43F exhibits 6.6 % cross-sensitivity to O₃, which also increases with time, as indicated by Spinelle et al. (2017). As a result, this interference induces a bias in the response.

Low-cost sensor biases can be partially mitigated through calibration, usually performed either under laboratory controlled conditions or by field co-location next to a reference monitoring station (Miech et al., 2021). Studies show that the latter approach is more satisfying and commonly used as it maximizes the performance of such sensors in real-world applications (Spinelle et al., 2017; Suriano and Penza, 2022; Kureshi et al., 2022). Successful calibration has the potential to significantly enhance the AQ measurement process and reduce overall costs (Zimmerman et al., 2018; Munir et al., 2019; Van Zoest et al., 2019). However, the type and amount of processing applied to the air quality sensor data can lead to confusion about whether the processed data remain a true sensor measurement or a blend of secondary data and predictions. To address this issue, Schneider et al. (2019) proposed a standardized terminology for processing levels of air quality low-cost sensor systems. A four-level sequence ranges from Level 0 (raw sensor output) to Level 4 (processed data with spatial interpolation or assimilation into models). Each level serves different purposes, and data usability varies depending on the application. The proposed terminology aims to enhance the use and understanding of this technology and to ensure that the methods applied are well-documented and fit for their intended purpose.

Several calibration techniques have been reported in the literature, spanning environmental factor correction (Miech et al., 2021; Van Zoest et al., 2019; Kim et al., 2018) and simple linear regression models (Okorn and Hannigan, 2021) to machine learning (ML) techniques (Nowack et al., 2021; Bigi et al., 2018; Zimmerman et al., 2018; Spinelle et al., 2015; Ionascu et al., 2021). Although some low-cost sensor outputs show an approximately linear relationship with the target pollutant, this linearity varies with time due to sensor aging (Li et al., 2021). ML algorithms have shown superior ability to interpret such complexity of low-cost sensors, especially when including covariates that account for meteorological and environmental variability. One of the most popular ML algorithms is random forest, which is an ensemble algorithm based on decision trees (Breiman, 2001). random forest, in addition to other commonly used methods such as multiple linear regression, support vector regression, and artificial neural networks, has been widely employed in air quality low-cost sensor calibration and in some aspects of atmospheric chemistry, as it tends to outperform linear regression models (Nowack et al., 2021; Bigi et al., 2018; Zimmerman et al., 2018; Spinelle et al., 2015; Ionascu et al., 2021). Most studies available in the literature investigate the individual localized calibration approach, in which a single calibration model is created for each sensor unit (SU) after being co-located with a reference instrument (Zimmerman et al., 2018; Spinelle et al., 2015). Recent works, such as Bigi et al. (2018), Sahu et al. (2021), Van Zoest et al. (2019), and Nowack et al. (2021), have studied individual calibration models considering site transferability, where they investigated whether a co-location-based calibration at one location produces reliable measurements at a different location. Bigi et al. (2018) found a performance range of about 6.5 ppb root mean square error (RMSE) for NO₂ and NO.

Only a few studies consider the calibration transferability (global calibration) among different sensors of the same make, including site transferability. A study conducted by Malings et al. (2019) evaluated the performance of individualized calibration models versus generalized calibration models. Individualized models are built based on data from a single sensor, while generalized models combine data from all sensors of the same type. The researchers found that the most effective calibration model type varied by sensor technology; for example, simpler regression models produced the best results for electrochemical CO sensors, while more complex models, such as artificial neural networks and random forest models, provided the best results for NO₂ sensors. Although the outcomes varied, it was found that generalized models performed better at new locations compared with individualized models, despite slightly lower performance during initial calibration. Vikram et al. (2019) proposed a method for improving calibration transfer of NO₂ and O₃ by training calibration models on multiple sites. They rotated nine SUs among three sites with reference monitors and introduced a novel split-neural-network (split-NN) approach which incorporates two sets of models: a global calibration model that combines data from a set of similar sensors spread across different training environments and sensor-specific calibration models that correct the sensor-to-sensor variations. The approach demonstrates versatility, accommodating linear regressors (LRs) or NNs for sensor-specific models and utilizing a two-layer NN for global calibration. The researchers found that the split-NN method performed better than random forest, reducing errors by 0 %–11 % for NO₂ and 6 %–13 % for O₃. Training their models on two sites and testing them on a third site with no overlap between the training and test data distributions (“Level2” benchmark as classified by Schneider et al., 2019) resulted in an RMSE between 6 and 8 ppb for NO₂. Another study by Okorn and Hannigan (2021) examined the transferability of simple LR calibration models between several metal-oxide sensor systems (pods), focusing on ozone and methane. In their study, calibration transferability was performed among pods within the same location (i.e., sensors here share the same environmental variability). They suggested using a standardization approach to normalize sensor signals for enhanced calibration transferability among units. A recent study by Wang et al. (2023) examined the calibration transfer performance of five low-cost SUs for PM and NO₂. The five SUs were co-located with a reference-grade monitor at one site for 4 weeks, and then two units were transferred to another site for a 16 d mobile campaign 6 months after the first deployment. The results show transferability between SUs located at the same site (same stationary settings), with the coefficient of determination (R²) of best-performing calibration models for PM exceeding 0.80 and with R² for NO₂ units ranging around 0.70. However, models trained in stationary settings are difficult to transfer to mobile settings with different environmental characteristics.

In our study, we developed global ML-based calibration models for electrochemical cells targeting NO₂ and NO, using data of low-cost SUs that were utilized in a previous study by Bigi et al. (2018). We focus on calibration transferability among SUs when deployed at the same location (i.e., the same environmental characteristics) and different locations (i.e., different environmental characteristics), given that no explicit overlap exists between the training and testing data distributions. This approach uses simple standardization to account for sensor-to-sensor variations, unlike the approach proposed by Vikram et al. (2019), which utilizes an ML-based method. In addition, this study presents potential improvements to model transferability using additional information (O₃) from nearby regulatory air quality monitoring stations. This approach assists in untangling the interference of O₃ that persists in the NO₂ cells despite the presence of an O₃ scrubber (Spinelle et al., 2017; Miech et al., 2021; Li et al., 2021). While there is abundant evidence supporting the integration of O₃ as an input variable for NO₂ calibration, as evidenced by the extensive literature (Mead et al., 2013; Miech et al., 2021; Spinelle et al., 2015), there is limited support for its inclusion in NO calibration. In this study, we present the results of this scenario, which may be of interest to researchers in this field. The incorporation of information from nearby regulatory monitoring stations is referred to as Level3 in the classification by Schneider et al. (2019). Finally this study provides an opportunity to study the influence of geographical and seasonal variations on calibration transferability.

In Sect. 2, the sensor units, deployment sites, and calibration methods are described. Results and discussion are found in Sect. 3. Finally, the main conclusions are drawn in Sect. 4. All data processing was performed with MATLAB (MathWorks, Natick, MA, USA) version R2021b.

Table 1 Details of the two-phase campaign of SU deployments.

2 Materials and methods

2.1 Sensor units

This study utilized data collected from four SUs developed jointly by Empa, the Swiss Federal Laboratories for Materials Science and Technology, and Decentlab GmbH. These SUs were described and employed in previous studies (Bigi et al., 2018; Kim et al., 2018). Each SU consists of four electrochemical sensors – two NO₂ sensors (Alphasense NO2-B43F) and two NO sensors (Alphasense NO-B4) – along with temperature (T) and relative humidity (RH) sensors (Sensirion STH21). All signals were sampled every 20 s, aggregated to a 1 min mean value, and transmitted to a central database every 180 min. The four SUs are denoted as AC009, AC010, AC011, and AC012, and the electrochemical sensors are denoted as NO_A, NO_B, NO2_A, and NO2_B, provided in millivolts. Throughout this study, signals of each electrochemical sensor represent the voltage difference between the working electrode (WE) and auxiliary electrode (AE). Data collected from the SUs and their corresponding reference instruments were preprocessed for outlier removal, smoothing, and averaging over 10 min, following the same procedure as explained in Bigi et al. (2018).

2.2 Deployment sites and co-location

Over a two-phase campaign, SUs were deployed at three locations representing different emission and meteorological conditions in continuous co-location at quality regulatory stations within the National Air Pollution Monitoring Network (NABEL) (Bigi et al., 2018). A detailed description of the two-phase campaign can be found in Table 1. The first phase began in April 2017 and lasted for approximately 3 months, during which the four SUs were installed in the rural site of Härkingen (HAE), facing a major highway. This peculiar location allowed sensors to be exposed to both traffic-related pollutants, as the southern wind carries polluted air from the highway, and cleaner air masses, as the northern wind flows over the rural area. After the first phase of the campaign was accomplished, the SUs were transferred to two different locations: AC009 and AC010 were installed in Zurich-Kaserne (ZUE), while AC011 and AC012 were installed in Lausanne (LAU). The second phase lasted for around 4 months (from 28 July to 5 December 2017). All reference instruments provide measurements for NO, NO₂, O₃, temperature, and relative humidity. Figure 1 summarizes the meteorological variables and pollutant concentrations at the different deployment sites, as measured by the reference instruments. In the vicinity of co-location site ZUE, there are four other nearby regulatory air quality monitoring stations located within an approximately 2.7 km radius. In Lausanne, there are two nearby stations situated within a radius of about 10.7 km of the co-location site (LAU), while none is available in Härkingen; see Fig. 2. These nearby monitoring stations provide O₃ measurements that are used to assess the potential enhancement of calibration models when addressing cross-sensitivity issues arising from O₃. For comparison purposes, the same assessment is conducted utilizing O₃ measurements collected from the co-location reference stations (ZUE and LAU).

Figure 1 Box plots showing meteorological variables and pollutant concentrations at deployment sites using 10 min averaged data. The central line indicates the median; the star represents the mean; and the bottom and top edges of the box indicate the 25th and 75th percentiles of the data, respectively. The whiskers extend to the minimum and maximum values.

Figure 2 Map view of low-cost SUs co-location sites featuring nearby monitoring stations, in both Zurich and Lausanne (© Google Earth 2023).

2.3 Calibration

The application of calibration transfer methods may facilitate the effort needed to obtain valuable measurements of air pollutants from low-cost sensors. Figure 3 illustrates the two cases investigated by this study to examine the transferability of calibration between different (but identical) SUs. For Case A, each global calibration model was trained on a dataset from one SU, denoted as “primary SU”, and then applied to the rest of the SUs, denoted as “secondary SUs”, available at the same location. This case is designed to examine the ideal scenario and to serve as a benchmark. For Case B, each global calibration model was applied to secondary SUs installed at different sites than the primary SU. Every SU was once a primary SU in both cases.

Figure 3 Scheme of the two cases of calibration transfer between different SUs (left) and the architecture of the global calibration model (right).

Calibration transfer approach is advantageous in networks consisting of a significant number of low-cost SUs. Instead of individually characterizing and calibrating each SU, it may suffice to characterize and calibrate a representative SU or a subset of units and then apply the acquired global calibration models to the remaining units within a network of low-cost SUs. These models can also be applied to other SUs of the same type, both those in close proximity to the calibrated units (primary SUs) (e.g., same city, similar emission conditions) and those further away (e.g., same city, differing emission conditions).

Our calibration strategy (illustrated in Fig. 3) is designed to enhance model performance by minimizing cross-sensitivity variance and sensor-to-sensor variability. As O₃ could cause significant interference in NO₂ low-cost sensors, O₃ measurements were included in the feature set of the calibration models.

2.3.1 Data investigation and preparation

Evaluating the consistency of SUs is recommended to determine whether similar electrochemical sensors respond to target changes similarly (Giordano et al., 2021). Higher consistency and reduced error sources such as sensor-to-sensor variations would pave the way for optimum transferability of calibration. Therefore, consistency was mainly assessed and addressed by (1) pairwise Pearson correlation (R) between identical low-cost sensors deployed at the same site, as shown in Table 2, where the results indicate significant correlations between the low-cost sensors; (2) Pearson correlation (R) between low-cost sensors and their corresponding reference measurements (Table 3); and (3) standardization of features because identical sensors may have different baseline levels, even if coming from the same manufacturer and deployed at the same location, as shown in Fig. 4. Therefore, to tackle this issue, standardization (Z score) was applied, in which all features have a mean of zero (μ = 0) and 1 standard deviation (σ = 1). This results in almost completely uniform signals from the electrochemical sensors, across all SUs, especially when exposed to similar environmental conditions. Overall, this reduces the sensor-to-sensor variations, making it possible for global calibration reproducibility.

Table 2 Pairwise Pearson correlation (R) between the electrochemical sensors of different SUs.

O₃ measurements acquired from the nearby monitoring stations are available at 1 h resolution; therefore, calibration models in this study are trained and tested based on 1 h data. When training a model (with primary SU data), O₃ measurements were obtained from the co-location reference stations (either ZUE or LAU). When testing the global models (with secondary SU data), (1) for Case A, O₃ measurements were obtained from the reference station within the NABEL network (either HAE, ZUE or LAU), since the secondary SUs are located at the same co-location site as primary SUs, and (2) for Case B, O₃ measurements were obtained from the nearby monitoring stations, replicating a real-world scenario in which secondary SUs are installed at a different location without being co-located with reference instruments.

Table 3 Pearson correlation (R) between electrochemical sensors of SUs and their corresponding reference instruments.

Figure 4 A comparison of raw NO2_A measurements before and after Z-score application, for each SU deployed at HAE. A negative voltage in the signal indicates that the auxiliary electrode has a higher voltage than the working electrode, which occurs, for example, if the electronic zero points in both electrodes significantly differ from each other. Applying a Z score to the raw data minimizes this artifact.

Figure 5 Daily O₃ measurements from the co-location reference station and the other nearby stations in both Zurich and Lausanne. The inset tables list the Pearson correlation (R) between each nearby station and the co-location reference stations.

O₃ measurements obtained from the co-location reference sites and other nearby monitoring stations throughout the entire year 2017 were analyzed in an effort to examine the consistency of O₃ concentrations among these stations (see Fig. 5). Analyzing the daily average of these measurements revealed a strong correlation between the co-location reference station and all nearby stations, in both Zurich and Lausanne, as depicted in the inset tables of Fig. 5. This indicates a consistent variability in O₃ across all stations at each site, implying that the positive contribution of any nearby station would enhance the model's capability to capture O₃ cross-sensitivity.

2.3.2 ML-based calibration transfer method

In this study, three different ML-based calibration algorithms were used: multivariate linear regression (MLR), support vector regression (SVR), and random forest (RF). These algorithms were employed to estimate atmospheric concentrations of NO₂ and NO based on a set of features (predictors). The choice of ML algorithms and features followed the approach by Bigi et al. (2018), as we utilized the same dataset. For the training of global calibration models for NO₂ and NO, six features were initially used: voltage signals of the four electrochemical sensors – NO_A, NO_B, NO2_A, and NO2_B; temperature; and relative humidity. Additionally, our proposal suggests incorporating O₃ obtained from nearby monitoring stations. To evaluate the influence of incorporating O₃ on global models' performance, two sets of models were formulated and assessed. One set exclusively relied on SU data as features, while the other integrated O₃ into the feature set. Following this, a comparative analysis was carried out.

The models were trained and analyzed in MATLAB utilizing the fitlm() function for MLR, the LIBSVM software package for SVR, and the TreeBagger() function for RF. A k-fold cross-validation approach was used to address overfitting, where the training dataset (primary SU) was divided into five folds (blocks) as depicted in Fig. 6. Here, we chose k = 5 based on the recommendation by Rodriguez et al. (2009). One block (20 % of the dataset) was used for validation, and the remaining blocks (80 % of the dataset) were used for training. This process was repeated five times (5 parts). In each split process, the block sampling approach introduced by Schultz et al. (2021) was followed to avoid the spurious correlation between training and validation sets. A grid search was applied to find the best hyperparameters, which were subsequently used to train the entire training set. The model was then evaluated using a test dataset (secondary SUs). In RF models, the variable (predictor) importance can be calculated by random permutation of each variable in the decision tree and averaging the estimation error over the forest (Breiman, 2001). The importance of a variable to the model increases as the estimation error increases.

Figure 6 An illustration of the k-fold cross-validation approach.

3 Results and discussion

3.1 Performance of calibration transfer

The global calibration models were evaluated using R² and RMSE as goodness-of-fit metrics, as described in Appendix A. Figure 7 summarizes the overall evaluation results for the two sets of global calibration models (with and without O₃) for NO₂ and NO in both cases. The results presented here and throughout this paper are based on the O₃ measurements obtained from “Station Z1” in Zurich and “Station L1” in Lausanne (Fig. 2). The results indicate successful transferability of the calibration models across SUs for NO₂ and NO, with Case A showing superior performance compared to Case B. In Case A, errors between the primary and secondary SUs are minimal, primarily because both SUs share the same environmental characteristics. Thus, Case A has a higher level of transferability than Case B. Moreover, the results indicate that, on average, RF consistently outperforms MLR and SVR, which aligns with the conclusions from Bigi et al. (2018), who investigated individual calibration models using the same dataset. The major outcome from this study is that global calibration models perform better when including nearby monitoring stations' O₃ measurements in the feature set. In Case A, the RF-based NO₂ models demonstrated their highest transferability performance with an R² of 0.96 and an RMSE of 2.0 ppb. The corresponding averages were 0.90 ± 0.05 for R² and 3.4 ± 0.9 ppb for RMSE. In contrast, Case B exhibited a different performance profile, with the best R² value being 0.76 and an RMSE of 5.0 ppb. The averages in Case B were 0.65 ± 0.08 for R² and 5.5 ± 0.4 ppb for RMSE. Comparing NO models to NO₂ models, the former displayed superior transferability. In Case A, the RF-based NO models achieved an impressive R² value of 0.99 and an RMSE of 1.6 ppb, along with averages of 0.97 ± 0.02 for R² and 3.1 ± 0.8 ppb for RMSE. In Case B, the best performance for NO models was characterized by an R² of 0.87 and an RMSE of 5.0 ppb, with averages of 0.82 ± 0.05 for R² and 5.8 ± 0.8 ppb for RMSE. Generally, NO models show better transferability than NO₂ models. Further details can be found in Tables S1 through S6 in the Supplement. In comparison with the existing literature such as Vikram et al. (2019) or Wang et al. (2023), these results demonstrate notable advancements.

Figure 7 Average results of evaluating the performance of global calibration models for NO and NO₂, based on MLR, SVR, and RF techniques for both Case A and Case B. O₃ measurements were obtained from Station Z1 in Zurich and Station L1 in Lausanne.

The inclusion of O₃ measurements has resulted in noteworthy enhancements in predictive accuracy and generalizability, as indicated by the increased R² values and reduced RMSE values, which are particularly pronounced in the SVR and RF models. To comprehensively assess the impact on the global models, we explored the incorporation of O₃ measurements from all nearby monitoring stations in Zurich and Lausanne. Interestingly, every station contributed positively to model performance. Figure 8 reports the average enhancements (%) in R² and RMSE for NO₂ RF global models by each nearby station in comparison with the co-location reference station, in both Zurich and Lausanne.

Figure 8 The R² and RMSE average positive improvements (%) of global NO₂ RF models in Case B, when including O₃ measurements from each nearby station in comparison with the co-location reference station in Zurich and Lausanne.

To better understand this interesting finding, we examined each global model's performance in terms of RMSE (%), as shown in Fig. 9. In Case A, notable enhancements in the performance of all RF-based models for NO₂ and NO were observed, with the NO₂ models experiencing a substantial improvement of up to 42 % and the NO models showing an improvement of up to 25 %. In contrast, Case B demonstrated more pronounced improvements when secondary SUs were located at ZUE. The RF-based models exhibited an enhancement of up to 17 % and 21 % for NO₂ and NO, respectively. Interestingly, no significant improvement was observed when the secondary SUs were located at LAU. This finding can be attributed to higher O₃ levels in ZUE (background site) compared to LAU (traffic site) (Fig. 1), leading to increased cross-sensitivity of low-cost sensors in ZUE. Consequently, the inclusion of O₃ measurements allowed the models to effectively capture and account for its influence, resulting in improved prediction accuracy. The feature importance plots (Fig. 10) provide further proof by indicating a higher significance of O₃ in the NO₂ and NO models of ZUE compared to LAU, thereby reinforcing its key role in capturing model variations.

Figure 9 An illustration of the performance enhancement achieved by incorporating O₃ for both Case A and Case B. The x axis represents primary SU → secondary SU. The relative improvement was computed using the following formula: (new − old) $/$ old × 100 %.

Figure 10 Feature importance plots for NO₂ and NO, for 1 h based measurements, including reference O₃.

The results also reveal that the transfer of NO₂ and NO calibration models to SU (AC010) resulted in the lowest performance among all secondary SUs. Table 3 provides insights into the potential reasons behind this outcome, showing that for SU (AC010), NO2_A exhibits a stronger correlation with the reference NO₂ measurements compared to NO2_B. Furthermore, the feature importance plots (Fig. 10) indicate that NO2_A has a more significant influence on predicting NO₂ than NO2_B for models trained with the primary SU (AC010), which is the opposite for the rest of the calibration models. Thus, we infer that the discrepancies in the correlation between counterpart features in the training and test datasets substantially impact the calibration transfer between SUs of the same make. Higher disparities suggest that the model may not generalize well with new data, which raises concerns about its overall performance. Accordingly, when selecting a primary SU for the final global calibration model, it is crucial to select an SU that demonstrates representative feature importance for the other SUs to which the model will be transferred.

In some cases, the poor performance of ML-based calibration models can be attributed to the nature of ML algorithms. As an example, many meteorological variables exhibit periodic variations and are correlated over time and space, with these correlations changing with time. Unfortunately, ML algorithms are unaware of these relationships and have difficulty extrapolating periodic features correctly (Grover et al., 2015). Another possible reason for poor performance is the existence of “unknown error sources”, whose influences are not captured by ML models. As a result of the spatiotemporal difference between primary and secondary SUs, different external errors are imposed on ML models, which significantly impact their performance. Therefore, future solutions of such problems can be achieved by incorporating various measures such as feature engineering, which calculates derived properties that assist ML models in recognizing the more complex relationships imposed by various environmental conditions (Schultz et al., 2021; deSouza et al., 2022).

Figure 11 Average results of evaluating the performance of global calibration models for the Modena dataset at the urban-background (UB) site and urban-traffic (UT) site.

3.2 Validity of the calibration transfer approach using a different dataset

Finally, in order to validate the reliability and effectiveness of our approach, we applied it to a different dataset collected in the town of Modena, in the Po Valley, a European air pollution hotspot. The dataset was described and investigated in a previous work by Baruah et al. (2023). This allowed us to assess its robustness in diverse scenarios and identify the conditions necessary for successful implementation. The Modena dataset consists of measurements obtained from 12 SUs deployed in Modena, Italy. Two different sites were selected for the co-location of these SUs with reference stations: an urban-background site, where NO₂, NO, and O₃ reference measurements are available, and an urban-traffic site, where only NO and NO₂ reference measurements are available. Figures S1 and S2 in the Supplement illustrate the temporal deployment of the Modena SUs and pollutant concentrations measured by the reference instruments. The deployment periods are sparsely distributed and span a period of approximately 20 months. Modena SUs were deployed for the shortest period of time at the urban-traffic site; some were deployed for around 2 weeks. This dataset can be used to validate our calibration transfer method. Modena SUs are equipped with three electrochemical sensors – NO₂ (Alphasense NO2-B43F), NO (Alphasense NO-B4), and OX (Alphasense OX-B431) – as well as temperature and relative humidity sensors. According to our calibration strategy, since that OX sensor is available, it will be utilized as a source of O₃ data. This dataset has been analyzed, and the best features combination was identified, as stated in Eq. (1).

$$\begin{array}{}\text{(1)}& \begin{array}{rl}{\mathrm{NO}}_{\mathrm{2}}=\text{function}(& \text{NO2\_we},\text{NO2\_aux},\text{NO\_we},\text{NO\_aux},\\ & \text{OX\_we},\text{OX\_aux},T,\text{RH})\\ \mathrm{NO}=\text{function}(& \text{NO2\_we},\text{NO2\_aux},\text{NO\_we},\text{NO\_aux},\\ & \text{OX\_we},\text{OX\_aux},T,\text{RH})\end{array}\end{array}$$

The correlation analysis was explored (see Table S7). According to these investigations, all NO low-cost sensors and some NO₂ and OX low-cost sensors have a very low correlation with their corresponding reference measurements at the urban-background site. Figure 11 shows results of the overall calibration transfer performance of NO₂ and NO models for the two sites. For additional details, see Figs. S3 through S7. The findings of these results can be summarized as follows: (1) there is consistency with the results from the Switzerland dataset, in which RF outperforms MLR and SVR, and calibration transfer within the same site (Case A) achieves better performance than in Case B. Also, NO models show better transferability than NO₂ models. (2) It is possible that some models were unable to be transferred, presumably due to low correlation (pairwise and with their corresponding reference measurements), which is more prominent in NO low-cost sensors at the urban-background site. Moreover, the sparse deployment of SUs at the urban-background site and the short co-location period at the urban-traffic site can affect the generalizability of global models. (3) Despite the urban-traffic measurements having a short co-location period compared to urban-background measurements, the calibration transfer of urban-traffic data performed better than that of urban-background measurements, especially in Case B.

Based on our analysis of the Modena dataset, it is evident that three main conditions are required for the proposed calibration protocol to provide the best transferability of calibration models: (1) high correlation (pairwise, as well as with the reference measurements); (2) a sufficient period of co-location; and (3) using multiple electrochemical sensors dedicated to the same pollutant, such as the Switzerland dataset, which can enhance data reliability. This claim is supported by several studies. For example, the study by Bigi et al. (2018) showed that using a pair of sensors for NO₂ and NO led to better performance in their calibration models compared to using a single sensor. Moreover, Smith et al. (2019) reported the effectiveness of employing an array of sensors rather than a single sensor. They utilized the instantaneous median signal from six identical electrochemical sensors for NO₂ and O₃, resulting in minimized random drifts and inter-sensor differences, thus addressing some limitations of individual sensors.

4 Conclusions

This study investigated the transferability of ML-based calibration models for NO₂ and NO across identical low-cost SUs deployed at similar and distant locations within Switzerland. Moreover, this study advocated enhancing NO₂ and NO global calibration models by incorporating O₃ measurements from available nearby monitoring stations. This strategic augmentation aims at effectively mitigating the cross-sensitivity issues associated with low-cost sensors in the absence of dedicated O₃ low-cost sensors (i.e., OX sensors), which is expected to improve the model's performance. The results of this study showed excellent calibration transferability between SUs located at the same site (Case A), with the average performance of RF-based models being R² = 0.90 ± 0.05 and RMSE = 3.4 ± 0.9 ppb for NO₂ and R² = 0.97 ± 0.02 and RMSE = 3.1 ± 0.8 ppb for NO. The results also showed good transferability between SUs deployed at distant locations (Case B), which resulted in an average performance of R² = 0.65 ± 0.08 and RMSE = 5.5 ± 0.4 ppb for NO₂ and R² = 0.82 ± 0.05 and RMSE = 5.8 ± 0.8 ppb for NO. These results reveal notable advancements compared to the existing literature.

Our study indicates that to achieve optimal performance of the global calibration model, there should be a strong correlation between sensors and their corresponding reference stations. Additionally, similar pollutant levels should be observed at both primary and secondary SU locations, as certain machine learning algorithms cannot extrapolate beyond the training data range. Employing multiple electrochemical cells within each SU targeting the same pollutant might be useful in enhancing data reliability, with caution required to prevent potential overfitting. Although this study demonstrated enhanced performance of NO calibration models by incorporating O₃, there is limited evidence in the literature to support this inclusion for NO.

To conclude, the outcomes of our study will provide novel insights into the capability of ML models to generalize calibration models and emphasize the importance of utilizing publicly available data sources to improve the reliability of low-cost air quality sensors.

Appendix A: Evaluation metrics and raw results of calibration transfer approach

Three parameters were used to evaluate the overall performance of the calibration performance: R², RMSE, and mean absolute error (MAE), given in Eqs. (A1)–(A3), respectively (Jolliff et al., 2009).

$$\begin{array}{}\text{(A1)}& {\displaystyle }{R}^{\mathrm{2}}=\mathrm{1}-{\displaystyle \frac{{\sum }_{i=\mathrm{1}}^n{\left(y_i-{\widehat{y}}_i\right)}^{\mathrm{2}}}{{\sum }_{i=\mathrm{1}}^n{\left(y_i-{\stackrel{\mathrm{‾}}{y}}_i\right)}^{\mathrm{2}}}},\text{(A2)}& {\displaystyle }\text{RMSE}=\sqrt{{\displaystyle \frac{\mathrm{1}}{n}}\sum _{i=\mathrm{1}}^n{\left({\widehat{y}}_i-y_i\right)}^{\mathrm{2}}},\text{(A3)}& {\displaystyle }\text{MAE}={\displaystyle \frac{\mathrm{1}}{n}}\sum _{i=\mathrm{1}}^n\left|{\widehat{y}}_i-y_i\right|,\end{array}$$

where y denotes the reference measurements, $\widehat{y}$ is the predicted values by the calibration model, and $\stackrel{\mathrm{‾}}{y}$ is the mean of reference values. R² values range between 0 and 1, measuring how much the independent variables (features) can explain the variation in the dependent variable (i.e., reference measurements). RMSE and MAE quantify the deviation between the calibrated values and their corresponding reference values.