Automatic pollen recognition with the Rapid-E particle counter: the first-level procedure, experience and next steps

Pollen-induced allergies are among the most prevalent non-contagious diseases, with about a quarter of the European population being sensitive to various atmospheric bioaerosols. In most European countries, pollen information is based on a weekly-cycle Hirst-type pollen trap method. This method is labour-intensive and requires narrow specialized abilities and substantial time, so that the pollen data are always delayed and subject to sampling- and counting-related uncertainties. Emerging new approaches to automatic pollen monitoring can, in principle, allow for real-time availability of the data with no human involvement.

The goal of the current paper is to evaluate the capabilities of the new Plair Rapid-E pollen monitor and to construct a first-level pollen recognition algorithm. The evaluation was performed for three devices located in Lithuania, Serbia and Switzerland, with independent calibration data and classification algorithms. The Rapid-E output data include multi-angle scattering images and the fluorescence spectra recorded at several times for each particle reaching the device. Both modalities of the Rapid-E output were treated with artificial neural networks (ANNs) and the results were combined to obtain the pollen type. For the first classification experiment, the monitor was challenged with a large variety of pollen types and the quality of many-to-many classification was evaluated. It was shown that in this case, both scattering- and fluorescence-based recognition algorithms fall short of acceptable quality. The combinations of these algorithms performed better, exceeding 80 % accuracy for 5 out of 11 species. Fluorescence spectra showed similarities among different species, ending up with three well-resolved groups: (Alnus, Corylus, Betula and Quercus), (Salix and Populus) and (Festuca, Artemisia and Juniperus). Within these groups, pollen is practically indistinguishable for the first-level recognition procedure. Construction of multistep algorithms with sequential discrimination of pollen inside each group seems to be one of the possible ways forward. In order to connect the classification experiment to existing technology, a short comparison with the Hirst measurements is presented and the issue of false positive pollen detections by Rapid-E is discussed.

1 Introduction

Pollen of many wind-pollinated plants has specific proteins that cause human allergies (Valenta et al., 1992; Bousquet et al., 2006, 2015; Radauer and Breiteneder, 2006; Choual et al., 2018), particularly affecting children (Skoner, 2001; Høst et al., 2003; Douladiris et al., 2018). Stress due to contact with the pollen-contained allergen can cause an allergic reaction or exacerbate some related diseases (Leynaert et al., 2000; Devillier et al., 2017; Poethko-Müller et al., 2018). Allergies impair the quality of life of about 30 % of the world population (Akdis et al., 2015). In most European countries, national organizations of various kinds provide information about pollen concentration in the air, publish pollen prognoses and issue warnings. The bulk of such efforts is based on retrospective pollen observations and climatological pollen calendars. Most observers use Hirst-type volumetric pollen traps where airborne particles (> 5 µm) are collected on a rotating drum covered by Melinex tape. Samples are identified by a microscopic analysis (Galán et al., 2014; Buters et al., 2018). This method is labour-intensive and tedious, requires narrow specialization abilities, and incorporates significant uncertainties (e.g. Oteros et al., 2017). In addition, due to the manual treatment of the collected samples and weekly cycle of the trap, the data are always delayed from a few days up to a few weeks. However, timely data about pollen concentration in the air are also needed for improving the accuracy of tools for personalized medicine (for example, PASYFO app, http://www.pasyfo.lt, last access: 14 November 2018; POLLEN app, http://www.polleninfo.org, last access: 14 November 2018; NORKKO forecast and app http://www.norkko.fi, last access: 14 November 2018; etc.) (Bousquet et al., 2017; Horgan and Pazzagli, 2017; Pereira et al., 2018; Tabatabaian and Casale, 2018). It can also be used for informing people about the current pollen concentration in the air. Finally, real-time data are needed for short-term pollen forecasts with statistical and atmospheric dispersion models (Sofiev et al., 2013, 2015, 2017; Prank et al., 2016; Ritenberga et al., 2016; Zink et al., 2017).

As the approach to information and personal responsibility for health is changing, it has become a necessity to develop new methods enabling the information on airborne pollen to become available in real time. The first attempts to obtain automated information were related to image recognition technologies (Bennett, 1990). Their development was accompanied by the formation of more potential possibilities (Ronneberger et al., 2002; Landsmeer, 2009). Currently, two types of technologies seem to be the most suitable for taxon-level classification of pollen: based on image recognition and laser fluorescence (or their combinations). Image-based technologies are used in detectors, such as BAA500 (Helmut Hund GmbH, https://www.hund.de); the laser-fluorescence-based approach is implemented in a wideband integrated bioaerosol sensor (WIBS) device (http://www.dropletmeasurement.com, last access: 19 November 2018), PA-300 and Rapid-E (Plair, http://www.Plair.ch, last access: 19 November 2018), whereas the new Poleno device (Swisens, https://swisens.ch/, last access: 19 November 2018) aims at integration of both features. The Hund- and Plair-manufactured devices were used in limited-scale scientific studies, Oteros et al. (2015) for BAA500 and Crouzy et al. (2016) for PA-300, and showed promising results. However, the large-scale evaluation and calibration suitable for European-scale applications have yet to be concluded (Oteros et al., 2015; Crouzy et al., 2016).

The goal of the current paper is to evaluate the capabilities of the new Plair Rapid-E pollen monitor and to construct and evaluate the first-level pollen recognition algorithms using particle scattering and fluorescent data from Rapid-E. The key questions to answer were as follows:

• can we identify different pollen genera using the Rapid-E data?

• can we identify different species within the same pollen genus?

• what is the recognition accuracy for the most common pollen types in Europe?

The experiment was performed in Šiauliai (Šiauliai University and Finnish Meteorological Institute), Novi Sad (BioSense Institute of the University of Novi Sad) and Payerne (Federal Office of Meteorology and Climatology MeteoSwiss) with three newly acquired experimental Rapid-E devices. The devices were provided with local pollen samples and several pollen recognition algorithms have been constructed independently in each centre. This organization of the study allowed us to account for variability of the actual technical characteristics of the individual devices and an absence of “good practice” for such types of measurements. The best classification results compared across the centres formed the basis of of this paper. Finally, outputs of the MeteoSwiss classifier are compared with airborne pollen data collected with the Hirst-type pollen trap. The provided time series were used to discuss the “false positive” identifications important for the operational context.

2 Methods

2.1 Description of the measurement instrument

The new Rapid-E instrument designed and produced by Plair SA is the successor of the first-generation particle analyser PA-300 used by Crouzy et al. (2016). It is a particle counter; i.e. it analyses all particles coming to its inlet one by one. Operation of the instrument is based on two physical principles: scattering of near-UV laser beam and deep-UV laser-induced fluorescence (Kiselev et al., 2011, 2013). Multi-angle scattering is used for determination of the particle's morphology, such as size and shape. The fluorescent light is analysed for its spectrum and lifetime. The instrument constantly takes in the ambient air through the air inlet on the top of its panel. Sample air flow is up to 2.8 L min⁻¹ with the counting rate of up to 4500 particle detections per minute; i.e. the theoretical saturation level is 1.6×10⁶ particles m⁻³. Since according to the device provider the smallest observable particle is 0.5 µm in diameter, this saturation level will not be reached in realistic ambient conditions.

The sampled air enters the nozzle, which creates a laminar flow in the measurement zone. Particles interact with the 400 nm laser light source and the scattered light is captured by 24 time-resolving detectors distributed at different angles. The information on chemical properties of the particles is obtained by a powerful deep-UV laser (320 nm) source that induces fluorescence. Its spectrum (32 measuring channels within a spectral range of 350–800 nm, eight sequential acquisitions with 500 ns retention) and lifetime (four particular bands) are recorded and used for the particle identification (Fig. 1).

Figure 1 Examples of scattering images, fluorescence spectra and lifetimes of selected pollen types.

The threshold of the particle fluorescence intensity (> 1500 units) was empirically determined as a cut-off level for sufficiently recorded pollen grains. The spectra were subsequently normalized to eliminate the difference in the signal strength between the instruments. Rapid-E has an embedded mechanism for collecting the particles, which passed through the registration chamber onto sticky slides for the follow-up microscopic analysis.

The device has several modes of operation. Since the deep-UV laser has a limited resource, the 400 nm scattering image is used for prior estimation of the particle morphology and deciding if it can be pollen. In the pollen mode, the device ignites the deep-UV laser only for the 5–100 µm particle size range (used in this study). Another mode allows detection of particles in the range of 0.5–100 µm for spores, particulate matter and bacteria identification. However, the expected lifetime of the deep-UV laser is much shorter in this mode, especially in polluted atmosphere.

2.2 Data processing and recognition methods

2.2.1 Šiauliai

Both modalities of the Rapid-E output (scattering image and the fluorescence spectra) were processed independently with artificial neural networks (ANNs) and the scores were merged to obtain the final classification result.

Both scattering and fluorescence signals (Fig. 1) significantly depend on the particle position with regard to the laser beam while passing through it. In particular, the apparent particle size (scattering) and the fluorescence intensity varied between the recordings. Apart from that, 15 %–50 % of particles are missed by the deep-UV laser. Therefore, pre-processing included (i) identification of a characteristic template of 44×20 pixels from the scattering image to localize the features characteristic for each pollen type; (ii) filtering of particles with insufficient fluorescence intensity (Table 1); (iii) normalization of fluorescence spectrum, (iv) at the first time moment, inclusion of only 16 of 32 wavelengths in the feature vector to exclude the saturated short-wavelength fluorescence bands.

Table 1 Pollen used for testing the identification capabilities of the instrument in Šiauliai.

^* The particle fluorescence intensity level > 1500 at the Rapid-E scale for at least one emitted wavelength. The initial number of pollen noticed by the scattering laser is not used in the analysis. The algorithms were based on data of fluorescent particles. Calibration datasets were normalized.

Several artificial neural networks (ANNs) were created. One of the best-performing networks included only scattering and fluorescence signals, taking them separately and disregarding the noisy lifetime component.

ANN for scattering images consists of two convolutional blocks for the feature extraction and two fully connected layers for classification (Fig. 2).

Figure 2 Neural network for pollen classification in Šiauliai based on separately treated scattering and fluorescence signals.

Every convolutional block consists of the 2-D convolutional layer, the batch normalization layer, the rectified linear unit (ReLU) activation layer and the max-pooling layer. One mask of the convolutional layer has a size of 5×5. The convolutional layer of the first block has 16 filters, and the one of the second block has 32 filters. The max-pooling layer selects the maximal response from the area of 2×2. At the output of the second convolutional block, the size of the feature vector is 1760. The first fully connected layer has 256 neurons. The second fully connected layer classifies these vectors to the number of pollen classes chosen for the calibration. The ANN was trained using the cross-entropy loss criterion. The fluorescence spectrum was processed by a multilayer perceptron ANN (Fig. 2) with dropout and batch normalization layers used for regularization. This ANN was also trained using the cross-entropy loss criterion. Results of two ANNs were fused by summing scores of every pollen type. The training process was monitored to avoid overfitting the networks – see the Discussion section.

2.2.2 Novi Sad

All Rapid-E signals (i.e. scattering, fluorescence and lifetime) were transformed into images and jointly processed by a single ANN (Fig. 3). Its architecture considers the same input dimensions of every image, and since the scatter signal could vary in the number of acquisitions, each image's width was equalized by finding its centre of mass and either cutting or zero-padding to fit to 24×70 pixels. The dynamic range of each image was reduced by replacing each pixel value with its logarithm, which resulted in enhancement of the low-intensity pixels. Images from temporally resolved spectrum data and all bands of the lifetime data were used unprocessed.

Similarly to Šiauliai, particles with a fluorescence intensity less than 1500 units at the Rapid-E scale at all wavelengths were filtered out. In addition, particles with calculated optical size out of the range of 5–100 µm were filtered out using the manufacturer's size approximation, depending on the sum of the scattering image. Size is 0.5 µm if the sum is less than 5 500 000. If the sum is between 5 500 000 and 500 000 000, the size is given by $\mathrm{9.95}\times {\mathrm{10}}^{-\mathrm{1}}\cdot \mathrm{np}.\log (\mathrm{3.81}\times {\mathrm{10}}^{-\mathrm{5}}\cdot x)-\mathrm{4.84}\times \mathrm{10}$. Finally, if the sum is greater than 500 000 000, the size is given by $\mathrm{0.0004}\times x^{\mathrm{0.5}}-\mathrm{3.9}$.

Each input signal is analysed with its own chain consisting of 2-D convolutional layers, replication padding layers, ReLU activation functions, batch normalization layers, max pooling and dropout layers, together forming the convolutional block (Fig. 3).

Figure 3 Neural network for pollen classification in Novi Sad using all three signals.

The convolutional layer of the first scattering block had 10 filters with the kernel size of 5×5 while the second one had 20 filters with the kernel size of 3×3. For the spectral images, the convolutional layer of the first block had 50 filters with the kernel size of 5×5, and that of the second block had 100 filters with the kernel size of 3×3. For the lifetime images, the first convolutional layer had 70 filters with a kernel size of 7×1, the second one had 140 filters with a kernel size of 5×1 and that of the final block had 200 filters with the kernel size of 3×3. At the output of the final convolutional block, the sizes of the feature vectors for scattering image, fluorescence spectrum and lifetime are 1800, 1600 and 1400, respectively. Each feature vector is passed through one fully connected layer with 50 neurons. Those features were concatenated, resulting in the feature vector of dimension of 150. The size of the second (last) fully connected layer was the same as the number of classes, after which the samples were classified with the log-softmax activation function.

The ANN was trained using negative log-likelihood (NLL) loss and the stochastic gradient descent with a learning rate of 0.001 and a momentum of 0.9.

2.2.3 Payerne

At the preprocessing stage, all three signals were normalized with their maxima. For scattering, the image was additionally centred and cut to a 24×100 shape. Extra filtering was imposed, retaining only calibrations with an optical size above 10 µm and a fluorescence signal in a range and spectrum compatible with single pollen grains (see Crouzy et al., 2016, for examples of spectra). The optical size corresponding to 10 µm was estimated by comparing the integral of the scattering signal of 5 µm polystyrene latex particles (PSLs) with the integral of the scattering signal for Urtica and Parietaria pollen grains.

For scattering ANN, 5×5 convolutions were applied with 32 filters, ReLU activation and the pooling layers with a 2×2 window. For lifetime, 1-D convolution was applied with ReLU activation, with a window size of 10×1 and 10 filters. For the spectrometer, asymmetric 2×4 convolution was applied with eight filters with ReLU activation. The ANN was trained using the Adam optimizer and categorical cross-entropy as loss function (Fig. 4).

Figure 4 Neural network used for classification at Payerne.

In order to retain flexibility, additional features were inserted before the final fully connected layers after Crouzy et al. (2016): the maximum and the integral of the scattering together with the maxima of each of the four lifetime bands and the maxima of the first three spectrum acquisitions.

2.2.4 Additional filtering of false positives in operational context

Even if high expectations on the performance of the classifier are met, problems are bound to occur in the operational applications due to false positive detections. For example, birch pollen concentrations regularly exceed 1000 pollen m⁻³ in Switzerland in spring. If just 2 % of these are misinterpreted as ambrosia pollen, for example, the false concentration of 20 grains per cubic metre would already be significant for allergy analysis. In order to cope with this, extra steps were introduced in Payerne. Additional filtering was applied disregarding the events with classification quality below a certain threshold as in Crouzy et al. (2016), where a reduction of sampling of 20 % led to an increase in precision of about 10 %.

For the operational monitoring, at least a few events with an extremely good classification score were required during the same or two preceding days to accept the middle-confidence recognition of the specific pollen type. This condition is applied uniformly over the pollen season to verify what pollen taxa are present in the air.

2.3 The scheme of the experiment

In this section, we present in detail how the calibration experiment was implemented in Šiauliai, followed by the description of specifics of the setups in Novi Sad and Payerne. Most importantly, only pollen characteristic for each location was used. Comparison of the results was based on pollen types belonging to the same plant families found in all three locations.

2.3.1 Šiauliai

The experiment in Šiauliai was carried out with 14 pollen morphotypes, the tested amounts of which are given in Table 1. Three genera (Salix, Acer, Pinus) were represented by two plant types. All 14 plants are naturally widespread in Lithuania and their airborne pollen is abundantly recorded annually (Šaulienė et al., 2016).

Pollen was taken from the plant inflorescences collected during the vegetation period in April–August of 2018 during the days with intense pollen release. The collected material was put in air-permeable paper bags and dried at a temperature of 40 ^∘C until the maximum release of pollen from the inflorescences. A vibratory sieve shaker Analysette 3 PRO was used for gentle shaking of the pollen grains out of the inflorescences. The extracted pollen was stored in petri dishes at +4 ^∘C.

Each experiment was performed twice and consisted of up to eight sample tests, using approximately 5 mg of pollen per sample test. The number of grains registered in the scattering signal is indicated in Table 1 as “total particles”, whereas the column “fluorescent particles” shows the number of grains with usable fluorescent and lifetime signals.

Figure 5 The scheme of the experiment for identification of pollen.

The experiments were carried out in laboratory conditions with a self-designed manual exposure method (Fig. 5). In order to isolate the environment of the experiment from the ambient particles, a plastic (PET) bottle was fitted tightly to the Rapid-E inlet. One of the bottle walls was cut open and two holes of ∼15 cm² were covered with a household air filter. The filter fabric was tested to hold ∼99 % of particles larger than 1 µm in diameter without any noticeable disturbance of the air inflow into the device.

The pollen was injected into the upper part of the bottle by inserting the pipette tip with the pollen sample into the narrow cut in the bottle and then gently blowing the air through the pipette. With the sampling rate up to 2.8 L min⁻¹, Rapid-E was collecting the pollen grains from the bottle within a few tens of seconds. This simple scheme enabled reduction of the environmental sample contamination by up to 5 times compared to the unfiltered air in the lab. Each new experiment used a new bottle and the nozzle of the instrument was cleaned, thus ensuring the removal of previously sampled pollen.

Figure 6 Fluorescence spectra (first acquisition) of Salix, Pinus and Acer species.

Quality and level of contamination of the samples was manually controlled by using sticky slides. The presence of non-pollen particles (debris from the remnants of inflorescences) was verified to be substantially less than 1 % by the visual inspection of a subset of the calibration events. Abundance of pollen aggregates (several pollen grains stuck together) was also low but their reliable identification by microscopic analysis was more difficult because of the thick layer of pollen on the slides. The calibration was performed in pollen mode, which excluded particles smaller than 5 µm of optical size.

2.3.2 Novi Sad

The scheme of the pollen exposure experiment was similar to that in Šiauliai. The exposure was conducted on the roof by fitting the PET bottle to the sampling pipe after removing the Sigma-2 inlet. Manual microscopic analysis of sticky slides was used to confirm the quality of samples and absence of non-pollen debris and pollen agglomerates. The device was also in pollen mode; i.e. it filtered out particles smaller than 5 µm of optical size.

Classification was tested for an adjusted set of pollen morphotypes accounting for the availability of the fresh material during the study season. In particular, Juniperus was replaced by Taxus and Festuca was replaced by Cynodon and Dactylis, aiming to assess the degree of discrimination between different grass genera. Similarly, Picea and Cedrus pollen was used for assessing differences between the same pollen morphotype. Only Acer negundo was analysed as it is the only Acer pollen that is recorded regularly in Serbia. Fraxinus (including both F. excelsior and F. ornus) was added to the test as it is commonly recorded throughout spring.

2.3.3 Payerne

Low ambient concentration of coarse particles allowed a less laborious approach: pollen calibrations in Payerne were performed by directly blowing the material into the Sigma 2 inlet, without protection from contamination. The details of the procedure are described in Crouzy et al. (2016). In order to obtain a reasonable panel of the relevant pollen types, 60 calibrations were performed for 21 different taxa. Focus was set on repeating calibrations, if possible under varying conditions. Only fresh pollen was used and time between collection and calibration was reduced to a minimum (range: 15–120 min). The presence of agglomerates and debris was investigated by collecting histograms of the optical size and of the fluorescence intensity of the recorded events. Cut-offs were introduced accordingly, in order to retain only single pollen grains. The device was also in pollen mode; i.e. it filtered out particles smaller than 5 µm of optical size.

Figure 7 Comparison of fluorescence indicators of the tested pollen. The blue line represents the first acquisition. All other lines are acquisitions delayed by a step of 500 ns from the last. Shadows show the standard deviation ranges for each acquisition. In the figures, the x axis represents the wavelength, nm. The y axis shows the amplitude; units not available.

3 Results obtained in Šiauliai

The analysis was started from a semi-qualitative consideration of the fluorescence spectra, primarily aiming at demonstration of the capabilities and the limitations of the approach and preliminarily assessing the principal possibility of constructing a reliable particle recognition algorithm.

3.1 Qualitative comparison of the fluorescence spectra of different pollen species

3.1.1 Comparison of fluorescence spectra of different species of the same genus

The experiment included three genera, for which we collected pollen from different species (Table 1): Salix, Pinus and Acer. Their fluorescence spectra are shown in Fig. 6, where the solid lines represent the normalized mean spectrum and shadows show the standard deviation range. The uncertainties of the mean spectra were a fraction of a percentage, leading to the statistically significant difference (p<0.001) at all wavelengths for both Pinus and Acer mean spectra and even for some wavelengths of the Salix spectra.

Despite statistically significant differences between the mean spectra, the sample standard deviation (shadowed ranges in Fig. 6) was quite large. Therefore, it was not possible to distinguish between Salix alba and Salix fragilis. The normalized spectra of Pinus sylvestris and Pinus mugo coincided at the maximum value of the amplitudes at the wavelength of 460 nm but the mean amplitude of the Pinus sylvestris spectrum was higher in the short-wave range (< 450 nm). At the longer wavelengths (480–550 nm) the amplitude was higher for the Pinus mugo pollen. However, these differences were well inside the sample standard deviation. The difference between the species of the Acer genus was the most pronounced and, even taking the sample variability into account, these were the ones that could be distinguished. The Acer pseudoplatanus spectrum showed a higher amplitude than Acer negundo in the short-wave range and lower amplitude in the central part of the spectrum (400–520 nm).

Therefore, two out of three tested genera allowed, in principle, an inter-genus species classification using the pollen fluorescence spectrum. However, the differences between them were evidently too small for the multispecies algorithm considered in the current paper. Practical work was therefore left for the follow-up studies.

3.1.2 Comparison of fluorescence spectra of species of different genera

The study included 11 different pollen genera (Table 1), whose spectra are shown in Fig. 7 for recordings at every 500 ns starting from the first pulse reaching the detector. For all species, the most intense fluorescence was observed for the wavelengths from 390 to 570 nm, with different locations of the maximum and with different amplitude. For example, the highest mean intensity of fluorescence was recorded for the Artemisia pollen: it exceeded > 7000. Meanwhile, the amplitudes of Betula and Quercus reached more than 4000. In all cases, the first pulse had a wider wavelength range than the subsequent ones. The amplitudes of already the second recording (500 ns from the first pulse) were close to zero for wavelengths longer than 600 nm.

In addition, Fig. 7 shows that not only the intensity of the first signal between separate genera differs, but the shape of the second recording is also specific, which is significant for the identification of the pollen morphotype. For example, the difference in fluorescence intensity of Salix pollen between the first and second signals was larger than for other tested taxa. Tests with Festuca pollen actually showed that, unlike all other species, the signal amplitude grows during the first 500 ns, resulting in the absolute maximum intensity of the spectrum registered at the second recording, 500 ns after the fluorescence is induced.

The qualitative analysis of the data was continued by grouping the data according to similarity of the fluorescence spectrum of the first recording (Fig. 8).

Figure 8 Groups with similar fluorescence spectra.

Alnus, Corylus and Betula plants are in one taxonomic family, and our results indicate that their pollen has a similar fluorescence spectrum. Interestingly, according to the similarities of the fluorescence spectra, Quercus pollen appeared in the same group with Betulaceae, although the maximum value of its mean of the normalized spectrum was the lowest in the group.

Another group in which the pollen fluorescence curves have similar shapes also consists of pollen of woody plants: Populus and Salix. They also bloom at a similar time; therefore their precise identification is an important but, as seen from Fig. 8, challenging task. The tested grass pollens form a separate group, which however also included pollen of the woody plant Juniperus. This group is characterized by the high mean amplitude in the short-wavelength (< 400 nm) range.

3.2 Recognition skills

The key practical question for Rapid-E application in daily pollen monitoring is the accuracy of the pollen type classification presented below via the confusion matrices. In these matrices, rows represent the actual type of pollen and columns are the assigned type. All values are a percentage, and the sum of values over each row is 100 %: every pollen has to be assigned to some type.

3.2.1 Comparison of the confusion tables obtained in Novi Sad, Payerne and Šiauliai

The recognition procedure in Šiauliai was built independently for scattering and fluorescence signals with subsequent fusion of the results. The tables for the individual components are presented in the Supplement. Tables S1 and S2 in the Supplement present the outcome of the combined identification using both the scattering image and the fluorescence spectra. With the exception of Alnus, the combination of the identification methods showed better recognition skills than each of the methods separately. Overall, the improvement over individual methods was ∼23 % compared to scattering images and ∼7 % compared to fluorescence.

Overall, 6 out of 11 tested pollen genera were identified with accuracy better than 75 %. The best results (> 91 % of correct classification) were achieved for Pinus pollen. The pollen of the Betulaceae genus was identified comparatively well but the recognition of the individual species of this family was poor.

The overall accuracy was very similar for Šiauliai and Novi Sad and somewhat better for Payerne, partially owing to the stricter filtering of the raw data. Although it is difficult to make an exact comparison of the confusion tables between the studies, it still sheds some light on the overall performance and also highlights the similarities and differences between the regions. Comparing the Tables 2 and 3, one can see that the difference in the recognition quality is about 10 % for most species, and practically identical for Betula (∼50 % in both studies) and Quercus (∼60 %).

Table 2 Confusion table obtained in Šiauliai. Accuracy is 73 %.

Table 3 Confusion table obtained in Novi Sad. Accuracy is 74 % (obs. different number of species).

Somewhat higher skill in Novi Sad was obtained for Corylus, Alnus and Populus while in Šiauliai higher skill was reached for Acer and Artemisia. It is interesting to note that the confusion between the two chosen grass pollen morphotypes in Novi Sad was not notable and for these genera the Rapid-E data have certain discrimination potential. One can therefore conclude that the multispecies discrimination algorithms applied in these studies showed similar recognition skills. It should be stressed however that the training of the ANNs was completely independent and used local pollen grains. Therefore, the similar recognition quality does not imply similar pollen in these regions.

As mentioned earlier (Sect. 2.2.3), the calibration procedure used at MeteoSwiss was slightly different than in Novi Sad and in Šiauliai. In addition, the focus at MeteoSwiss was more towards operational applications. As a consequence, only a subset of the 60 calibrations was used to train the classifier. Only taxa with high relevance for monitoring or for which very good calibrations were available were selected. It was noticed that increasing the number of taxa could worsen the problem of false positive detections (see below). An optimum for monitoring purposes was found when using 10 taxa. The performance of the corresponding classifier is shown in Table 4.

Table 4 Confusion table obtained at MeteoSwiss, Payerne. Accuracy is 80 %.

It is interesting to note that, as expected, most errors occur within the Betulaceae family, with an extremely low recall for Alnus. It was hypothesized that, although calibrations were repeated, the classifier may to some extent recognize the conditions under which the calibration was performed and quality of the sample. Obtaining a classifier working only on the generic features of the taxa is a very difficult task. A holistic validation procedure, going from the analysis of device raw output (Sect. 3.1) analysis to the comparison with reference measurements (Sect. 4.5), is therefore essential.

4 Discussion

4.1 Overtraining – a problem?

The problem of potential overtraining was addressed from two directions: via the standard training vs. test dataset evaluation and via an explicit verification of homogeneity of the datasets.

4.1.1 Performance in the training and test datasets

Prior to starting the ANN training, all datasets were split into the training and test subsets. The test subset in Šiauliai consisted of 1000 particles picked at the end of every calibration event while all other particles were used for training. The Šiauliai ANN training continued until saturation of the recognition quality for the training dataset (see example in Fig. 9), thus including the overfitting range. The maximum performance of the fluorescence-based recognition was obtained at the epoch of ∼900, after which the overfitting gradually picked up. Therefore, the ANN parameters after this epoch were taken as the study outcome. For the scattering-image-based training, a similar consideration suggested the epoch 3500 as the optimum.

Figure 9 Šiauliai ANN multispecies cost function for scatter-based (a) and fluorescence-based (b) recognition as a function of the training epoch.

Figure 10 Novi Sad ANN overall cost as a function of the training epoch. The evaluation error is lower than the training error due to dropout (0.5) in each convolutional and fully connected layer, not used in the validation round.

For Novi Sad (Fig. 10), the training was stopped before the overfitting picked up and thus the parameters of the last trained epoch 3000 were used.

4.1.2 Test of homogeneity of the calibration datasets

One of the concerns regarding the fluorescence-based technology is the stability of the spectra for different conditions of pollen grains, which are affected by ambient humidity, temperature, time they spend in the air, chemical interaction and degradation, etc. Full-scale evaluation of this problem lies beyond the scope of this paper. Here, we only present a brief check demonstrating that it was not the major issue.

As stated in the methodological section, the calibration set for each pollen type in Šiauliai consisted of up to eight independent calibration sessions for four species; these sessions were performed on different days and thus with pollen of different ages. A simple check of homogeneity of the fluorescence spectra is then to use the data of one of these days as the training set and those from another day as the test subset. A substantial difference in the recognition quality would point at the inhomogeneous data. Distinguishing the four species is simpler than 11 but the difference between the training and test recognition quality is important.

Figure 11 Performance of the Šiauliai ANN for the test subset taken from different days than the calibration subset. Values are percentages.

Comparing the upper and lower rows of Fig. 11, one can see that for the above epochs (3500 for scattering- and 900 for fluorescence-based ANNs), the quality of recognition for the training subset (one day) and test subset (another day) differs by < 5 % for all four species. Therefore, we conclude that the conditions during the different days of calibration did not affect the homogeneity of the dataset.

4.2 Comparison with other studies on pollen recognition

During recent years, a number of attempts to obtain information about pollen concentration in the air in real time have been undertaken. However, even the most successful tests carried out with WIBS-4 (O'Connor et al., 2014), Hund BAA500 (Oteros et a., 2015), Yamatronics KH-3000-01 (Kawashima et al., 2017) and Plair PA-300 (Crouzy et al., 2016) devices, strongly advancing the pollen monitoring field, left open the questions of scalability and replicability of the results. They also did not touch the topics related to application of the tested systems in the operational context. Application of yet another new device – Plair Rapid-E – in our study pursued, apart from the scientific objectives, operational implementation as a mid- to long-term goal. However, having tested 14 different pollen morphotypes, we found that significant work is still needed.

One of the challenges to the automatic monitors is the rich mixture of pollen types in Europe that all pose significant allergenic threat. This makes it particularly difficult for the monitors to satisfy the needs of allergic people and allergists – unlike in many other regions. For instance, Cryptomeria japonica identified more than 10 years ago by Kawashima et al. (2007) is still the main pollen type recognized by that system (Kawashima et al., 2007; Wang et al., 2013, 2014; Takahashi et al., 2018). However, it seems to be more or less sufficient for that region.

Varying levels of allergenicity of species within a single genus or a family raise the question of whether intra-genus classification is possible. Hirst-based manual techniques do not allow it: pollen grains are too similar in the microscopic analysis. Our results show that such a level of identification is not immediately possible using Rapid-E information either. In particular, our data demonstrated that the fluorescence spectra of the Salix alba and Salix fragilis species were almost identical. More promising were the experiments with Pinus and Acer (Fig. 6) and some grasses (Table 3) where the work should be continued with different identification algorithms built for these very species after their separation from other pollen types. Other genera should also be tested.

Fluorescence spectra can be similar not only between species of a particular genus but also between different families. Several groups of otherwise unconnected species manifested very similar spectra, to a degree that did not allow their reliable differentiation (Fig. 8). Similar results were obtained in the studies conducted by O'Connor et al. (2011). They assessed the fluorescence spectrum of pollen of the Betulaceae family and stated that “birch and alder spectra closely resemble each other although there is a possibility that the birch pollen is less fluorescent than alder”. Our results show that in the case of Alnus, the fluorescence amplitudes are higher than of Betula but the spectra are indeed similar.

Similar spectra of Salix and Populus pollen (Fig. 7) also resulted in poor differentiation between them. At the same time, the degree of confusion was higher for Populus than for Salix (Tables 2–4). This is in agreement with the results obtained with Hund BAA500 by Oteros et al. (2015), who identified Salix pollen as the worst of pollen types analysed (Oteros et al., 2015). The BAA500 algorithm is based on recognition of the particle shape, which can be weakly related to the scattering images in our study – the very part that showed substantial confusion of almost all studied pollens with Salix and Populus.

Crouzy et al. (2016) suggested that a non-zero fluorescence amplitude around the 600 nm wavelength is incompatible with pollen from the Betulaceae family (Alnus, Carpinus, Corylus and Betula) but could possibly be observed for grass pollen (Dactylis and Phleum) (Crouzy et al., 2016). Our results support this suggestion and in addition the test in Novi Sad shows that ANN could show some discriminatory power between Dactylis and Cynodon. Noteworthy, recognition of the herbaceous plants (Festuca, Artemisia) was also considerably better than that of pollen of the Betulaceae family in Šiauliai (Table 2).

One can note that the recognition accuracy of this study (just above 70 %) is in an apparent contradiction to the results of Crouzy et al. (2016), where the skill was significantly higher: 91 % was obtained with PA-300. However, there are several important differences between the approaches. Firstly, the pre-filtering of the particles is substantially stricter and about 20 % of classifications were filtered out as uncertain (failed the threshold of the classification quality). Secondly, the accuracy of the recognition depends significantly on the number of pollen morphotypes used for the test (eight by Crouzy et al., 2016). In an extreme case, automated discrimination of just one species (Cryptomeria japonica) from non-pollen particles using KH-3000 was already high 10 years ago (Kawashima et al., 2007, 2017). Similarly, the high fraction of BAA500 true positive counts (93.3 %) against manual analysis of individual species by Oteros et al. (2015) went down to 65 % when the recognition of 13 pollen morphotypes was requested. It took an additional training of the algorithm to raise it up to the same 72 % as in our study. Finally, it should also be noted that PA-300 delivers fewer parameters than Rapid-E, possibly making it difficult to identify the important combinations in the raw signal in a single-level many-to-many identification task. Application of additional levels of the discrimination filters can improve the results.

4.3 Possible ways to improve the recognition skills

The dependence of the recognition quality on the number of categories is one of the directions of future research. It may be possible to consider independent groups of pollens that never (or very rarely) appear in the air at the same time – but it can make the algorithm place-specific. It is vital however to obtain improvement of the algorithm for reliable separation of pollens that can be in the air together (e.g. Betulaceae, Quercus and the like).

Figure 12 Comparison between automatic (Plair Rapid-E) and manual (Hirst-type) pollen counts for Betula, Fraxinus, Pinaceae and Poaceae.

Considering improvements of the recognition algorithms, Matsuda and Kawashima (2018) suggested the “extract window” method of analysis of the scattering images, which enabled us to distinguish unique ranges of light-scattering intensities for each taxon. However, the reliability of the algorithm is known only for five pollen morphotypes. Development of this and similar approaches for the Rapid-E scattering images may eventually improve this line of analysis and, subsequently, push up the overall scores.

Since the output of the ANN can be transformed to give a vector of probabilities, where each element i of the vector represents the probability that the sample belongs to class c_i, we expect improvement of the classification accuracy if we demand that the classification occurs only if the highest probability in that vector is greater than some probability threshold, but with the price of discarding the samples below the defined threshold. This direction was initially explored by Crouzy et al. (2016) and showed high potential: discarding 20 % of samples led to an increase in precision of about 10 % (see also Sect. 2.2.4).

As a more radical approach, one can challenge the usage of ANNs without a priori relations derived from physical or chemical features of each pollen type. Even generic considerations of scattering and fluorescence theories might hint at quantities, which show enhanced contrast in comparison with the raw data. This idea was tested in the Payerne algorithm and showed its potential.

4.4 Lessons from the comparison of the Hirst and Rapid-E measurements

Comparison of Rapid-E of MeteoSwiss with the operational Hirst measurements in Payerne from February to June 2018 extended the results of Crouzy et al. (2016) to more important taxa (Fig. 12), but also showed that robust determination of the sampling still needs to be achieved.

In order to obtain pollen concentrations, large particles presenting bimodal fluorescence spectra with position and intensity of maxima compatible with the observations made from calibrations (see Sect. 3.1) were first selected. Then, the classifier presented in Sect. 2.2.3 was applied. The effective sampling of Rapid-E is the result of a series of physical and algorithmic processes: the sampling efficiency of the Sigma-2 head, the imperfect targeting by lasers and the drop-offs due to the below-threshold classification. In Fig. 12, the Rapid-E data are scaled with species-dependent factors (constant over the season), bringing the seasonal mean to that of the Hirst time series. The issue deserves attention since, as shown by the Novi Sad results, tightening the thresholds improves the recognition skills but increases the drop-offs at the recognition stage. Sampling with Poaceae is the highest, Pinaceae presents a 2 % decrease in sampling and Betula presents a 33 % decrease in sampling with respect to Poaceae. False positives are a significant issue with Fraxinus: the sampling is dramatically reduced (75 %) for higher fluorescence thresholds. As a consequence of those limitations, the results presented here should not be taken as a complete demonstration of operational capabilities.

The suppression of the false positive detections as described in Sect. 2.2.4 worked quite efficiently but still an evident false positive event resulting from the Betula misinterpretation as Poaceae is visible in the beginning of April. Further work is required to completely remove such events, and, as a last resort, expert supervision could be used in an operational setup.

4.5 Opinion of the Rapid-E producer

During the work, we have been in periodic contact with the Plair company regarding features and issues of the Rapid-E devices used by our groups. With the paper finalized, we asked for their feedback.

Denis Kiselev, Plair, said

Our impression concerning the presented material is mixed. While I see some positive and encouraging results, my main critiques would be addressed to your calibration sets, whose cleaning and filtering falls short of the actual needs. Our results are 5 %–10 % better without overfitting the data or other special processing. The time series Plair gets for “problematic pollens” like Betula, Corylus and Alnus are actually very good. Our goal is to obtain high-quality time series calculated in real time by the instruments and the good calibration is essential for that.

We agree with importance of the calibration datasets; the procedures ensuring their quality are described in the Methodology section and further explained in the discussion above. Note that our groups were working largely independently using local pollen and original methods of the data collection and processing. Therefore, the similarities in the observed features provide additional support for our conclusions. Unfortunately, details of the Plair analysis were not available when the paper was prepared. Therefore, independent evaluation of that algorithm against the common criteria described in this paper was not possible.

5 Conclusions

We conducted the first analysis of the pollen monitoring capabilities of the new automatic pollen detector Plair Rapid-E. Using the very limited data pre-processing and basic ANN classification it was shown that, if comparatively large numbers of pollen types are considered, stand-alone scattering- and fluorescence-based recognition algorithms fail to produce reliable results for the majority of species. The combination of these algorithms performed better, exceeding 80 % accuracy for 5 out of 11 species. Therefore, this combination can be considered the first-stage classification of pollen types. It should be followed by more in-depth discrimination efforts, also including lifetime of florescence in the classification model.

The fluorescence spectra showed similarities among several tested species ending up with three groups: (Alnus, Corylus, Betula and Quercus), (Salix and Populus) and (Festuca, Artemisia and Juniperus) – as identified from the Šiauliai data. The classification between the groups was comparatively easy. Attempts to distinguish between the species of the same genus showed certain potential for some genera but more work is needed.

The results obtained in Šiauliai and Novi Sad with a very similar experimental setup but independent analysis, showed comparable results confirming the overall conclusions. They also pointed out certain limitations of replicability of the raw data features between the devices, which will require an additional conversion step to make them compatible. In this line, the comparison performed at MeteoSwiss shows a reasonable potential for automatic monitoring of important taxa; however it is not clear to which extent algorithms can be transposed from one device to another.

The in-depth discussion and improvement of the methodology and the extension to more taxa goes beyond the scope of this paper. We decided to communicate the current results early, as well as the methods developed independently by the three teams currently working with the Rapid-E counters, in order to stimulate parallel developments by the user community of the Rapid-E devices. The emergence of such a community is a good opportunity to address generalization and replicability of the device-specific results. We also believe that moving from expert supervision or calendar methods to the approach presented here and based only on device outputs for elimination of false positive detections could be of help for other automatic monitoring systems.

• Among the main challenges to be resolved in future work, the most important ones are

• to obtain reliable recognition skills at least for the pollen types that can be in the air at the same time

• to reach full replicability of the algorithms and results across the different copies of the same monitors (we are thankful to the Plair team for suggesting the scripts addressing this problem, which are now under evaluation)

• to resolve specific questions related to the algorithm construction and training – including the minimal sample volume, problems of over- and underfitting, preprocessing and pre-filtering of the data, false positive identifications, etc.

Successful resolution of these questions will open the way for wide applications of the automatic particle counters for pollen observations.