SMEARcore – modular data infrastructure for atmospheric measurement stations

We present the SMEARcore data infrastructure framework: a collection of modular programs and processing workflows intended for measurement stations and campaigns as a real-time data analysis and management platform. SMEARcore enables new SMEAR (Station for Measuring Ecosystem–Atmosphere Relations) stations to be integrated in a way that is consistent with existing stations and transfers the existing data curation experience to the new station. It establishes robust data pipelines that allow easier diagnosis of problems. We show practical examples of how SMEARcore is utilized at operational measurement stations. This work differs from earlier similar concepts, such as those used at stations within ACTRIS (Aerosols, Clouds and Trace Gases Research Infrastructure) and ICOS (Integrated Carbon Observation System) networks, in three important aspects: firstly, by keeping all the processing under the control of the data owners; secondly, by providing tools for making data interoperable in general instead of harmonizing a particular set of instruments; and thirdly, by being extensible to new instruments. As such it is not meant as a replacement for these infrastructures but to be used in addition to them and to bring structured data curation to more measurement stations not yet using these practices.

1 Introduction

The volume of environmental data doubles faster than every 2 years (Guo, 2017). Atmospheric composition is continuously monitored with a combination of satellite remote sensing (e.g., Drusch et al., 2012; Beamish et al., 2020) and comprehensive in situ observations (e.g., Kulmala, 2018; Petäjä et al., 2021). The data are integrated and synthesized in a suite of Earth system models (e.g., Hurrell et al., 2013; Randall et al., 2018). There are ambitions for developing a digital twin of the whole Earth system (Bauer et al., 2021), which would enable upscaling, incorporation of human actions, and taking advantage of advances in digital information technology to provide solutions for a sustainable future.

Managing the big data related to the atmosphere is a challenge. Here we place a focus on ground-based in situ atmospheric observations. In this field, recent technological advances and particularly a wide use of online atmospheric high-resolution mass spectrometry allow us to determine concentrations of trace gases and the chemical composition of atmospheric aerosol particles with unprecedented accuracy (e.g., Junninen et al., 2010; Yao et al., 2018; Wang et al., 2020). At the same time, there is a constant need for observations at a higher spatial resolution and therefore more stations that provide targeted observations for the region, e.g., to tackle specific issues related to air quality or climate change (Kulmala, 2018). More stations and instruments also mean more data. Doing as much processing as possible at the station can aid in the management of the volume of data. Modern atmospheric observations are not single observation points but are operated in a network, providing harmonized and high-quality data (Laj et al., 2020). To take full advantage of these observations, it is imperative that these systems are well defined and documented. The measurements need to be watched for measurement instrumentation, hardware malfunctions, and anomalies. Measurement systems must also be flexible and ease changes in hardware, personnel, and software as these are inevitable in practice.

On the European scale, topic-specific research infrastructures have been set up to provide harmonized observations, such as the Integrated Carbon Observation System (ICOS; Yver-Kwok et al., 2021) and Aerosols, Cloud, and Trace Gases Research Infrastructure (ACTRIS; e.g., Pandolfi et al., 2018). A global perspective is available through the World Meteorological Organization's Global Atmospheric Watch (WMO GAW; Laj et al., 2020). Comprehensive and co-located European infrastructures are endorsed by the SMEAR (Station for Measuring Ecosystem–Atmosphere Relations) concept (Hari et al., 2016; Kulmala, 2018). Such stations can be tailored to tackle different grand challenges, such as air quality (e.g., Liu et al., 2020) or climate change (Hari and Kulmala, 2005; Noe et al., 2015).

The various large-scale networks provide mutually different and network-specific standard operating procedures for the stations, ensuring harmonized end-user data in their thematic context. The journey from raw data to the data formats provided to end users is often very labor-intensive and done by different people for different instruments. Documenting the steps taken to process the raw data clearly and reproducibly is not simple. The traceability of data deteriorates further when it is used in scientific articles, where reproducibility is a known problem (Buck, 2015). Simply put, there can be no reproducibility without proper documentation of what was done. Data analysis, in this paper meaning any process from raw data to products such as end-user data or diagnostics, often lacks such rigor. Developing documented workflows for situations not covered by large-scale network protocols is a problem many stations need to solve.

We introduce SMEARcore, which aims to answer the problems of data management pertinent to both experimental campaigns and long-term measurement stations such as the SMEAR stations (Hari and Kulmala, 2005). It provides a set of modular tools for acquiring, transporting, indexing, monitoring, storing, and analyzing data. SMEARcore also provides a consistent interface to access the collected data. This enables the development of other programs on top of it or embedding the results in, for example, web pages. The interfaces also enable building small networks on top of SMEARcore directly. We illustrate the key features of SMEARcore and show that a station running SMEARcore will provide a streamlined data pipeline from the instruments, measurement computers, and databases all the way to the end user. SMEARcore features functionalities that provide station managers with real-time updates on data quality; data collection problems; and the status of instruments, measurement computers, and accessories. We store these metadata for further analysis. This allows us to automatically make calibrations and visualizations, which aid in identifying and solving problems with data collection. This standardization of operations and analysis allows us to do science faster and more reliably, and there is a continuous process of supporting metadata and documentation generation for future reference. SMEARcore provides a flexible and scalable framework that can be applied at the instrument, station, or multiple-station level.

SMEARcore has four main goals: collect raw data from disparate sources, monitor and display this process, provide access to these raw data in a common format for further analysis, and do routine analysis to aid operators. We decided to make a modular architecture, which allows us to use already-existing software solutions whenever possible. It also allows us to program on top of stable interfaces so that the modules are replaceable, and indeed our adjustments to different stations swap the implementation of modules. We hope that in the future this will also allow independent development of data infrastructures based on our interfaces.

2 Workflow

To effectively operate and expand a network of atmospheric stations, the observations need to be supported by coherent data and document management. This is to keep the processes from observational raw data to data products as simple as possible. There seem to be no common tools for getting from measurements into well-structured data that could be widely used in the atmospheric science community. Thus, creating a consistent set of processing tools for collecting and processing station data is necessary, and in some subsets of measurements this is already being done (e.g., Mammarella et al., 2016).

Data management is more than making consistent calculations. Any system can experience errors, and a typical measurement station has many things that can go wrong. If errors are not detected, it is not possible to intervene in time and data are lost. It is critical to detect errors promptly, and that requires automation of the detection. This monitoring should watch the data transfers, computing hardware, and calculations in addition to the measurements.

Figure 1 Workflow of the processing of a raw file of time series data and creating derived data products from it. The data file is collected, parsed, and stored; then various further processing can be done to it to create derived data. The differently colored dashed boxes show which implementation part of SMEARcore is involved in each processing step. The implementation parts are explained in Sect. 3. The black-bordered boxes are steps in the workflow. The black dot on the right of the figure marks the end of the workflow.

What does one need from a station-scale solution when systemically collecting observational data? In general, the requirements can be summarized in six categories:

1. the ability to collect raw data from several measurement computers;

2. performing routine analysis that combines several measurements, such as inversions;

3. storing raw and derived data for an intermittent period for analysis and visualization;

4. displaying these data to the people conducting measurements for quality control;

5. transferring data to long-term storage as backup and vacating local space for new data;

6. logging and presenting file collection metadata.

It should be noted that within SMEARcore we assume that the raw data for the analysis are provided by a combination of sensors, instruments, and the associated data acquisition software producing a raw data file. This means we leave making these raw data files up to the data acquisition software.

A SMEARcore installation is defined by a set of computations, defined as workflows, implementing these steps for a set of instruments and analyses and the backing computational infrastructure. We will now go through some of these workflows to explain what we mean by this concept. It should be noted that the individual workflows are simple, since they are focused on a single purpose. Any complex analysis will use the results of previous workflows, and the challenge is mostly in coordinating their execution. One usually also includes checks to avoid duplicating work already done in earlier workflows, but these are omitted in this paper for clarity. The underlying technical solutions will be described in later sections of this paper.

2.1 Time series data

This is the simplest case of data processing in SMEARcore. The process is visualized in Fig. 1. We read the raw data in and provide them for visualization and long-term storage in different forms. The input data from different instruments differ mostly in how the instrument-specific raw data format should be interpreted and parsed for visualization, meaning most instruments can be handled by SMEARcore almost identically. A practical example of such a data process is acquiring the total submicron aerosol number concentration from a condensation particle counter (e.g., Mordas et al., 2008). Conceptually this dataset is a time series of parameters with native averaging from the instrument. The data streams include a timestamp, number concentration, information on the time resolution, and relevant metadata for the instrument and measurement location.

We often create derived datasets from our raw data, for example to do calibrations or calculate new variables. This is done in the workflow in Fig. 1 by having a node that uses previously collected raw data. In practice, plots and collecting several datasets for export in different formats also conform to this pattern, as they are just data products.

Figure 2 A schematic picture of the different modules and interfaces between them in SMEARcore. The implementation within the box depends on the available hardware on the station, while the interface allows the implementation to change. For example, the analysis does not care where the data are stored if the interface allows for retrieving them. The connections with the circle and cup represent the interfaces between the components. The labels in the interface refer to the technologies currently supported in at least one of our SMEARcore installations. The technology abbreviations are hypertext transfer protocol secure (https), secure shell (SSH), secure file copy (SCP), simple storage service (S3), structured query language (SQL), and influx query language (InfluxQL). These are all protocols or languages the components use to transfer data. Rsync is the name of a file transfer program and protocol associated with it. InfluxDB is a database program, and Flux is a scripting language for it.

2.1.1 Examples of workflows

An example of producing derived datasets is the inversion of the raw data files produced by the differential mobility particle sizer (DMPS; for more details see, e.g., Aalto et al., 2001, and Kulmala et al., 2012). The DMPS selects aerosol particles based on their electrical mobility using a differential mobility analyzer (DMA). The concentration of differently sized particles is measured by a condensation particle counter (CPC). Therefore, the primary data measured by a DMPS are particle concentration from the CPC at different operating voltages of the DMA, and the voltages need to be converted into a size range. The auxiliary data (various flow rates, pressures, and temperatures) which are needed for this inversion are usually stored in the same raw data file (for more details on the inversion process of DMPS raw data, see, e.g., Kulmala et al., 2012). This means the required procedure corresponds to the workflow in Fig. 1, with the inversion function handling the processing node. Section 3 explains how the various parts are implemented in SMEARcore.

Another similar, independent, workflow is the processing of flux data from eddy-covariance (EC) measurements. The EC is a technique which uses high-frequency measurements of wind and atmospheric variables (e.g., CO₂, H₂O or particle concentrations) for calculating vertical turbulent fluxes between atmosphere and Earth surface. The collected data files are 10 Hz raw measurement data, and the EC flux is calculated from the covariance of the fluctuating components of vertical wind and the quantity of interest over some representative time window (typically 30 min). The EC data are further processed with the help of auxiliary meteorological data. The creation of derived data involves applying several data processing methods such as detrending, despiking, coordinate rotation, dilution correction and covariance calculations (for detailed descriptions of these methods, see Mammarella et al., 2016). Most atmospheric data processing implemented within SMEARcore can be abstracted to such branching workflows.

2.1.2 Metadata and conventions

Metadata are used to interpret data. Metadata consist of information such as measurement units, calibrations, and column names. We also create metadata about file processing, such as when files were processed, their sizes, where they were saved and what data files were combined to produce the resulting file.

There are several ways to store metadata. Some file formats such as NASA-AMES used by ACTRIS, or hierarchical data format (HDF) files store it in the data file itself. For us this would result in duplication of the metadata, and it is not well suited for file processing metadata. We store the metadata as database tables and link to files, as necessary. It is possible to export metadata from the databases to file formats when required by workflows.

3 Practical execution

In this section we describe the software and hardware used for SMEARcore. Any centralized solution needs at least three things: a server, storage space and a network connecting the computers. The server handles the computing work involved in collecting, indexing, and serving the data to end users. It also provides the platform for any data analysis tasks defined by the workflows. An overview of the system is shown in Fig. 2.

3.1 Software

SMEARcore is built mostly with Python, which is a general-purpose programming language. The choice was based on the permissive license and availability of relevant libraries for data handling and analysis. Python also allows one to call other programs via command line interfaces, which extends our available options by using analysis and processing codes written in other languages.

The entire system is packaged as Docker containers, allowing easy installation to a single server via Docker Compose or on any Kubernetes-enabled platform with a series of configuration files. Additional configuration is required to set secrets such as passwords and various routing options and select the systems' different modular components.

3.1.1 Storage

There are two kinds of storage we need. First is the local storage. This is used to store the files while they are being processed. The other is long-term storage which is used to save the files when they are not in active use. These can be on the same device or service. Long-term storage may also be completely external such as another service or offline backups. These storage options act as the repository of all the measurement data in the system.

We can use multiple backends from local disks to cloud-based S3 storage. For local installations, failure protection is a desirable feature, so setting up RAID (redundant array of independent disks) for the storage disks and a regular backup schedule are recommended.

3.1.2 Databases

Databases are used for monitoring the service itself, what files have been collected, what workflows have been run on which data files, and what was produced. They also feed time series data to the plotting software. This means the databases store the status of the station.

We use three different databases: InfluxDB is a time series database that can be integrated with visualization software and holds the time series data. PostgreSQL and MongoDB, both database programs, hold the information used to coordinate running the workflows. SQLite, which is another database program, is used in some versions to hold information about which files have been processed, but this can also be done with the other two databases. The raw data are stored in the original files and accessed on demand. This is due to a variety of formats, which are not suited for column storage, such as multidimensional fields.

3.1.3 Visualization

For online visualizations we are using Grafana (Grafana, 2022). It provides a simple web interface with multiple views that the user can customize. Multiple views allow us to get an overview of the stations' health and enable the diagnosis of specific problems by consulting the details from the interface. By default, views are configured to show the status of the measurement computers, as measured by configurable monitoring scripts, and the status of the data collection, as indexed by the SMEARcore database. The interface also allows one to set alarm levels to get a quick notification of the station status.

3.1.4 Data collection

We have currently implemented four different data collection methods for different circumstances:

1. The first is SSH (secure shell) access and rsync-based transport. This is our first choice since it allows the server to control transport, but the measurement computer needs an SSH server to allow incoming connections.

2. The second is SCP (secure file copy) transport from the measurement computer with scripted WinSCP, an SCP program.

3. The third is shared folders and scripted copying to them on the measurement computer.

4. The fourth is storage into an S3 filesystem.

Options 2–4 are similar, since the measurement computer handles transport and it is difficult to change without physically going to the computer. They are sometimes necessary due to installation limitations. All forms of transport offer security via passwords or key-based authentication. They also allow us to isolate instruments so that each has their own folders, and they cannot even accidentally overwrite other data. This provides protections from unintentional data corruption and allows easy management when instruments are changed or the people responsible change.

3.1.5 Analysis

The analysis part of the software runs on top of Apache Airflow (Airflow, 2022). This software allows us to define workflows such as calibrations and inversions based on multiple measurement files and run them on a schedule. The software also comes with visualizations of the states of the workflows and possible failures. The workflows themselves are composed of Python functions.

There is also an alternative implementation that is done by launching containers directly to do the analysis. In this case each workflow is self-contained.

3.2 Hardware

The server and storage can be co-located and thus far have been in our installations at Estonia (Noe et al., 2015), Beijing (Liu et al., 2020), and the Arctic Ocean on board RV Polarstern (AWI, 2017) during the MOSAiC (Multidisciplinary drifting Observatory for the Study of Arctic Climate) campaign (see, e.g., Krumpen et al., 2021). In these cases, both roles were fulfilled by a network-attached storage (NAS) system that supports container virtualization. In general, any computer that supports container virtualization and has enough storage can work as the server.

Figure 3 Online visualization of the number–size distribution of positive (top panel) and negative (bottom panel) air ions measured with a Neutral cluster and Air Ion Spectrometer (NAIS) at Järvselja SMEAR station on 6–7 June 2021 (all times in the figures are local time). The y axis displays the ion mobility in units of sV cm⁻² on a logarithmic scale. The color scale indicates the number concentration with units of $\mathrm{d}N/\mathrm{d}{\text{log}}_{\mathrm{10}}\left(\mathrm{d}p\right)$ cm⁻³. The user can select the period of interest and set the color scales to look at different concentration ranges as they wish.

The hardware parts can also be distributed and run on any cloud provider or external server, allowing functionality without any extra hardware at the station. However, this solution requires a robust network connection from the station. This is the case with our SMEAR III (Järvi et al., 2009) installation, where we use a cloud platform provided by CSC (the Finnish IT Center for Science).

The choice of hardware boils down to how much processing needs to be done and how many data need to be stored. It is also possible to separate the data collection and do postprocessing on any other platform as is traditionally done. One part of the hardware must be well planned, the local network infrastructure. Since the network serves as the primary way of both transporting the data and watching the station, throttling or disruptions in the network result in degradation of the service. It is possible to run the system without access to the internet, but a local network is still necessary. In practice this usually means setting up routers and a wired ethernet connection between the computers.

Figure 4 Concentrations of atmospheric ions with 1 nm (green line) and 2 nm (yellow line) geometric mean diameter cluster ions and 3 nm (cyan line) intermediate ions measured at the Järvselja SMEAR station during 6–7 June 2021 measured by the cluster ion counter (CIC). Negative ions are shown in the top panel and positive ions in the bottom panel. The plots are linked, and the user can select any period they want from the database.

4 Implementations

4.1 Case study: SMEAR Estonia

The first installation was for the SMEAR station in Järvselja, Estonia (Noe et al., 2015). It uses a centralized server on location and rsync agents installed on the measurement computers so that data can be pulled by the server with data collection method 1 outlined in Sect. 3.1.4. SMEARcore software containers are run on this server with Docker and are defined using Docker Compose. The data collection and parsing workflows are organized as data acquisition units (DAQs), one pair for each monitored instrument type. The pairs are coordinated using RabbitMQ message queues and a MongoDB database for persistence. SMEAR Estonia offers visualization and metadata, such as collected filenames and times, about the collection process through Grafana. There is also direct access to the collected files using sftp (secure file transfer protocol) and data transfer to off-site storage at the University of Tartu high-performance computing center.

At the SMEAR Estonia station there are currently nine instruments which are monitored by SMEARcore at two locations, a main container, and a separate measurement cottage. The instruments measure aerosol properties, meteorological parameters, and background radiation. SMEARcore is integrated with the mass spectrometer data analysis software tofTools (Junninen et al., 2010) and allows time-of-flight mass spectrometer data to be processed in near real time (once per hour). This makes it possible to present online processed data as concentrations of chemical groups rather than intensities of single peaks.

Figure 5 Chemical ionization atmospheric pressure interface time-of-flight (CI-APiTOF) mass spectrometer online data of highly oxidized organic molecule (HOM) concentrations measured at the SMEAR Estonia station during 6–7 June 2021. In the legend capital letters denote chemical elements present in the molecule, “mon” denotes monomers and “dim” dimers, and the prefix “sq” denotes that HOMs are formed from sesquiterpenes. The others are formed from monoterpenes. Both sesquiterpenes and monoterpenes are volatile compounds emitted by vegetation. Inorganic acid concentrations could be presented similarly or even added to the plot by the user as needed. The processing done by SMEARcore allows showing these concentrations instead of raw counts from the instrument.

The case study is from measurements in the SMEAR Estonia station from the period 6 to 7 June 2021. The figures presented show screenshots from instrument-specific real-time dashboards. During this period, we see two daytime new particle formation (NPF) events and a nighttime clustering event at 20:00 on 6 June 2021 – 02:00 LT (local time) on 7 June 2021 (Figs. 3 and 4). The NPF events are seen in the number–size distributions measured by the Neutral cluster and Air Ion Spectrometer (NAIS) as formation of initially 2 nm air ions and their subsequent growth to larger sizes over several hours (Fig. 3). The NAIS data visualization is done with a custom plugin developed at the SMEAR station. The color map and the number concentration scale can be changed by the user for their preferred viewing experience. The concentrations of sub-2 nm cluster ions measured by the cluster ion counter (CIC) show diurnal variation, and the concentration of 3 nm ions shows a maximum during NPF events (Fig. 4). During a nighttime clustering event, the atmospheric pressure interface time-of-flight mass spectrometer (CI-APiTOF) observed a simultaneous increase in highly oxidized organic molecules (HOM) dimer concentrations (Fig. 5). Similar nighttime clustering events producing small (sub-10 nm) particles which do not grow into larger particles as in daytime NPF events have been observed also at SMEAR II station in Hyytiälä, Finland (Rose et al., 2018).

Figure 6 Instrument monitoring parameters for the APiTOF are plotted online. Alerts are triggered if values are out of the operation range. The orange and green arrows in the pressure graph indicate that an alert was present.

Figure 7 Data-syncing statistics from several instruments. Red and green indicate missing and mistimed data, respectively. This allows for the detection of problems in the measurement and network.

4.1.1 Utilizing metadata

Figures 6 and 7 show dashboards on the instrument level and station level. An example dashboard from the CI-APiTOF mass spectrometer is shown in Fig. 6. For the instrument it is important that pressures in different chambers are in the correct range; too low pressure in the first chambers (SSQ and BSQ) indicates clogging of the orifice, and mechanical cleaning is required, while too high values again indicate malfunction of pumps, and pump maintenance is required. In the dashboard monitoring, values change colors from magenta (too low) to green (correct) to red (too high). In addition to current parameter readings, time series of pressure readings are also displayed; this helps to identify the reasons for the problem and to see when the problem surfaced. In the case of critical operational parameters, an alert is given on the screen and an email is sent to the operator.

Various auxiliary measurements can be constantly monitored to ensure the integrity of the measurement devices. Alerts call attention to abnormal readings and can be collected into figures such as Fig. 7. In this dashboard successful file readings are marked with background color, but timing problems due to slow internet or intranet speeds are marked with green color and completely missing files are marked with red. Figure 7 shows current problematic measurements, like with GammaTRACER and RADOS Cottage and longer-lasting problems like with TSI Flow Järvselja, which is not at the station, and thus the data are missing. When the instrument is fixed and brought back to the station, the status will return to normal.

In addition to measurement metadata, we collect metadata from the data collection and parsing processes themselves. Last file access times, file sizes, counts of parsed columns, and how long the parsing took are all things that are monitored. Figure 8 shows an example of how these data can be used to determine the effect of a power cut on collected data. Input voltage UPS (uninterruptible power supply) indicates if the UPS is being charged; if there is no input voltage, then the system is operated from backup power. Gaps in graphs on 12 and 13 January are the result of such a long power cut that communication with the station was also interrupted. The file size is also smaller due to limited measurement time (Fig. 8).

4.2 Case study: SMEAR III

Our newest installation is an implementation of parts of the SMEAR III (Järvi et al., 2009) data analysis. The SMEAR III station is located at the University of Helsinki Kumpula campus in Helsinki, Finland. The instruments included are the DMPS; basic meteorological instrumentation such as temperature, wind direction, and wind speed; and trace gas measurement of ozone, carbon monoxide, and nitrogen oxide. The storage backend is S3 in the CSC – IT Center for Science cloud service, with a computer simply pushing new files there as they are generated. The data are on a set retention policy, which means that old data are cleared automatically.

Figure 8 Effects of a long power cut on data collection at the SMEAR Estonia station on 10–15 January 2021. UPS is fully discharged, and the station experienced network-related issues due to no power. The observed data file size is considerably smaller due to missing data, and this is easily diagnosed from the collected metadata.

Figure 9 The DMPS processing graph in Airflow. It follows the same structure as the workflow in Fig. 2. Influx refers to the database used for storing the processed data. Sum files are the processed file type, and surface plots are used for visualization. The arrows represent dependencies, and the last three tasks can be done in parallel. All files are stored in an S3 instance. Colors stand for the status of the task. In this case everything but make_surface_plots has been completed successfully, while that task is still running.

Visualization is done in Grafana, and the interface is facing the public internet, allowing the users access from predetermined remote locations. The analysis workflows in Airflow are defined by graphs. Figure 9 shows one such graph from the Airflow user interface. The analysis and visualization components are run in an OpenShift cloud service also at CSC. In this case the main design choice was enabling remote access to users, so the system could not be co-located with the measurements.

Figure 10 Example of measurements of temperature at heights of 4 and 32 m (a), wind direction (b), trace gas (ozone, nitrogen oxides, and carbon monoxide) and particle concentrations (total concentration in size range 3–820 nm) (c), and wind speed (d) at the SMEAR III station in Helsinki, Finland, on 18 February 2021. All the data are visible in a single dashboard.

Figure 10 shows an example visualization of the SMEAR III measurement data. On the morning of 18 February 2021 during 06:00–11:00 LT, a clear surface temperature inversion is visible from the increase in the temperature profile from 4 to 32 m height above ground (Fig. 10a). At the same time, wind speed is also very low (below 1 m s⁻¹), and the wind is from north to northeast from the direction of a nearby highway a few hundred meters from the measurement site (Fig. 10b and d). The temperature inversion and the low wind speeds lead to inefficient mixing of the air close to the ground and accumulation of pollutants emitted from vehicles in the morning traffic and other nearby anthropogenic sources. The particle concentration in the size range 3–820 nm and the concentration of nitrogen oxides (NO_x, the sum of NO and NO₂) and carbon monoxide (CO) start increasing between 06:00–08:00 LT, and at the same time the ozone concentration is depleted by more than a factor of 10 to below 1 ppb (Fig. 10c). Similar ground-level ozone depletion episodes have been observed at the Hyytiälä SMEAR II station mostly in autumn and winter, connected with a low mixing layer, high relative humidity, and low solar radiation intensity (Chen et al., 2018). The temperature inversion is strongest around 10:30 LT, when the temperature measured at 32 m height (−10.9 ^∘C) is almost 5 ^∘C higher compared to temperature at 4 m (−15.8 ^∘C). We see the highest particle concentrations (above 5 × 10⁴ cm⁻³) and NO_x concentrations (above 200 ppb) at the same time. The ozone concentration has already started to increase. By 11:30 LT the temperature inversion has disappeared; the wind speed has increased; and as the result of more efficient vertical and horizontal mixing of the air, the measured pollutant concentrations are close to their typical background levels for the Helsinki area. This case study shows how easy-to-use data visualization tools, which allow efficient comparisons between datasets from multiple instruments, can help in identification of interesting phenomena in the measurements.

5 Conclusions

We present a concept for station data management and acquisition using interchangeable components. The concept is in operational use at SMEAR Estonia and has been tested in several campaigns. The components we use are built on popular, well-known open-source projects. This framework is suggested for use at new SMEAR stations and could be useful for larger campaigns as well. Since our system is completely modular, different configurations allow it to be adapted for most common use cases. The system can also easily be extended for more instruments and in the future new technological solutions, as necessary. Compared to centralized solutions such as ICOS or ACTRIS stations, this allows users to fully control how their data are processed, monitor them in real time, and control how they are transferred outside the station. This makes it useful for measurements not controlled by the centralized solutions or it can be used as a backup for the data owner in parallel with data transfer to centralized networks. While using our framework does require some technical planning to ensure sufficient hardware resources, we believe the benefits and possibilities of automated data analysis outweigh the costs. We demonstrated with two case studies how continuous visualizations of data and metadata, such as device diagnostics and data file availability, can help quickly identify interesting phenomena and abnormal situations in measurements.

Since the SMEARcore software allows one to combine multiple data sources, it also provides new opportunities for networking measurement stations together and automatically cross-referencing diverse sources of data in the routine operation of the station. This means it is possible to establish smaller networks more easily with the software. An improvement for the management of measurements would be shared storage between stations, where one could check instrument settings or normal operating values at different stations. Another possibility for improving the data usage would be automatically integrating model or satellite data into the analysis or automatically producing the input files for such models, since they can be considered just data products in the SMEARcore framework. In short, automating data processing in the way SMEARcore does also provides opportunities to automate further steps of the scientific process.