Continuous measurements of long-lived atmospheric greenhouse gases (GHGs) at ground-based monitoring stations exhibit variations at multiple timescales. These include a well-established diurnal cycle and an annual pattern linked to seasonality which generally exist on top of the long-term trend of the background concentration. Other variations, related to localized surface fluxes or regional-scale atmospheric transport patterns, are observable at synoptic frequencies lasting from 1–2 d to several weeks, while others reflect longer-term meteorological occurrences such as droughts or ocean circulation anomalies. Identification of these latter components can reveal much about the intensity and geographic extent of specific atmospheric events while also improving understanding of background signal evolution. Extracting them, however, requires a methodology to decompose the signal into “background” and “non-background” components and to differentiate meteorology-driven regional signals from spikes due to local emissions, biospheric uptake and other forms of signal noise.

We define “background” here as “the concentration of a given species in a pristine air mass in which anthropogenic impurities of a relatively short lifetime are not present” (IUPAC, 1997). Various methods exist to extract background signals in atmospheric time series. These include back-trajectory analyses that categorize readings based on air provenance (e.g., Schuepbach et al., 2001; Balzani Loöv et al., 2008; Cui et al., 2011) and the application of chemical filters using markers such as 222Rn (e.g., Biraud et al., 2000; Pal et al., 2015; Chambers et al., 2016) or NOy and CO (e.g., Parrish et al., 1991; Zellweger et al., 2003). Although such approaches yield reliable estimates, they are often labor-intensive or require sophisticated transport modeling or additional instrumentation and must take into account site-specific measurement conditions and data availability. Statistical algorithms provide high-precision, computationally inexpensive alternatives to these techniques. These commonly involve a two- or three-step process in which data are first smoothed using filters or polynomial curve fitting then subsequently refined through the identification of outliers, characterized as points which deviate from the curve by more than a specified threshold (e.g., ±σ, ±2σ, or ±3σ, where σis the standard deviation of the residuals about a smooth curve fit to the data).

Already in the late 1980s, Thoning et al. (1989) developed a filtering technique to separate the annual cycle from the long-term trend and approximate the background signal of the CO2 record at Mauna Loa (Hawaii). More recently, O'Doherty et al. (2001) extracted non-background components of atmospheric CHCl3 time series by fitting a polynomial to the daily minima of a moving 121 d span of measurements. They then subtracted the polynomial fit from the data and estimated σ from the measurements below the median of the residual distribution. Measurements on the middle day of each 121 d period exceeding ±3σ were flagged as “polluted” and removed. In a second iteration, readings between ±2σ and ±3σ above the median of the newly refined residual set were marked as “possibly polluted” and subsequently removed if immediately adjacent to “polluted” data points. Giostra et al. (2011) applied a similar approach to atmospheric halocarbon records. They calculated a probability density function (PDF) using the deviations of all data points from σ, predefined as the 16th percentile of measurements within a 30 d span. A Gaussian curve was then defined using σ and the median value of the PDF, and a Gamma curve was fit such that the sum of the two curves yielded a best-fit to the PDF. The background was approximated using all data points below the intersection of the Gamma curve and the right-hand branch of the Gaussian curve. Ruckstuhl et al. (2012) estimated background signals in atmospheric CO and HFC-152a series by applying a localized linear regression to a given span of data points and removing points which deviated by more than the σ value of the negative (left side) residuals within each successive, overlapping span. Individual points were then weighted for robustness according to their distance from the newly defined background curve, with iterative applications further refining the dataset. Apadula et al. (2019) developed an algorithm to subtract outliers from hourly CO2 datasets. They first removed all values which differed by more than a specified threshold ρ from the median value within a sliding 21 d window and subsequently rejected values that differed by more than ρ from the mean value of the remaining points.

Such methods have been widely applied to estimate baseline concentrations of atmospheric trace gases and, in some instances, to identify the occurrence of short-term signal spikes in time series (e.g., El Yazidi et al., 2018). Lacking in the current literature, however, is a comprehensive statistical framework for the extraction of non-background events occurring at synoptic (1–2 d to several weeks) to seasonal timescales. We thus present here a novel approach to identify exceptional non-background events (“anomalies”) in atmospheric time series based on statistical curve-fitting, LOESS smoothing and outlier detection with the aim of developing a protocol for the detection of anomalous episodes of synoptic and seasonal duration. The methodology is designed for application to station data from the Integrated Carbon Observation System (ICOS) network. In particular, our goal is to investigate whether seasonal- and synoptic-length deviations from background concentrations can be discerned in near real time (NRT) through statistical filtering and cross-referencing observations from multiple sites and to present a framework for communicating information about such events to station managers and other end users.

We focus primarily on CO2, although we validate our detection of wintertime CO2 signal peaks by applying our methodology to concurrent CH4 time series. In the winter months, since carbon exchanges related to terrestrial ecosystem exchange are relatively limited, the timing of synoptic-length anomalies observed in CO2 and CH4 signals should be similar as these are linked principally to changes in the predominant upwind air source. Validation of summer CO2 anomaly patterns using CH4 is impractical due to the dominant role of the biosphere on CO2 concentrations during the growing season.

We place particular emphasis on the discernibility of anomalies observable at multiple European sites since we reason that these are most likely to represent continent-wide terrestrial biosphere changes or synoptic-scale transport patterns as opposed to localized (within ∼ 100 km) contamination effects or other forms of noise. Moreover, the ability to identify these multi-site events is critical in communicating to station managers in near real time the presence of atypical signals and in mapping the footprint of regional carbon cycle fluctuations.

Finally, we present the methodology in the context of a near-real-time anomaly detection algorithm (ADA) developed and employed at the ICOS Atmospheric Thematic Centre (ICOS ATC). The algorithm is concise and portable and is intended to be used with multi-year datasets consisting of validated (level 2) and NRT (level 1) daily datasets from sites in the ICOS network. Both R and Python implementations of the algorithm currently exist, but the methodology can theoretically be adapted to any programming language by any user with access to the ICOS Carbon Portal (ICOS CP) or other standardized GHG data. The methodology described in the following sections refers to the R implementation of the algorithm.

We conduct our analysis using daily aggregated CO2 and CH4 data from 10 European sites. At each, we approximate the background signal of both trace gases using the curve-fitting method of Thoning et al. (1989). We then define an “envelope” representing the range of normal or expected seasonal variability in the signal. The envelope is calculated from the smoothed cycle, which consists of a polynomial function fit to the data and a short-term residual filter. The upper and lower bounds of the envelope are defined by the second standard deviation (2σ) of the smoothed cycle about the background signal and are adjusted to account for seasonal effects on signal stability.

We then smooth the daily data using a LOESS function and evaluate the smoothed daily data in relation to the ±2σ envelope. In our case, we select two different settings for the short-term filter and the LOESS smoothing span: 30 and 90 d. These settings are user-definable. The 30 d analysis is applied to the extended winter season (November–March), where our goal is to discern anomalies indicative of shifts in atmospheric transport patterns. These synoptic-scale anomalies (SSAs) are identified as peaks where consecutive smoothed daily measurements fall outside the ±2σ envelope. The 90 d analysis is applied to the growing season (April–October) with the aim of identifying seasonal anomalies. At this wider bandwidth, the smoothing function should be minimally affected by shorter (< 1 month) regional signals or SSAs, and thus large spikes detected are taken to reflect seasonal-length perturbations such as droughts, springtime carbon uptake or mesoscale circulation anomalies.

The distinction we make between wintertime and summertime signals is not meant to imply that SSAs occur exclusively in winter nor that “seasonal” anomalies are best characterized as summertime-only events. It is rather, in our view, the most logical way to divide the analysis while illustrating the scope of the algorithm's functionalities. This is because the distinction we make between SSAs and seasonal anomalies is primarily based on the perceived underlying causes of each and not necessarily their duration. For example, the persistent formation of easterly transport patterns in the wintertime could, in some cases, result in positive CO2 anomalies that will appear at the 90 d seasonal bandwidth. However, such anomalies would be more representative of a sequence of similar synoptic-scale transport regimes than a seasonal-length reduction in photosynthetic activity or other irregularity in regional carbon cycling. In the summertime, identifying transport-driven SSAs in the record is complicated by a slightly less well-defined North Atlantic Oscillation (NAO) index than in winter (Bladé et al., 2011) and the contemporaneous effects of variations in terrestrial net primary production (NPP), which often occur over slightly longer timescales. Our aim in applying a 90 d smoothing span to the summer data is thus to filter out these synoptic signals to the extent possible and focus only on longer-term perturbations.

Several potential challenges may arise in the technical application of the ADA. These include the presence of large data gaps in time series which could arise from instrument malfunctions or other technical issues. CCGCRV is ill-suited to handle such gaps (Pickers and Manning, 2015), meaning missing data points must be artificially imputed using a structural modeling function or linearly interpolated. We opt for the latter by applying the method of Moritz and Bartz-Beielstein (2017), which is sufficient for small data gaps but generally impractical for large ones (∼ 1 month), where simple linear interpolation can potentially result in erroneous anomaly selection. This appears to be the case, for example, in late October 2018 at GAT, where the algorithm detects negative anomalies in both trace species coinciding with the start of a month-long gap in the daily readings (Figs. 4 and 5). Significant data gaps such as this should thus be identified before performing the linear interpolation and applying the anomaly extraction so that any quantification of the signal during these periods is regarded with caution. One potential workaround is to interpolate data gaps using the values of the multi-year average detrended seasonal cycle at a site. This has not been attempted here since it would require an a priori interpolation of data gaps, i.e., via a linear interpolation, followed by an a posteriori re-interpolation of the same gaps, followed by a secondary application of the CCGCRV fitting procedure. It is unclear whether any reduction in gap-related false positive detection, which appears to be quite rare overall, justifies such a trade-off in efficiency.

The reliability of the results may also be strongly influenced by the range of available data. Sites with longer historical records will have a larger residual dataset to draw from, allowing for more precision when extracting the mean annual cycle and resultant ±2σ envelope, while sites with shorter records may produce less precise results. The potential drawbacks of this are twofold; (1) very slight anomalies might tend to be obscured at sites with a limited number (∼ 3–4 years) of relatively capricious measurements, and (2) low-amplitude localized fluctuations might be detected at sites where the full range of expected seasonal variability is underestimated by the ±2σ envelope, or the background curve is an imprecise fit to the true seasonal cycle. Anomaly patterns at sites with shorter records should thus be regarded with caution and cross-validated with patterns from other nearby sites if possible. In the future, as longer historical records become available at a greater number of ICOS stations, a standardized setting for the record length used to estimate the background (e.g., 10 years) may be adopted.

Furthermore, the method may have limited applicability at more isolated sites that have no clear analog within the ICOS network. Evaluation of anomalies through cross-validation is difficult if a site has relatively few nearby sister stations which can reasonably be expected to sample from similar air masses most of the time. This drawback is apparent when considering the results at PUY and JFJ, both background sites which sample frequently from well-mixed air above the planetary boundary layer (Asmi et al., 2011; Herrmann et al., 2015). At PUY, for example, planetary boundary layer (PBL) height analyses reveal that the station samples from the free troposphere more than 70 % of the time and up to 81 % in the winter (Lopez et al., 2015). With such infrequent sampling of surface air, air parcels most likely to contain the carbon signatures of bellwether biospheric events (such as droughts and their legacy effects) or shorter-term anomalies linked to localized contaminant plumes or sudden changes in atmospheric transport patterns may go undetected. At JFJ, PBL air is sampled only intermittently when conditions favor mountain venting or advection (Zellweger et al., 2003; Griffiths et al., 2014), meaning short-duration anomalies may merely reflect the prevalence of such localized phenomena rather than broader atmospheric transport patterns. In the future, this limitation will become less important as the ICOS Atmosphere network continues to grow in size; 26 stations across Europe currently possess the ICOS label, and another dozen or so are set to join soon.

The occasional selection of localized fluctuations is an additional concern. In some cases, low-amplitude spikes classified as anomalies at a particular station might not truly represent significant regional-scale excursions from the background signal. This implies the need for station-specific protocols to classify anomalies based on duration and magnitude, which may require cross-validation using multiple station readings or manual inspection by principal investigators. The similarity in the patterns at the six northernmost sites, for example, offers a means of validation for the detection of SSAs; since true synoptic-scale anomalies should produce a signal over a broad swath of the continent, those anomalies observed only at certain sites can reasonably be assumed to indicate localized events. Note, for example, the very slight positive anomaly in December 2016 seen only at GAT (Fig. 6).

In other cases, signal excursions might register as anomalies at certain sites but not others. In such cases, users may determine that these events are noteworthy enough that they should be classified as anomalies more broadly. The recourse then is a site-specific refinement of the selection criteria or tuning of the algorithmic parameters. For example, although we use a bandwidth of 30 d for SSA detection, this choice may not be the most appropriate in all cases. Different stations have different ambient signal variability ranges depending on their geographical setting and proximity to emission sources; those with higher overall variability (and hence wider ±2σ ranges) might record too few SSAs if applying an excessively wide smoothing span as peaks in the smoothed signal would be dampened sufficiently to be contained within the envelope. Users may thus find a bandwidth of, for example, 15–25 d to produce more informative results at some locations. Likewise, sites with lower overall variability might tend to record too many SSAs when using a bandwidth that is too short. By definition, the envelope width also affects the anomaly selection. Note, for example, that the 2018 drought pattern typified at the six northernmost sites does not appear at OPE or TRN in Fig. 9. A closer examination of Fig. 8 reveals that while measurements at these two sites during, for example, May 2018 were below the mean annual cycle, no anomalous springtime CO2 dip was registered since the smooth curves at OPE and TRN were still contained within the ±2σ envelope bounds. As mentioned, the specification of a ±2σ threshold stems from our desire to avoid excessive selection of low-amplitude, site-specific signal peaks. However, this width might mask noteworthy seasonal patterns at certain sites with greater year-round variability, making cross-examination all the more critical.

Uncertainties can also arise in the interpretation of the results. For example, the distinction we make between synoptic-scale and seasonal anomalies is primarily based on the length of observed signal spikes. Normally, SSAs which persist from 1–2 d to several weeks are presumed to be linked to prevailing wind conditions at a given site and hence changes in the source regions of sampled air parcels, e.g., from relatively clean North Atlantic air to continental-sourced air parcels bearing the signatures of terrestrial emissions. However, in some cases, anomalies deemed to be seasonal in length may simply represent the frequent occurrence or unusual persistence of synoptic-scale atmospheric transport patterns. For example, as alluded to previously, the extreme heat waves in northern and central Europe throughout much of June and July 2018 were associated with the persistence of a high-pressure blocking system which formed over the region. Although synoptic in size, this pressure anomaly – combined with high temperatures – had resounding effects on European forests and resulted in subsequent CO2 anomalies in the fall of 2018. It is thus more appropriately considered to be part of a broader seasonal anomaly. The implication is that in some cases, “seasonal” anomalies are rather patterns which consist of a series of shorter, related signal irregularities. These irregularities will often be visible at shorter bandwidths and could be directly linked to meteorological events. Summertime anomalies should thus be considered in the wider context of terrestrial ecosystem production, indicators of which may lag well behind the occurrence of exceptional transport episodes.

In general, we find that the algorithm captures well the signature effects of unusually strong or persistent atmospheric transport regimes in the wintertime. This interpretation is reinforced by the fact that the CO2 and CH4 anomaly patterns during the extended winter season are strikingly similar for the 30 d implementation of the methodology. The ADA also shows promising potential with regard to detecting the imprints of regional and continent-wide extremes in NBP and other ecosystem indicators, specifically the distinctive markings of the 2018 European drought, on which we have placed particular emphasis. An analysis of the effects of summertime atmospheric transport is required to more definitively score the method's capacity to correctly identify exceptional biospheric episodes.

The robustness of the results is reliant on cross-validation of anomaly detection across multiple sites. However, we note that anomalies of sufficient size – e.g., ∼ 3 ppm CO2 greater than our ±2σ envelope boundary when using a 30 d bandwidth or ∼ 0.5 ppm CO2 when using a 90 d bandwidth – will usually register simultaneously at multiple sites within the same region and seem generally unlikely to stem from localized contamination. The method also has a low computational cost, and the process of including additional sites in an analysis is relatively straightforward. As new NRT (level 1) GHG data are uploaded to the ICOS CP, users have only to concatenate these to existing datasets and reinitiate the method beginning with the steps outlined in Sect. 2.3. A fully automated Python implementation of the code is available online in which L2 and L1 data for a given station are extracted directly from the ICOS CP and concatenated to form a continuous multi-year time series for all dates up to the present. Users also have the option to extract longer historical records than those available through the ICOS CP and preprocess these themselves before running the code. The current default implementation is to produce time series plots in the style of Figs. 6, 7 and 9.

The ability to detect in NRT the occurrence of non-background signal events at multiple timescales is central to an improved understanding of GHG variability and regional carbon cycling processes at multiple timescales, and the ADA represents an important step toward this end. Eventually, our aim is to make results available online so that station managers and other end users can be alerted in NRT when anomalous signal events occur, i.e., through the automated generation of data files and time series plots to display on the sites' respective panel board pages.