The concept of the audit procedure using travelling standards has been described in detail elsewhere (Buchmann et al., 2009; Klausen et al., 2003). In brief, an audit involves the comparison of travelling standards (i.e. compressed gas in high pressure cylinders) on the analytical system of the audited station (WMO, 2011b). The travelling standards are calibrated against primary laboratory reference standards traceable to the CCL before and after the audit. The audited station's personnel analyse the travelling standards and report the mole fractions, which are compared to the values assigned by the WCC. The result is analysed by a linear regression between the reference (WCC) and the station values. For the calibration of the travelling standards at WCC-Empa, a GC/FID (Varian 3800) system was used from 2000 to 2009 for CH4; since 2009 a CRDS (Picarro Inc., G1301 CO2/CH4/H2O analyser) has been used for both CH4 and CO2 calibrations. Several standards of the CCL (NOAA/ESRL, National Oceanic and Atmospheric Administration/Earth System Research Laboratory) are used as reference standards at WCC-Empa ensuring traceability to the CCL.

For the current study we analysed performance audit results for methane (2005–2014) and carbon dioxide (2010–2015). Details of the comparisons including instruments and analytical techniques are given in Table 1 for CO2 and Table 2 for CH4. In order to assess the performance of the individual comparisons in a standardised way, the bias in the centre of the mole fraction range (405 ppm for CO2, 1900 ppb for CH4) of the unpolluted troposphere (WMO, 2014) (360–450 ppm for CO2, 1700–2100 ppb for CH4) was calculated for these comparisons based on the linear regression analysis. This allows displaying the result of a performance audit using travelling standards as a single dot in a bias vs. slope plot, as illustrated in Fig. 1 for the example of CO2 audits. The green dashed line in the left panel of Fig. 1 shows a case with no bias at 405 ppm CO2 but with the corresponding minimal slope that is possible for the data still meeting the data quality objective (DQO) of 0.1 ppm in the range of 360–450 ppm CO2. This case translates to a single point in the bias vs. slope plot, as shown by the green dot in the left panel of Fig. 1. For illustrative purpose, two additional cases are shown: the maximum allowed bias with the corresponding slope that still meets the extended DQO of 0.2 ppm (orange dashed line/dot) and a case with a slope/bias combination that does not meet the DQOs (red dashed line/dot) over the entire relevant mole fraction range.

Quality assessments based on the comparison of travelling standards alone have their limitations, since they do not cover all parts of the analytical system that may bias a measurement, such as inlet and drying systems (WMO, 2011a). Furthermore, comparisons of travelling standards during on-site audits as well as round robin experiments are only snapshots and are potentially biased, e.g. by coincidental instrument malfunction (results worse compared to normal operation) or by extraordinary care taken during analysis (results better compared to normal operation). Therefore, it has been recommended that the quality control procedure during on-site audits should include parallel measurements with a travelling instrument whenever feasible (WMO, 2011a, 2012, 2014).

The concept of the practical realisation of the on-site data comparison is illustrated in Fig. 2. The core part is a CRDS analyser (Picarro Inc., G2401) as travelling instrument (a). An independent inlet system (b) is used during the parallel measurement with the travelling instrument, and, if feasible, the travelling instrument samples from the station and the independent inlet system sequentially (c). Furthermore, the travelling instrument is independently calibrated using its own set of standards (d), which normally comprises a subset of the travelling standards used for the audit. For further confirmation of the compatibility of the two systems, the travelling standards (e) are measured on both the station analyser and the travelling instrument. Dry air mole fractions are compared for both the travelling standards and the travelling instrument comparisons.

For this study we used the data from two Picarro G2401 CO/CH4/CO2/H2O CRDS instruments (Picarro Inc., USA). The instruments were calibrated every 30–40 h using dry compressed air as working standard. In most cases the sample air was not dried prior to analysis with the travelling instrument and a humidity correction using the Empa method (Rella et al., 2013) was applied to all data. Experimental details of the Empa method are described in Zellweger et al. (2012). Briefly, a small amount of water (approximately 0.8 mL) was directly injected into a constant flow (approximately 500 mLmin-1) of a working standard which was delivered to the instrument. The resulting water vapour influence was then fitted by a quadratic function. The correction function was determined several times for each instrument, and a single function that was initially obtained was used to apply the correction. Yver Kwok et al. (2015) have recently published a comprehensive assessment of the performance of the Picarro G2401 analyser, and our setup of the travelling instrument was done along the lines of their recommendations. However, in contrast to their approach, we were running calibrations more frequently but only as one cycle. Furthermore, the background signal of both travelling instruments was initially adjusted using so-called zero air (CO2 and CH4-free natural air). The following calibration strategy was used: -

A working standard (calibrated against certified CCL laboratory standards before and after each campaign) with mole fractions close to ambient air was analysed every 30–40 h.

An example of the working and target standard measurements is shown in Fig. 3. It can be seen that the variation of the target gas measurements did not exceed the range of ± half of the WMO compatibility goals of 0.1 ppm for CO2 and 2 ppb for CH4, which is usually required for intra-laboratory repeatability (WMO, 2014). Furthermore, the maximum drift between two consecutive working standard measurements was always smaller than half the compatibility goal, indicating that calibrations were made with sufficient frequency. Similar stability was achieved during all measurement campaigns.

The optimal calibration frequency was further investigated with laboratory experiments. For this purpose, a gas standard (dry natural air) was continuously measured using one of the Picarro G2401 WCC-Empa travelling instruments. To prolong the length of the measurement period, the sample flow was reduced to 30 mLmin-1 by a needle valve at the inlet port, which still allowed the stabilisation of the cavity pressure to 140 torr (186.6 hPa). The standard gas was calibrated against NOAA/ESRL standards before and after the experiment to ensure that no drift occurred over the observation period, which may happen when a standard gas is losing pressure (Leuenberger et al., 2015). The initial pressure of the standard was 12.96 MPa and dropped to 6.21 MPa at the end of the experiment. Figure 4 summarises the results of these measurements. The upper left panel shows the CO2 variation of the standard gas (5 s raw data) during the experiment duration of 455 h (approx. 19 days). A slight upward drift was observed, which we consider as instrumental drift since the (secondary) standard has been proven to be stable with respect to the NOAA standards over the course of the experiment. The variations for methane are shown in the upper right panel; again, instrument drift was observed but, in contrast to CO2, no monotonic trend was detected. The lower panels of Fig. 4 show the Allan deviation for CO2 and CH4 (Werle et al., 1993) using the data above, which allows an estimate of the optimal calibration intervals. Based on this experiment, the optimal averaging time is approximately 20 min for both CO2 and CH4. This is shorter than the CO2 minimum at 58 min that was found by Flowers et al. (2012) but compares well with the results of Yver Kwok et al. (2015). However, the Allan deviation only slightly increases up to 1×105 s (27.8 h); therefore, a calibration interval of 30 h is regarded as an optimal compromise with regard to stability, data coverage and the consumption of calibration gas.

On-site comparison experiments with the travelling instrument were made at the following GAW stations: i.

Danum Valley, Malaysia (DMV), operated by the Malaysian Meteorological Department, Kuala Lumpur, Malaysia, with support from Commonwealth Scientific and Industrial Research Organisation, Oceans & Atmosphere, Aspendale, Victoria, Australia;

Figures 6–8 show CO2 comparisons made at the PAL, DMV and CVO GAW stations. In the upper panels, the comparison with the highest available common time resolution (1 min) is shown, together with the deviation of the station instrument compared to the travelling instrument as function of time, and a histogram of the observed bias. The middle panels show the same but for hourly aggregated values where all available data were considered. The lower panels also show hourly aggregates, but mean values were calculated using only high-resolution data with concurrent data availability of the travelling and station instruments.

The temporal variation was well captured at all stations even at the highest time resolution of 1 min. To account for different residence times in the inlet system, data of the travelling instrument were slightly shifted (up to 53 s) to obtain the best possible agreement between the time series. The mean CO2 bias based on 1 min data was 0.08±0.06 ppm at PAL, 0.01±0.67 ppm at DMV and 0.06±0.08 ppm at CVO. These deviations are in good agreement with the results obtained when WCC-Empa travelling standards were measured on the station analysers as summarised in Fig. 9, where the bias is shown for individual travelling standards (black dots), including a linear regression analysis with 95 % confidence intervals. The resulting deviation is very close to the bias observed during the ambient air comparison, which is also shown as small red points (hourly data) in Fig. 9. Normally, the performance audit covers a wider mole fraction range compared to the ambient variability of a station. This gives valuable information about either the compensation of the instrument nonlinearity, the consistency of the used standards or a combination of both, which would not be available from the ambient air comparison alone. The above results indicate that the nonlinearity of the analysers was well corrected at PAL and DMV. In contrast, the travelling standard comparison shows a larger bias at CVO. Most likely this is due to the fact that the travelling standards are significantly out of the CVO calibration range, which is narrow in response to the small variability of the CO2 mole fraction at this station.

The mean agreement and standard deviation remained almost unchanged after aggregation to hourly values at all stations, indicating that the procedures for time synchronisation and sample residence time correction were appropriate. The data availability was different for the travelling instrument and the station instruments mainly due to different requirements concerning calibration frequency of the station analysers. The CRDS travelling instrument was normally calibrated every 30 h for 45 min, which results in a very high data availability and a uniform data coverage. In contrast, the NDIR CO2 instruments at PAL and DMV require more frequent calibrations, which have been implemented using different approaches. The data availability was more or less homogeneous over time at PAL, whereas at DMV the hourly average CO2 mole fraction value is calculated using only the final 44 min of each hour due to the automated hourly zero drift correction mode employed at the start of every hour to monitor short-term detector drifts. For OA-ICOS instrument at CVO, a scheme alternating between a working standard and ambient air measurement in an interval of 1 min was in place resulting in a homogenous data availability of every second minute. This has been done to account for short-term drifts. The data availability of the station analysers and the travelling instrument as a function of the minute of the hour as well as the minute of the day is shown in Fig. 10. Data coverage becomes extremely important with rapid atmospheric changes, e.g. during flux measurements (Peltola et al., 2014), and at sites influenced by local biospheric processes.

The influence of the data coverage was assessed by the calculation of hourly averages with concurrent data availability for the highest time resolution, which is shown in the lower panels of Figs. 6–8. As expected, no influence was found for the CVO data series, which is due to the combination of very small ambient air variability and homogeneous data coverage of the CVO analyser. Some effect was observed at PAL, where the standard deviation of the mean bias changes from 0.05 to 0.03 ppm if only concurrent data points are considered for averaging. It should be noted that in this case all occurrences with a bias > 0.2 ppm disappear. These results can be compared to the simultaneous comparison exercise of the travelling instrument with an on-site CRDS instruments (Rella et al., 2013), where also a mean standard deviation of the difference between the two instruments of 0.03 ppm was observed. As expected, the effect of data coverage on hourly mean values was the most pronounced at DMV due to the largest ambient variability and the omission of the first 16 min of data from each hourly calculation. The standard deviation of the mean bias decreased significantly from 0.62 to 0.21 ppm when only concurrent data were considered for the aggregation of hourly means. In order to test the influence of the temporal coverage, we also calculated the difference between hourly averages of the travelling instrument using all available data and only data of the travelling instrument with concurrent data of the DMV instrument. The resulting distribution of the differences looks very similar to the bias observed between the travelling instrument and the DMV analyser, with a standard deviation of 0.54 ppm. Therefore, the differences that we observe between the DMV analyser and the travelling instrument originate almost entirely from different temporal coverage. The bias induced by the data coverage can well exceed the extended WMO/GAW compatibility goals for hourly values in case of rather large variability of the ambient air CO2 mole fraction. As reported here, homogeneous data coverage over the period of consideration can be more important than the absolute data availability. As an example, measurements made every second minute (CVO) better characterise an hourly average compared to a setup in which the first 16 min of each hour is not measured (DMV). Consequently, in the case of reduced data coverage due to the applied calibration scheme or analytical technique the resulting uncertainties cannot be neglected but should be evaluated and reported along with the uncertainty budget of the analysis.

Figures 11 and 12 show CH4 comparison experiments made at the CVO and MHD GAW stations in the same format as Figs. 6–8 for CO2. At CVO, the CH4 data coverage is the same as for CO2, since it is measured with the same OA-ICOS instrument (Los Gatos Research, LGR-GGA-24EP), whereas the GC/FID system (Carle 100A) at MHD analyses only two ambient air samples per hour, resulting in a maximal data availability of 3.3 % if single injections are being considered to be representative for 1 min. The actual data availability at MHD, however, was 2.2 % due to further instrument downtimes. The observed CH4 bias between CVO and WCC-Empa is -0.61 ± 0.49 ppb based on 1 min data, well within the WMO/GAW compatibility goal of ±2 ppb. Further averaging to hourly values slightly reduces the standard deviation of the bias to 0.34 ppb, whereas similarly to CO2 no significant improvement is observed if only concurrent high-resolution data are considered for the hourly aggregate. The scatter of the observed bias is significantly larger for the MHD comparison, averaging to -0.57 ± 3.79 ppb based on comparison with single injections of the MHD GC/FID with concurrent travelling instrument 1 min data. This is slightly larger compared to the difference that was observed during a 2-month comparison campaign by Vardag et al. (2014), when a mean bias to a FTIR of -0.04 ± 3.38 ppb was observed. The standard deviation, however, is comparable, and a significant part of it can be attributed to the repeatability of the GC/FID system at MHD. The observed relative standard deviation under repeatability conditions (multiple injections of travelling standard) was 0.12 % during the audit, which results in a scatter of 2.25 ppb for the travelling instrument–GC/FID comparison. The aggregation of hourly values nearly doubles the scatter to ±4.20 ppb. This can be expected since two single injections per hour lead to poor statistics and do not sufficiently reflect the observed variability of ambient CH4 at MHD.

As for CO2, the results of the comparison of the travelling standards during audit were confirmed by CH4 ambient air comparisons both at CVO and MHD. The deviations were in good agreement with the results of the travelling standard comparison. This is shown in Fig. 13, where the bias is plotted for individual travelling standard (black dots), including a linear regression analysis with 95 % confidence intervals. Additionally, the bias observed during the ambient air comparison (orange points in Fig. 13) agrees well with the bias determined by the performance audit. In the case of CVO, the observed scatter during the ambient air measurement was comparable with the 95 % confidence bands of the linear interpolation of the audit results, whereas the ambient air scatter at MHD was larger. Again, this is expected due to the different data coverage of the different techniques.

During the comparison experiments at PAL and CVO the travelling instrument switched occasionally from the WCC inlet to the station inlet. This was used to further investigate the influence of the inlet system. At PAL, no difference was observed between the station and the WCC-Empa inlet systems for CO2. The vertical grey shaded areas in Fig. 6 denote the periods when the travelling instrument sampled air from the PAL inlet, whereas the dedicated WCC-Empa inlet was used the rest of the time. The difference between the two inlets was not significant (0.01±0.06 ppm), indicating that the PAL inlet system is fully appropriate. This was also the case for the CVO CH4 measurements, where the mean bias of the CVO instrument was -0.71 ± 0.50 ppb measured at the CVO inlet compared to -0.59 ± 0.49 ppb at the WCC-Empa inlet. However, a small CO2 bias was observed at CVO, with a mean deviation of 0.05±0.06 ppm measured at the CVO GHG inlet compared to 0.13±0.09 ppm at the reactive gases inlet. This is not unexpected, since the reactive gases inlet is located 24 m below the CVO GHG inlet and thus may be more influenced by the local and regional CO2 emissions. Again, this result indicates that the inlet system is fully appropriate at CVO as well.

In other cases (Zellweger et al., 2013), which are not shown here due to the scope of the present study, significant deviations between inlet systems can be observed. Those were mainly caused by leaks or inefficient drying procedures. The use of independent inlet systems during ambient air comparisons can clearly provide valuable information on the overall performance of the measurement setup, which cannot be obtained by the comparisons of standard gases alone. In the cases presented here, the inlet designs were appropriate and did not contribute to a potential bias.

Traditional measurement techniques for GHGs require drying of the sample gas to achieve the WMO/GAW compatibility goals. A recent study by Rella et al. (2013) shows that measurements of humid air can be made using CRDS instruments when an appropriate correction for water vapour interference is applied. They concluded that it is possible to make CO2 and CH4 measurements within the GAW compatibility goals (for the Northern Hemisphere) for water vapour levels up to at least 2 % by determining the water vapour correction function once on a per-instrument basis. During our study, all WCC-Empa measurements, except at CVO, were made without drying the air sample.

The water vapour correction functions were experimentally determined several times per instrument, which provides information about the stability and reproducibility of the correction function. The correction function is a second-order polynomial, as shown by following equations: CO2(dry)=CO2(wet)/1+a⚫H2O+b⚫H2O2,CH4(dry)=CH4(wet)/1+c⚫H2O+d⚫H2O2. CO2(wet), CH4(wet), and H2O are the (humid) mole fractions in ppm (CO2, CH4) or % (H2O) reported by the analyser.

Figure 14 shows the differences between the first and the subsequent measurements for the two instruments at a nominal mole fraction of 400 ppm (CO2) and 1900 ppb (CH4) with the WMO/GAW compatibility targets. The CRDS instruments used for this study have built-in water vapour correction functions and report also dry mole fractions. However, to achieve the best possible correction equation, it is advisable that instrument specific correction functions are determined for each instrument using the method described by Rella et al. (2013). The robustness of the correction can be further improved by pooling a number of correction functions obtained by several experiments or by selecting a correction function that is representative for a larger set of experiments.

Figure 14 shows that the CO2 correction functions result in dry mole fractions that are within the WMO/GAW compatibility goals for water vapour up to 2 % when functions obtained at the same day are pooled (CFKADS#2001) and up to >2.5 % for the newer analyser (CFKADS#2098), which is in line with the results published by Rella et al. (2013). The differences between determinations of the water correction function at same day and after longer time periods were similar, which indicates that the short-term and the long-term sensitivity changes are in the same order or that the repeatability of the droplet test using the Empa method (Rella et al., 2013; Zellweger et al., 2012) is the limiting factor. Also, it was confirmed that the CH4 correction functions result in dry mole fractions that are within the WMO/GAW compatibility goals for water vapour up to 3 %. In our case, only one correction function that was initially retrieved was used to correct the data. These initial coefficients, as determined using the Empa method described by Rella et al. (2013), are summarised in Table 4.

The difference between dry station instrument measurements and humid travelling instrument measurements as a function of water vapour is illustrated in Fig. 15 for all comparisons. No dependency was found between the observed CO2 bias and H2O measured by the travelling instrument at PAL. This result is consistent with a comparison using the data of the travelling instrument and another CRDS instrument with dry sample air that has been published by Rella et al. (2013). It is noteworthy that the same result was obtained at DMV despite the much higher H2O content of the ambient air. A small remaining dependency cannot be excluded based on the current study due to the high variability of the observed bias; however, the contribution to the overall uncertainty would be small compared to other sources of uncertainty. Since the water vapour interference is the limiting factor for the CO2 measurements due to the uncertainty caused by the correction, it is important to note that the results of PAL and particularly DMV confirm the applicability of CO2 correction functions.

With our study, we aimed to evaluate the performance of modern spectroscopic analysers (CRDS, OA-ICOS, and FTIR) in comparison to the traditional techniques. WCC-Empa conducted 32 station audits with a travelling standard for CH4 (2005–2014) and 20 for CO2 (2010–2015). Details of the comparisons including instruments and analytical techniques are given in Table 1 for CO2 and Table 2 for CH4. Each comparison was evaluated by linear regression analysis as shown in Fig. 13. To judge whether the resulting slope/intercept combinations meet the WMO/GAW compatibility and extended compatibility goals, the bias in the centre of the mole fraction range (405 ppm for CO2, 1900 ppb for CH4) of the unpolluted troposphere (WMO, 2014) (360–450 ppm for CO2, 1700–2100 ppb for CH4) was plotted against the slope of the individual travelling standard comparisons. This is shown in Fig. 16 along with the allowed bias/slope combinations corresponding to the compatibility (dark grey area) and extended compatibility goals (light grey area) over the entire mole fraction range of the unpolluted troposphere. Only comparisons that were on the same calibration scale and without any known instrument malfunctions were considered. It can be clearly seen that large differences exist among the evaluated analytical techniques. Newer spectroscopic techniques such as CRDS and OA-ICOS show generally better performance with respect to accuracy and measurement uncertainty compared to NDIR (CO2) and GC/FID (CH4). Moreover, these techniques also provide better data coverage, which further reduces the uncertainty.

The results of the above analysis are further presented in Fig. 17, which summarises the percentage of comparisons that met the compatibility and extended compatibility goals. We show all comparisons and then separately only travelling standard–CRDS, travelling standard–NDIR (CO2) and travelling standard–GC/FID (CH4) comparisons.

It is obvious that reaching the compatibility goals for CO2 remains a challenge; out of the 20 comparisons, only 2 (10 %) met the compatibility goal with 7 (35 %) meeting the extended goal. However, these results include the entire CO2 mole fraction range relevant for the troposphere. Often, the calibration ranges at stations are intentionally limited to the ambient mole fraction ranges typical for their location. Such ranges can be significantly smaller than those used for Figs. 16 and 17, e.g. at CVO. Therefore, slope/bias pairs that are outside the compatibility goals do not necessarily imply that the measurements at a station are biased. However, they provide useful information about the performance of the instrument as well as the calibration over the entire mole fraction range of the unpolluted troposphere.

For CO2, the overall compatibility of the travelling standard–CRDS comparisons was significantly better compared to the travelling standard–NDIR comparisons. From the total nine travelling standard–CRDS comparisons, five (56 %) were within ±0.2 ppm and one within ±0.1 ppm (11 %). Out of the eight travelling standard–NDIR comparisons, one (12 %) reached the ±0.1 ppm and another one (12 %) the ±0.2 ppm limit. All other comparisons (75 %) were outside the compatibility goals. This is primarily due to the poorer repeatability and limitations of the NDIR technique in general (drift and nonlinearity of the detector) and, secondly, to the calibration linearity issues which become relevant when comparison results are expanded beyond the typical mole fraction range for a particular measurement site. This was also reflected by the error bars in Fig. 16, which in most cases were significantly larger for travelling standard–NDIR comparisons compared to travelling standard–CRDS CO2 comparisons.

Meeting the WMO/GAW compatibility goals for CH4 is significantly less challenging. Out of the 32 comparisons, 17 (53 %) were within the compatibility goals and 10 (31 %) were within the extended compatibility goals. A total number of five (16 %) comparisons did not meet the extended compatibility goals.

Figure 17 presents the results analysed as a function of the measurement techniques. All CH4 travelling standard–CRDS comparisons were within the extended compatibility goals; nine (82 %) of the comparison met the ±2 ppb limit, whereas the ±5 ppb limit was met in the remaining two (18 %) cases. This is significantly more than for the GC/FID systems, of which only eight (44 %) met the ±2 ppb limit and five (28 %) the ±5 ppb limit. The remaining five (28 %) of the CH4 travelling standard–CRDS comparisons were outside the compatibility goals. Focusing on the station relevant CH4 mole fraction range should not be the reason of the worse result compared to CRDS, since the GC/FID method is known to be linear in the mole fraction range discussed here. More likely, worse instrument repeatability compared to CRDS plays an important role here, which is further illustrated by the significantly larger error bars of the bias/slope pairs in Fig. 16 for GC/FID systems.

Other techniques (OA-ICOS, FTIR) also indicate better repeatability compared to GC/FID (CH4) and NDIR (CO2); however, an insufficient number of comparison studies have been made to reliably show the superior performance of these techniques.