An interlaboratory comparison to quantify oxidative potential measurement in aerosol particles: challenges and recommendations for harmonisation

This paper presents the findings from a collaborative interlaboratory comparison exercise designed to assess oxidative potential (OP) measurements conducted by 20 laboratories worldwide. This study represents an innovative effort as the first exercise specifically aimed at harmonising this type of OP assay, setting a new benchmark in the field.

Over the last decade, there has been a noticeable increase in OP studies, with numerous research groups investigating the effects of exposure to air pollution particles through the evaluation of OP levels. However, the absence of standardised methods for OP measurements has resulted in variability in results across different groups, rendering meaningful comparisons challenging. To address this issue, this study engages in an international effort to compare OP measurements using a simplified method (with a dithiothreitol (DTT) assay).

Here, we quantify the OP in liquid samples to focus on the protocol measurement itself, while future international OP interlaboratory comparisons (ILCs) should aim to assess the whole chain process, including the sample extraction. We analyse the similarities and discrepancies observed in the results, identifying the critical parameters (such as the instrument used, the use of a simplified protocol, the delivery and analysis time) that could influence OP measurements and provide recommendations for future studies and interlaboratory comparisons even if other crucial aspects, such as sampling PM methods, sample storage, extraction methods and conditions, and the evaluation of other OP assays, still need to be standardised. This collaborative approach enhances the robustness of the OP DTT assay and paves the way for future studies to build on a unified framework. This pioneering work concludes that interlaboratory comparisons provide essential insights into the OP metric and are crucial to move toward the harmonisation of OP measurements.

1 Introduction

Over the last decade, many studies demonstrated associations between exposure to ambient air pollution and adverse human health outcomes (Hart et al., 2015; Laden et al., 2006; Lepeule et al., 2012; WHO, 2013, 2021). Adverse health effects attributable to particle matter (PM) are complex and diverse. Among environmental factors, PM is considered to be the largest contributor to morbidity and mortality globally (WHO, 2024). The causal mechanisms underpinning these adverse associations are diverse, with oxidative stress and inflammation (Leni et al., 2020; Li et al., 2003, 2008), genomic alterations (Huang et al., 2022), and damage to the nervous system function (Wilker et al., 2023), as well as epigenetic alterations and cognitive decline (Nicholson et al., 2022; Zare Sakhvidi et al., 2022), all cited as potential contributing pathways. Across these broad domains, the capacity of particles and particle-derived chemicals to cause damaging biological oxidations appears to be the ultimately response of oxidative stress, both through the introduction of pro-oxidants and stable free radicals into the body, but also through secondary radical/oxidant generation through altered metabolism and induction of inflammation (Li et al., 2008). By definition, oxidative stress is a condition where excess production of reactive oxygen species (ROSs) and nitrogen species (RNSs) overwhelm endogenous antioxidant defences (Shankar and Mehendale, 2014). Generally, ROS and RNS production in the cells is regulated within physiological limits through the actions of antioxidant enzymes (e.g. superoxide dismutase, catalase) and low-molecular-weight water-soluble (e.g. glutathione, ascorbate) and fat-soluble (vitamin E) antioxidants (Alkoussa et al., 2020). This antioxidant system plays a valuable key role by limiting ROS and RNS damage, which is associated with cytotoxicity and the induction of inflammation due to changes in the cellular redox balance (Cassee et al., 2013; Gao et al., 2020; Kelly and Fussell, 2017; Sies, 2018). The capacity of PM to invoke biological oxidations has therefore been proposed as a proxy measure of their toxicity and has been referred to as their oxidative potential (OP), which is intrinsic to their possession of pro-oxidants or encompasses their capacity more holistically to simulate ROSs and RNSs through interaction with cells (Ayres et al., 2008; Bates et al., 2015; Cho et al., 2005; Sauvain et al., 2008; Uzu et al., 2011).

Consequently, the OP of PM is increasingly being studied as a potentially health-relevant metric to evaluate effects due to exposure to PM, in addition to PM mass concentration, in multiple regions across the globe (Bates et al., 2019; Bhattu et al., 2024; Daellenbach et al., 2017; Weichenthal et al., 2019). OP is a relatively simple estimation of PM redox activity that reflects a complex interplay of all physico-chemical properties (chemical composition, surface-area, solubility and particle size) contributing to the ROS and RNS generation and the oxidation of target biomolecules or probes. Implicit within this approach is the contention that not all constituents of ambient PM are equally as harmful and that those that drive damaging redox reactions, either directly or indirectly, present a greater hazard. Thus, PM composition should be considered a factor more directly linked to adverse health effects than PM mass concentration, highlighting the need to study additional health-relevant metrics such as the OP (Park et al., 2018).

In last decades, there has been an increased interest in measuring and developing OP studies, applying different in vivo or in vitro assays and aerosol characterisation techniques to estimate the main sources of OP related to PM (Guascito et al., 2023), and attempting to integrate this metric into health studies. Several acellular chemical methods have been applied for the estimation of the OP of atmospheric particles since these assays allow faster measurement and are less labour-intensive than cell culture or in vivo methods (Bates et al., 2019). In addition, these assays aim to mimic the interaction between PM and different lung antioxidants (e.g. glutathione, ascorbate) during inhalation. The acellular assays which are most commonly applied include several probe approaches based on antioxidants or surrogates, such as the dithiothreitol (DTT) assay, ascorbic acid (AA) assay, glutathione (GSH) assay, ferric–xylenol orange (FOX) assay, 9,10-bis(phenylethynyl)anthracene-nitroxide (BPEAnit) ROS assay and 2,7-dichlorofluorescein (DCFH) assay for bound particles. Molecular probes display variable sensitivities to PM components due to their unique redox potentials and chemical reaction routes, contributing to aerosol OP values. Therefore, it may be necessary to use several assays simultaneously for a broader assessment of the chemical species in PM potentially triggering oxidative stress and to evaluate which of these probes might be most indicative and closely linked to health effects. Furthermore, one of the main challenges within this rapidly expanding research field is the diversity of analytical methods and protocols used for each OP assay, which require standardised protocols to support synthesis across the evidence base (Ayres et al., 2008).

In 2008, a previous workshop gathered experts on OP and developed consensus statements addressing the importance of standardised samples, the comparison between oxidative potential tests, the formulation of consistent standard test protocols, and the establishment of connections between OP tests and epidemiological findings as reliable predictors of adverse health outcome (Ayres et al., 2008). Despite more than 15 years having elapsed since that workshop, whilst protocols have matured, evolved and proliferated worldwide, little concrete work has been performed regarding the harmonisation and standardisation of these methods.

One of the main objectives of the RI-URBANS European project (https://riurbans.eu/, last access: 30 December 2024) is to bring accessible service tools to enhance air quality monitoring networks, including evaluating air pollution exposure. As OP has been proposed and recommended as a parameter to be measured in the proposal for a new European Air Quality Directive (Council of the European Union, 2024), an international OP interlaboratory comparison (ILC) was launched to assess the consistency of OP measurements between participants that apply different OP DTT protocols, hindering comparison of results obtained worldwide. The main goal of the ILC was to identify potential discrepancies in results (obtained with the OP DTT assay, one of the most common acellular assays used for measuring the OP) that may arise due to differences in experimental procedures, equipment or analytical techniques. This ILC constitutes a first step to identify potential sources of variability, resulting in the enhancement of the overall accuracy, reliability and comparability of OP measurements.

This paper presents the set-up and results of this first large ILC study based on the dithiothreitol (DTT) assay, with a large number of participants (20 groups). The first section includes a description of how a simplified protocol was obtained, along with the coordination and management of the ILC. Subsequently, the results are presented in the second section, combined with statistical analysis, for both the harmonised protocol and the home protocols of each participant. Finally, some major findings and recommendations are presented.

2 Intercomparison strategy

This ILC was proposed within the RI-URBANS European project framework to evaluate the discrepancies and commonalities of OP measurements obtained by the different participating laboratories. The set-up of the protocol was led by a working group of laboratories with considerable experience in oxidative potential: FORTH (the Foundation for Research and Technology – Hellas (FORTH, Greece); NOA (National Observatory of Athens, Greece); ICL (Imperial College London, United Kingdom); IGE (Institute of Environmental Geosciences, France); and UoB (University of Birmingham, United Kingdom) (i.e. the “core group”). Considerations regarding sampling techniques of PM filters or monitoring strategic approaches are beyond the scope of this exercise.

Multiple OP assays are available; however, following a literature review, it was decided to prioritise the DTT assay for this first ILC due to its widespread adoption and long-term application facilitating broad participation from laboratories. The core group first produced a harmonised and simplified method, detailed in a standardised operation procedure (SOP), that was integrated, implemented and tested by IGE, the organiser for this ILC. This SOP is called the “RI-URBANS DTT SOP” in the following and is presented in detail in Sect. 2.2. Section 2.1 presents the selection procedure for some parameters of the DTT assay according to variations observed in the literature. Section 2.3 to 2.7 comprise different parts of the implementation of the ILC, along with the procedure for data processing.

2.1 Testing the parameters for implementing the simplified RI-URBANS protocol

The simplified RI-URBANS DTT SOP was adapted from the original DTT protocols published in the early 2000s (SOP1: Li et al., 2003, 2009; SOP2: Cho et al., 2005; SOP3: Kumagai et al., 2002; called SOP1, SOP2 and SOP3, hereafter). The principle of the DTT assay relies on the production of superoxide radicals, with DTT acting as a surrogate for cellular reducing agents. This probe contains thiol groups similar to GSH and is subject to oxidation, forming stable cyclic disulfides by donating electrons to oxygen through intermediate redox-active species from PM. In the assay proposed by Kumagai et al. (2002), PM is incubated with DTT and its rate of oxidation over time is assessed through the use of 5,5^′-dithiobis(2-nitrobenzoic acid) (DTNB) with the reaction product 2-nitro-5-thiobenzoic acid (TNB), being detected at 412 nm. Whilst DTT is in excess, the rate of DTT oxidation is proportional to the concentration of redox-active species in the PM (Cho et al., 2005; Sauvain et al., 2008). The DTT loss over time can then be expressed per unit concentration of PM (usually µg) or by volume of air (m³) to provide a measure of the intrinsic reactivity of the particles and assess human exposure (Cho et al., 2005), respectively.

A review of the pioneer DTT assay protocols revealed differences, and therefore to derive a simplified protocol, some variations in the parameters were examined by the ILC organiser laboratory. These results were evaluated by the RI-URBANS core group to obtain the final harmonised protocol.

Three key parameters were tested. First, the necessity for the inclusion of trichloroacetic acid (TCA) was evaluated. In the original DTT protocol by Kumagai et al. (2002), the reaction between samples and DTT is quenched at a specified incubation time by the addition of 1.0 mL of 10 % TCA to the incubation mixture. Subsequently, 0.5 mL of the reaction mixture is extracted and combined with DTNB and Tris-HCl buffer (pH 8.9). However, more recently, Li et al. (2009) demonstrated that DTNB rapidly reacts with DTT, with the absorption reaching its maximum value immediately and remaining stable for over 2 h. In our initial tests, we found that this parameter was consistent, showing no differences over all the samples tested with or without TCA (Fig. S1 in the Supplement). Thus, in the simplified OP RI-URBANS SOP, TCA addition was omitted, and we introduced the DTNB solution directly into the mixture at the prescribed incubation times and recorded the absorption after the 30 min reaction.

Second, ethylenediaminetetraacetic acid (EDTA) is present in some OP DTT protocols, whether in the buffer of the incubating PM sample with DTT (Kumagai et al., 2002; Li et al., 2009) or in the titration mixture of DTNB (Cho et al., 2005; Kumagai et al., 2002). EDTA is a strong chelating agent which is widely used in biological assays to prevent microbial contamination and to facilitate cell lysis and the extraction of cellular components. Using EDTA is helpful in purifying buffers at a low cost and decreasing a high rate of DTT loss in the blank by scavenging metal ions. However, it can lead to artefacts during the assay. This is especially critical when used in the reaction mixture with the PM, where it could induce complexation with redox changes. Moreover, this is particularly relevant for iron where complexation increases solubility and the resultant EDTA–Fe complex is redox-active (Gao et al., 2024) or, in the opposite way, EDTA can chelate some metallic species, preventing their reactivity (Charrier and Anastasio, 2012). In addition, EDTA also has antioxidant properties, which may compete in the solution with DTT (Thbayh et al., 2023). Our results for different samples show that the presence of EDTA in the buffer leads to underestimation of the OP DTT loss rates in comparison to that without EDTA. An impact was observed in the solutions, mainly the copper reference solution (1 µM) and an ambient PM filter for testing both protocols (Fig. S2). That augmentation could also be related to an increase in the blank absorbance without EDTA due to the impurities in the buffer, but this is controlled by subtracting the blank. Little or no impact was shown in the 1,4-naphthoquinone solutions, an organic component. To prevent such undesirable interactions, many laboratories have introduced Chelex^® 100, a sodium-form resin to purify the buffers used in the OP DTT assay (Calas et al., 2017, 2018; Charrier et al., 2015; Charrier and Anastasio, 2012), or used commercially available high-purity (e.g. LC-MS/MS grade) water and buffer mixture to create the OP reaction medium (Shahpoury et al., 2019, 2022). Nevertheless, the Chelex^® commercial resin comes with a basic pH and requires a pre-treatment that could be an extra source of error for this current, first international intercomparison. Finally, EDTA was not included in the SOP to prevent complex ligand chemistries and because the use of Chelex^® was too complex to be introduced for the first such ILC; high-grade buffer powders were sent to all the participants instead.

Third, original protocols include the use of a Tris-HCl buffer at pH 8.9 in the solution of titration with DTNB, and this remains widely used. However, Li et al. (2009) showed that the pH of the solution drives both the catalytic redox reaction rate and also the molar extinction coefficient of the product of the reaction, TNB. This report confirmed the previously obtained results by Danehy et al. (1971) that showed that DTNB suffers from alkaline decomposition above pH 8, increasing absorbance values (Fig. S3). As a result, for the simplified RI-URBANS DTT SOP, the core group selected a potassium phosphate buffer at a physiological pH of 7.4 to replace Tris-HCl in the DTNB solution. This prevents the DTNB alkaline decomposition because the pH is in the favourable range of 5.5–8 where TNB is in the TNB²⁻ form and the DTT+DTNB system does not show significant pH-dependent changes in the absorbance values (Li et al., 2009).

Finally, the simplified RI-URBANS DTT SOP also includes a variation considering the instrument used for the measurements. The SOP was elaborated and tested for both cuvette and plate-reader spectrophotometers, including the potential application of automatic samplers. However, the simplified SOP remains to be tested for other instruments (such as an liquid waveguide capillary cell (LWCC) instrument).

2.2 Simplified RI-URBANS DTT SOP and other measurements

The simplified RI-URBANS DTT SOP is proposed in Sect. S1 in the Supplement​​​​​​​. It contains a first step for the preparation of the reagents needed for the analyses included in the ILC. A second step describes how to perform a calibration of the analytical device, using a DTT calibration curve with at least four points for a concentration range between 0 and 60 µM (titration with 1 mM DTNB and reading of TNB formation at 412 nm). In the third step, the SOP defines the performance of the measurements for the test samples provided, including the assessment in triplicates of each test sample and control point (blanks). It should be noted that the DTT protocol also integrates the analysis of control points (blanks), which allows quantifying the inherent DTT background oxidation.

The duration of the analyses required for the ILC is variable depending on the instrument used. About 30 min completion time is required, including the assessment of ILC test samples and control points, when performed with a plate reader, and a similar time is needed to perform the calibration curve. When a cuvette-type spectrometer is used, the total analysis time can be at least 2 h to perform all the triplicates and the corresponding calibration curve.

Furthermore, apart from the analytical equipment for monitoring the chemical reactions, the simplified protocol also requires some standard laboratory equipment and conditions, such as access to ultra-pure water (18.2 MΩ cm⁻¹, total organic carbon (TOC) < 5 ppb) for the preparation of reagents, the use of a vortex for the homogenisation of samples, refrigerated baths to conserve the reagents, and transparent 96-well plates and dark tubes. The samples also need to be kept under agitation during the experiment and at a constant temperature of 37.4 °C.

The calculation of the chemical reaction rate of OP DTT during this ILC involves a conversion using a calibration curve. Once the results are obtained, the kinetics of the DTT oxidation can be calculated by subtracting both the intrinsic absorption of each sample (absorption obtained from the samples before the addition of reagents to remove a potential matrix effect between samples) and the inherent DTT auto-oxidation rate (slope of a control sample) from the DTT consumption rate in the presence of PM.

Participants were asked to perform additional OP measurements on the same samples, following the protocol in use in their laboratory (“home protocol(s)” or DTT-home) if they wanted to do so. In this case, they had to provide the results in the particular units requested, depending on the applied assay.

2.3 Type of samples – test materials

The ILC was performed using three samples (SP1–SP3), including different concentrations of ambient PM and a commercial positive control (1,4-naphthoquinone; CAS 130-15-4, Sigma Aldrich), all extracted in ultra-pure water and prepared by the ILC organiser. Providing ultra-pure water extracts – instead of filter fragments – to the participants was selected for the current first ILC to avoid additional uncertainties associated with the use of different procedures of sample extraction, different quality of the ultra-pure water for sample extractions and changes linked to the processing equipment available. More specifically, the samples sent to the participants included SP1 – 1,4-naphthoquinone solution (reference compound, at 5 µg mL⁻¹); SP2 – extract from a PM sample influenced by biomass burning emissions (obtained from a chamber experiment, at 25 µg mL⁻¹); and SP3 – an urban PM extract highly influenced by traffic emissions (from pooled roadside samples obtained from TEOM–FDMS (tapered element oscillating microbalance – filter dynamics measurement system) reference samplers, at 25 µg mL⁻¹). Additionally, a fourth sample, SP4 – a sample extracted from a blank/clean quartz filter, was sent to the participants, but it was not included in the evaluation since the measured values were close to the instrument limits of detection for most participants. For each of the four samples, all the sub-samples, distributed to participants, resulted from a unique 1 L solution obtained from the original sample substrate. For instance, SP1 and SP3 were powder and concentrated extraction, respectively, that were solubilised and homogenised for 75 min by vortex agitation in ultra-pure water. SP2 and SP4 were quartz fibre filters subjected to a 75 min vortex extraction in ultra-pure water.

Several 5 mL sample aliquots in dark polypropylene tubes were sent to each participant, according to their needs, allowing them to perform triplicate measurements for the RI-URBANS DTT SOP and all the home OP protocols implemented in their labs.

Solid potassium dihydrogen phosphate (CAS 7778-77-0, Roth), di-potassium hydrogen phosphate (CAS 7758-11-4, Roth), 5,5'-dithiobis(2-nitrobenzoic acid) (CAS 69-78-3, Roth) and 1,4-dithiothreitol (CAS 3483-12-3, Roth) were also distributed to the participants to prepare the solutions for the RI-URBANS DTT SOP, including the DTT solution, the dithiobis(2-nitrobenzoic acid) (DTNB) solution and the potassium phosphate buffer solution.

2.4 Transport of samples, ILC performance and duration

Test samples (SP1–SP4) were shipped to all participants on 17 January 2023 via courier in refrigerated and isolated ice packs and received as chilled liquid samples. The parcels were delivered between 18 January and 2 February 2023. On average, the participants received the samples in 3 ± 2.5 d and performed the analysis in 14 ± 10.5 d and up to 31 d after the reception of samples. The recording of these parameters allows their integration into the multiple linear models used in this work.

2.5 Reporting of the results

Participants were asked to report OP DTT results from the RI-URBANS SOP in nanomoles DTT per minute per microgram (nmol DTT min⁻¹ µg⁻¹) and the percent of DTT consumed in micrograms per minute (µg⁻¹ min⁻¹), applying three decimal digits for all three replicates of test samples. An Excel spreadsheet with all the calculations pre-included was prepared by the ILC organiser and shared with all the participants to avoid calculation errors and to facilitate the standardisation of results. In addition, participants were invited to report, in the same format, the values for other OP tests, such as OP DTT-home, and other OP tests like AA, DCFH, OH, ESPR (electron spin (or paramagnetic) resonance), GSH and RP (redox potential) (routinely applied by each participant) on the same samples.

2.6 Number of participants

It is worth noting that for the first time, a total of 20 research groups participated in the exercise: 14 of them from Europe, including the United Kingdom, Italy, France, Switzerland, Greece, Germany, Serbia, the Czech Republic, the Netherlands and Sweden; 3 participants from the United States; 2 participants from Canada; and 1 from Australia. The participants were invited for their contribution to the RI-URBANS project through a public call to participate or because they contacted the ILC organiser directly and were selected due to their active role in the OP scientific community. The ILC was performed using anonymous participation; thus, a number was randomly assigned to each participant to present the results. Participant L5 cancelled their participation in the ILC, and two participants (L3 and L16) did not send their results for RI-URBANS DTT SOP.

2.7 Data evaluation

The ILC results were analysed by the European Commission Joint Research Centre (JRC), which provides independent, evidence-based science and knowledge support for EU policies and in conducting ILC exercises. The participation of an external independent evaluation was required following the International Global Standard ISO 5725-2, related to the accuracy of measurement methods and results (“Part 2: Basic method for the determination of repeatability and reproducibility of a standard measurement method”). The data evaluation includes the assessment of the test sample homogeneity and stability, of each laboratory's repeatability, and of participants' performance.

2.7.1 Estimation of the assigned value and participant performance

Participant performance is evaluated based on biases against pre-assigned criteria including the assigned values and standard deviations for the proficiency test.

The choice of the methods for determining these assigned values and standard deviations is under the responsibility of the ILC organiser. The standard uncertainty of the assigned value should be as small as possible to minimise the risk that participants will receive underperformance signals because of inaccuracy in the determination of the assigned value. Three methods including the use of (i) a simple mean and standard deviation including all participants, (ii) the Q/Hampel test, and (iii) the consensus value and standard deviation from expert laboratories were compared.

Six participants (L2, L4, L8, L12, L13 and L19) were selected a priori (without knowing their results) as expert laboratories based on their previous pioneering experience in developing OP measurement protocols and their strong expertise (i.e. > 10 publications) in the field. The “consensus among expert laboratories” approach was selected for determining the assigned values (X = 0.53, 0.14, 0.07 nmol min⁻¹ µg⁻¹ for SP1, SP2 and SP3, respectively) and standard deviations (σ^∗ = 0.16, 0.06, 0.04 nmol min⁻¹ µg⁻¹ for SP1, SP2 and SP3, respectively) for the proficiency test because it led to the smallest relative uncertainties for the assigned values, ranging from 10 % to 13 % for test samples SP1 to SP3.

The performance of each participant was further evaluated using z scores, a metric indicating the deviation of each data point from the assigned value as compared with the standard deviation for proficiency assessment, $z=(x_i-X)/{\mathit{\sigma }}^{\ast }$, where x_i is the result from participant i.An “action signal” is triggered if a participant's entry produces a z score exceeding +3z or falling below −3z, indicating a deviation of more than 3 standard deviations from the assigned value. Similarly, a “warning signal” is raised for a participant z score above +2z or below −2z, representing a deviation between 2 and 3 standard deviations. A participant z score between −2z and +2z signifies satisfactory performance concerning the standard deviation for proficiency assessment.

2.7.2 Analysis of the variability in the results

Additionally, the statistical distribution of results was evaluated using multiple linear regression models. In the first model, the effects of the protocol variables on the measured OP were investigated using linear regression models adjusted on the instrument (three class variables: plate reader, cuvette and LWCC), delivery time (time between sample shipment and reception, continuous) and analysis time (time between reception and analysis, continuous). An additional model (M2) compared the RI-URBANS SOP and the DTT-home protocols was further adjusted based on the protocol (two class variable: RI-URBANS, DTT-home). Finally, the evaluation of the average performances (three class variables: low – 0 < | z score | < 2; middle – 2 < | z score | < 3; and high – | z score | > 3) was added in the M2 model to assess whether performance affected DTT activity in the same direction (i.e. positive or negative) while considering other protocol variables. Each model was run separately for SP1, SP2 and SP3 samples. All analyses were conducted using R (version 4.2).

3 Results and discussion

Out of the group with a total of 20 participants, 18 presented results obtained using the RI-URBANS DTT SOP. Different instruments were used to apply the simplified SOP. Overall, nine participants (47.5 %) used the cuvette-type spectrophotometer, eight (42 %) used the plate-reader-type spectrometer, and two (10.5 %) implemented LWCC measurements (one participant used two instruments).

3.1 Homogeneity of the samples

An initial assessment of the homogeneity of the OP measurements with the three test samples was performed by the ILC organiser using a plate-reader-type protocol, deriving the mean and the standard deviation of 10 replicate analyses performed on the same day. The results obtained from the coefficient of variations showed the sample variabilities were up to 12 %, 7 % and 9 % for SP1, SP2 and SP3, respectively (Fig. S5), showing a higher variability for the 1,4-naphthoquinone solution (SP1) compared to the two filter extracts (SP2 and SP3).

The overall uncertainty of the OP DTT assay has been evaluated to 18 % for PM₁₀ and 16.3 % for PM_2.5 by Molina et al. (2020). Despite some differences observed between our samples, the results are deemed acceptable, presenting a variability of around 10 %, which indicates a very good performance of the analysis.

3.2 Ageing of samples

To reduce the number of parameters affecting the preparation of the sample solutions, liquid solutions of each sample were prepared and sent to the different participants. However, liquid samples can undergo ageing processes, impacting OP levels over time. For this purpose, an ageing test was performed by the ILC organiser to evaluate the potential changes over time. It consisted of regularly implementing the RI-URBANS DTT SOP to obtain the values of each test sample over time. Figure S6 shows these results, where SP2 and SP3 do not show a strong change over time, while sample SP1 presents a pronounced ageing effect. In routine tests of the ILC organiser, the 1,4-naphthoquinone mother solution at such a high concentration is usually stable in a glass container for weeks, but here, a potential interaction with the PP tubes' inner surface may have happened. Consequently, ageing could be a variable of importance for the participants who analysed the samples toward the end of the required period, and such a parameter (date of analysis) was thus included in the parameters to be tested for the research of critical parameters.

Figure 1 Coefficients of variation of each participant (L1 to L21) for the three samples in triplicates tested using the RI-URBANS DTT protocol and the median and mean repeatability for each laboratory. Yellow-highlighted participants are the ones selected for the calculation of the assigned values. The dashed line indicates the participants with a COV lower than 20 %.

3.3 Statistical distribution of results: participants' variability

In order to assess the intra-laboratory variability in the results, the coefficient of variation (COV = standard deviation $/$ mean × 100) values of the results for each sample and each laboratory are presented in Fig. 1, while the standard deviations of the replicates reported for each sample are presented in Fig. S7 and Table S1. Overall, higher COVs are observed for SP3 and SP1, where most participants (44.4 % and 38.9 %, respectively) presented higher values for this sample compared to the SP2 sample. Specifically, high average COV values are observed by L1, L12, L14, L15 and L21, with an average variation higher than 40 %. Only a few participants (six groups) presented a variation lower than 10 % for the three samples. This is the same pattern for the results obtained by the ILC organiser during the homogeneity test (see Sect. 3.1), where the COV for SP1 was larger than those for SP2 and SP3 but with the highest COV below 15 %. These findings confirm more homogenous results for samples SP2 and SP3 compared to SP1 but could also indicate that some participants failed to achieve repeatability observed by the ILC organiser. However, some groups (L2, L10, L19, L20) were able to produce very homogeneous results with COV < 10 % for all three samples.

3.4 Laboratory performances

The assessment of laboratory performances first presents the bias in results across participant groups compared to the assigned values and their associated standard deviation for each sample. As illustrated in Fig. 2, SP1 exhibited the highest variances, ranging from 130 % to −35 %, with only five groups displaying differences within ±10 %. The distribution of results for SP1 indicated a mix of overestimations and underestimations. For SP2, differences are within a narrower range from 43 % to −7 %, primarily favouring overestimations. For this sample, 12 participants returned results that were within ±10 % of the assigned value (see highlighted laboratory numbers in Fig. 2). Finally, the results for SP3 demonstrated the least variation among participants, with differences ranging from 30 % to −6 % and 16 participants within ±10 % of the assigned value, again favouring overestimation compared to the assigned value. In total, 14 laboratories obtained data with a ±10 % difference from the assigned value for SP2 and SP3 (see highlighted laboratory numbers in Fig. 2). These results show again that the reference samples with 1,4-naphthoquinone (SP1) most probably present some characteristics leading to this variability and may be associated with a less-stable solution or led to the saturation of some detectors considering the relatively high concentrations, while the samples from filter extractions do not. Additionally, it is interesting to note that there is apparently no systematic pattern where a given participant would obtain out-of-range results for all samples. The results are really diverse, and most participants can obtain “acceptable” results for one or two samples (SP1–SP3) and larger variability associated with one “unacceptable result” for one of them. In 2020, Molina et al. (2020) explored the total uncertainty in OP DTT of a collection of samples and evaluated it to be 18 % for PM₁₀ and 16.3 % for PM_2.5. The leading factors identified were the DTT consumption rate (regression and repeatability of experimental data) and the extraction volume operations (pipette). This underscores the need for further investigations on the experimental causes of the variations observed, possibly in the next ILC.

Figure 2 Percentage differences from the assigned (consensus) value for each sample (SP1, SP2 and SP3). The results compared the average of the triplicates reported by the participants. Yellow-highlighted participants are the ones selected for the calculation of the assigned values and the underlined ones are those that obtained data with a ±10 % difference from the assigned value for SP2 and SP3.

The individual performance of each group was further evaluated using z scores. The results are presented in Fig. 3. All underestimations fall within the acceptable range (lower than −2z). Additionally, it is noteworthy that no laboratory exhibits unsatisfactory performances across all three samples; for almost all participants, while one sample can present poor results, it coexists with two acceptable ones. This has strong implications for spatial and temporal analyses often conducted for OP, emphasising the need to assert that there are no systematic biases in the analyses. While factors like sample inhomogeneity may be playing a role (particularly for SP1), some other issues, including variability in the performance of the analysis, may have an impact. Hence, participants exhibiting significant deviations (| z scores | > 2) for some of their results should thoroughly examine their procedures and possibly implement appropriate corrective actions to avoid similar outcomes in future ILCs.

Figure 3 Z scores were calculated to evaluate each participant's performance in the interlaboratory comparison for each sample tested. Yellow-highlighted participants are the ones selected for the calculation of the assigned values. Horizontal black and red lines indicate boundaries for triggering an action signal as described in Sect. 2.7.1.

However, half of the participants achieved results within the acceptable limits of this test. Despite disparities, these findings are really promising, especially considering that this is the first intercomparison of its kind. For instance, such results are in the same range as those obtained for some of the first ILCs for polycyclic aromatic hydrocarbons (PAHs; Grandesso et al., 2012; Verlhac et al., 2014).

To gain more knowledge about the factors causing the variability in the results, we first tried to perform a cluster analysis using the Ward method. This grouped the participants into four clusters (see Fig. S8), with the main cluster (in yellow) including the 10 participants encompassing mainly the ones with satisfactory z scores. The clustering seemed independent of the instrument used and/or the time taken between the sample delivery and analysis (i.e. near the delivery time or later in February).

Figure 4 Associations (beta in nmol min⁻¹ µg⁻¹) between OP DTT values for SP1, SP2 and SP3 using the RI-URBANS protocol and technical parameters, including the instrument used and the delivery and analysis time obtained by applying an adjusted multiple linear regression model. Coloured dots represent the results with p values ≤ 0.05, and white dots represent the results with p values > 0.05.

In a second step, a multiple linear regression model was run to evaluate the associations of the results obtained for the three samples, SP1, SP2 and SP3, considering a range of parameters, including the instrument used and the delivery and analysis time (Fig. 4, Table S2​​​​​​​). The beta values are shown in Fig. 4, representing the association (effects) between the different parameters evaluated and the OP results obtained. In the model, the reference variables were the RI-URBANS DTT SOP and the results obtained with the plate-reader instrument. Regarding the instrument performance, the values provided by the cuvette-type spectrometer were higher than those obtained with plate readers in the case of SP1 and SP2 (showing significant overestimation in the case of SP2 p values < 0.05), while the results for SP3 were quite similar. In the case of LWCC, higher variability is observed when compared to both cuvette and plate reader for all the samples. SP1 LWCC results presented the highest variability, and SP2 results were significantly overestimated (at a 95 % confidence level) when compared with those obtained by the plate reader. The RI-URBANS SOP was adapted for plate readers and cuvettes in order to perform the measurements in similar conditions of concentrations for the reagents. This was not the case for the LWCC since we did not have all the necessary information concerning the specific devices used by participants. Figure 4 suggests that the specific conditions of the reaction are probably important factors for delivering an accurate value of OP. In Fig. S6, we showed that SP1's OP activity decreased over time during storage, but this ageing effect was not found to be significant in the model for either delivery or analysis time. The storage effect remained consistent for SP2 and SP3, as there was a clear association between OP and delivery or analysis time (Fig. 4).

A ranking of the samples is also proposed to evaluate the OP activity of the samples tested in this ILC and its relative variability within the participants (only considering the results obtained with the simplified RI-URBANS protocol). For this purpose, SP1 was arbitrarily selected as the one with the highest OP activity with an assigned value of 100, and SP2 and SP3 were evaluated in function of SP1. Figure S12 shows the results obtained for the relative ranking of the samples. It can be noted that most of the participants presented similar relative variability with SP1 > SP2 > SP3. Some exceptions were observed for L1, L17 and L19, which obtained higher ranking for SP3 than SP2. A higher variability in the relative activity is obtained for SP2 than for SP3. Within the participants showing a higher relative ranking for SP2 (higher than 50 % compared to SP1), most of them used either cuvette-type or LWCC instruments (except L20), suggesting some overestimation in the results using these instruments. Overall, this similar ranking for the samples achieved by most of the groups is noticeable and very encouraging. In fact, most of the data treatment performed on OP with atmospheric variables or health data relies on associations and regressions where the relative variability of a time series is of utmost importance, more than the absolute value.

3.5 Comparison with other OP tests provided

Participants were also invited to report results obtained using other OP assays. Since not all participants submitted results from equivalent “home OP” tests, we exclusively focus on the outcomes obtained through the DTT-home protocols involving 13 participants (Table S3, Fig. S9). It is important to note that the RI-URBANS DTT protocol was simplified and does not include EDTA, Tris-HCl, TCA or Chelex^®, whereas DTT-home protocols are diverse and should exhibit at least one of up to three of the last-mentioned compounds at different steps of the reaction; however, this is challenging to evaluate because not all groups have submitted the protocols related to their home results.

Figure 5 Coefficients of variation of each participant for the three samples tested from triplicates using both the RI-URBANS DTT and the DTT-home protocols. The average and standard deviation of each participant are also detailed in Table S3.

We have first evaluated the COV individually from the results obtained with protocols (RI-URBANS SOP versus all DTT-home) for each test sample. Figure 5 shows that lower COVs are generally observed in the performance of the DTT-home protocols (more details can be found in Table S3). However, 6 out of 12 participants presented similar COVs (within 20 %) for the two protocols. These results could indicate that the use of a simplified OP protocol needs some extent of training and guidance before its application. In addition, some of the participants presented higher COV values (L1 and L13 for DTT-home) when using the LWCC instrument. The lack of a simplified protocol for this instrument did not seem to be a major issue, as the application of the DTT-home protocols was also associated with high COV.

Figure 6 Associations (beta in nmol min⁻¹ µg⁻¹) between OP DTT values obtained for SP1, SP2 and SP3 and the different parameters of the ILC, including the DTT protocol, the instrument used, and the delivery and analysis time obtained by applying a multiple linear model. Coloured dots represent the results with p values ≤ 0.05, and white dots represent the results with p values > 0.05.

Another multiple linear regression model was run to evaluate the main differences in the results obtained between RI-URBANS SOP and DTT-home protocols (Fig. 6). For SP1, a significant overestimation was observed for the DTT-home protocol and the opposite was observed for SP3. In the case of SP2, there was no statistically significant difference between both protocols. More details on the concentration of DTT for each DTT method could provide more insight into this trend. Regarding the instrument performance, the LWCC presents poorer results compared to the cuvette and plate reader for all the samples, which is opposite to the trend observed with the results of the RI-URBANS protocol only (Fig. 4). However, the results are significantly underestimated for SP3 only (p < 0.05). For the cuvette-based measurements, the results are higher than those obtained with plate readers in the case of SP1 and SP2 (significantly overestimated for SP2) and similar for SP3, which is in line with the direction observed with the RI-URBANS protocol only. The delivery and analysis time show a statistically significant lack of effect for SP2 (analysis time) and an underestimation for SP3 (delivery time) but nothing significant for SP1, although it had undergone ageing in the tests of the ILC organiser. Since the effects are very small compared to the effects of the protocol, or the instrument, these two variables (delivery and analysis time) may cause a greater impact on DTT values when protocols are harmonised but not to date.

The results obtained by the participants (the z-score evaluation) were also added to the former model to evaluate the effect on OP values while adjusting the protocol variables (Fig. S10). The results show a significant OP overestimation of all the samples for the labs with poor performances in SP2 and SP3 samples, as well as a significant underestimation in the OP value obtained for SP2 for the group with intermediate performance.

Hierarchical cluster analysis was conducted, incorporating both the RI-URBANS DTT and DTT-home outcomes (Fig. S11). Because some participants did not implement a DTT-home protocol, the cluster analysis involved a reduced set of OP values. The results reveal the presence of four primary clusters, with the predominant cluster encompassing most participants (8 out of 12 for DTT-home). The participants' results within the green main cluster largely align with those derived from the RI-URBANS DTT outcomes, encompassing the groups in the two primary clusters (Fig. S11). This assessment illustrates some consistency of results obtained across various OP DTT protocols. Some of the participants with more reliable results for the RI-URBANS DTT SOP maintain their consistency regardless of the protocol used. However, some of those that did not show an acceptable performance for the simplified protocol (i.e. L1 and L10) presented a better performance for DTT-home, and the opposite was observed for L19 (almost for SP1).

Figure 7 Z scores were calculated for the DTT-home protocol results to evaluate each participant's performance in reference to the RI-URBANS assigned (consensus) values for each sample tested. Horizontal black and red lines indicate boundaries for triggering an action signal as described in Sect. 2.7.1.

Finally, to assess the performance of the participants in the DTT-home protocols, a comparable approach to the simplified RI-URBANS SOP was employed for those participants who supplied OP results. The z scores were computed using the assigned values of each sample (SP1–SP3), obtained with the RI-URBANS SOP application. Figure 7 illustrates the z scores of the OP results obtained through the application of the DTT-home protocols, revealing a significant variation in the outcomes. Only five participants managed to produce satisfactory results for all the tested samples. Despite the fact that the COV of the participants using DTT-home protocols showed an improvement over the results of the simplified DTT SOP (Fig. 5), the outcomes are still distant from the consensus values of the samples obtained in this exercise. The results indicate a high degree of variability in the OP activity using home OP methodologies, underscoring the pressing requirement for standardised methods and harmonised protocols to ensure more reliable OP research.

4 Strengths and limitations of this first intercomparison

The greatest strength of this ILC was the high number of participants (20) enhancing the comprehensiveness and diversity of the study and allowing for a broader range of perspectives and expertise. This also allowed comprehensive collaborative discussions during the preparation phase, promoting knowledge exchange and consensus-building, as well as contributing to a more robust ILC design. These all show a willingness from the groups to be actively part of the development of the intercomparison and to pursue a harmonisation of the OP measurements.

The development of the first simplified OP DTT protocol (available in Sect. S1) also consolidates the experimental experience of the participants, fostering methodological consistency across different research groups as a first step toward method harmonisation. Finally, the collaboration with the JRC, an independent organisation for the assessment of results, adds credibility and objectivity to the study, ensuring that findings are impartially evaluated.

The sharing of liquid samples in this comparison comes with both advantages and limitations. On one hand, it eliminates biases associated with extraction methods and solvent purity. However, some samples exhibited signs of ageing during the interlaboratory comparison duration (though this was not identified as a critical parameter when identifying the main causes of variability). In addition, this approach introduced certain challenges with some of the home OP protocols which were designed originally to be used with solid samples. Finally, all liquid extracts should be provided with a similar PM concentration to limit known non-linearities and to avoid potential saturation issues, as can be the case with LWCC instruments. Next, ILCs should, in the future, include the whole chain assessment, including the extraction step.

The testing of three samples with different patterns makes it difficult to draw unilateral conclusions. A larger sample size could support a more robust statistical analysis of the results, particularly for the factors determining OP variability. Moreover, the inclusion of samples that are readily accessible worldwide, such as standard reference materials, could facilitate the future adaptation of a simplified OP DTT protocol while allowing comparison with an assigned (consensus) value.

The sole focus on the DTT method for the ILC could limit the broader evaluation of OP. The addition of other OP assays, such as those included in previous inter-comparison studies (Ayres et al., 2008; Calas et al., 2018; Shahpoury et al., 2022), could provide a more comprehensive understanding of the performances of the different groups involved in the OP domain of research. There are contrasting reports about the relative sensitivity of the DTT assay to various organic and inorganic PM components, with some studies showing higher reactivity towards the organic fraction. Therefore, additional consensus studies would be needed to assess this aspect and the comparability of DTT to other OP metrics that rely on proper lung antioxidants and could be considered more physiologically relevant. Such studies could support the identification of chemical species which should be prioritised for future air quality management programmes.

In conclusion, while this ILC of OP has highlighted considerable variability in the performance of the assay between groups, it has notable strengths and provides a starting point towards the harmonisation of OP measurements.

5 Recommendations for standardisation of OP protocols

Based on the findings of this ILC and also on the general literature about OP, some recommendations for further standardising OP DTT measurements are proposed (Table 1). These include guidelines for sample and laboratory conditions, instrument type and calibration, and the reporting of results. Additionally, a reference material (1,4-naphthoquinone, copper or other solution at a known concentration) should be proposed to facilitate future ILCs. Additionally, since some differences were observed in the results obtained from the use of the simplified SOP compared with the “home-developed” DTT protocols, the harmonisation of procedures is needed to ensure data comparability. We describe below the crucial parameters that need to be considered in the move toward greater harmonisation of OP methodologies.

Table 1 Summary of recommendations for future OP measurements on filters.

OP assay selection

• To date, it remains unclear which oxidative potential (OP) assay is most effective at predicting health outcomes related to oxidative stress. This uncertainty arises because different assays yield markedly different results for the same particulate matter (PM) samples. Additionally, even OP values from the same assay are often linked to various PM components and sources, depending on the different studies (He and Zhang, 2022).

  Thus, based on current knowledge and epidemiological evidence, two complementary OP assays (a thiol-based probe (OP DTT or OPGSH) and another one (OPOH, OPAA or another)) are recommended to provide a better picture of the potential oxidising damage from PM compounds and to strengthen the power of epidemiological studies. These aspects were previously discussed in a recent work integrating five different OP assays (Dominutti et al., 2023). Finally, the final choice of the best OP test (or combination) must be based on epidemiological evidence, which has begun to be studied only recently and needs more hindsight to be determined.

Sampling

• OP can be analysed in filter samples conventionally collected for air quality monitoring using small portions of these if adequately preserved (frozen). Pre-burn quartz filters or Teflon filters are appropriate, and blank filters must be measured to remove the background induced by the matrix of the material. A previous study had shown no differences in the OP DTT values observed using Quartz or Teflon filters (Frezzini et al., 2022). However, it should be further evaluated when other OP assays are considered.

Sample storage

• Previous studies that evaluated the effect of storage time and conditions did not show a substantial effect on the OP DTT results (Frezzini et al., 2022). However, we recommend that PM samples should be immediately transported to the lab after sampling. The filters must be kept cold after sampling (at 4 °C if the OP analysis is done within a few days after collection or −20 °C if the analysis is delayed).

• The lifetime of the ROS may be very short, and measurements of OP on PM-extracted filters are likely affected both by the age of the samples, how they have been sampled and stored, and the nature of the extraction methodology. How all of these processes have an impact on the ageing of samples and the ultimate quantification of OP needs to be addressed. Ageing studies should be performed for each OP assay in the long term to define the maximum storage time of aerosol filters at low versus ambient temperature conditions.

  Some OP components might be so short-lived that only online techniques are warranted.

Laboratory conditions

• OP assays are “trace” detection assays that require clean ambient conditions and high-quality reagents free of metal contamination. Considerations should be given to the use of certified clean rooms or proper laminar flow bench stations to prevent contamination of the samples.

• Regarding the use of clean material, vials, cones and spatulas have to be washed before use (5 % HNO₃ bath to remove metals and rinsed three times in ultra-pure water before drying in laminar flow).

• Control laboratory temperatures and light exposure by using dark polypropylene tubes at least for reactants

Extraction step

• The extraction step may be highly variable according to the procedures used, and several parameters are known to have an impact on OP results, such as the choice of the solvent, the concentration of buffer, the way of agitation and the quantification of the final extracted mass. Notably, the ultrasonication of PM samples in aqueous solutions generates ROS (Miljevic et al., 2014), and it could introduce artefacts in OP measurement. This effect was also observed in the work of Frezzini et al. (2022), where different extraction methods were evaluated, with ultrasonic baths overestimating the results observed.

  The effect of the solvent used was not evaluated in this ILC exercise. However, we recommend the use of ultra-pure water or simulated lining fluid for the sample extraction. Future ILC exercises should include the evaluation on the extraction conditions, including solvent use and methods.

Reaction step

• Several aspects in the reaction process affect the OP value, like the initial concentration of reactants (since the DTT test is mass-dependent; Charrier et al., 2016), ratio of reactant$/$sample, time of reaction (some compounds present a non-linear reaction over time), the temperature of the reaction (which should be standardised to 37 °C), agitation (mixing samples) and the type of measurements (kinetic or end-point value), etc.

• The current literature mainly addresses the extraction or reaction parameters separately. We advise that the whole chain factors should be evaluated together to quantify their relative impact on the results.

Development of a reference material with a certified “OP value”

• The set-up of reference material or in-house standard solutions (in collaboration with reference institutions JRC or NIST, for instance) with a known OP value could help laboratories test and train themselves on the OP protocol before testing the unknown ILC samples. This is something to be developed and tested in future ILCs.

Instrument calibration

• Investigate the optimal frequency for the calibration of spectrophotometers for such assays.

Report of results and units

• The calculation of OP DTT activity during this ILC involved a conversion using a calibration curve. Since the OP activity measures the rate of a chemical reaction and not a concentration, for future comparison exercises, the possibility of exploring alternative methods for OP calculation should be tested. To date, results are mass-normalised in nmol antioxidant min⁻¹ µg⁻¹ or volume-normalised in nmol antioxidant min⁻¹ m⁻³. The OP per microgram refers to the reactivity of 1 µg of the tested PM, whereas the OP per cubic metre refers to the exposure of 1 m³ of inhaled air.

6 Conclusions

This study represents an innovative effort as the first interlaboratory OP exercise specifically aimed at harmonising this OP assay. This exercise provides the very first roadmap for refining interlaboratory comparisons of OP, fostering greater confidence in the reliability of OP data and encouraging the scientific community to advance towards global OP harmonisation.

This first exercise focused on OP DTT, as it is widely used within the scientific community and has already shown positive associations with health outcomes (Bates et al., 2015; Borlaza et al., 2022; Dabass et al., 2018; Donaldson et al., 2001; Gao et al., 2020; Marsal et al., 2023; Weichenthal et al., 2016b, a, c). Even if there are several crucial points to be evaluated and harmonised in the whole chain of the determination of OP (sampling methods, sample storage, extraction conditions and methods) as well as the use of different OP assays, this first ILC engaging several research laboratories paves the way for future developments towards the standardisation of OP methods.

Our findings emphasise both the strengths and challenges associated with the use of the current OP DTT assay for driving a measurement of PM OP. Overall, half of the participants achieved results falling within a satisfactory range of z scores for this test. The participating group performance levels are comparable to those observed in initial ILCs for PAHs in the 2010s (Grandesso et al., 2012; Verlhac et al., 2014). While notable agreement was observed in certain samples and between several groups, discrepancies and variability were also identified, emphasising the need for harmonisation in the procedures and conditions. A number of factors may contribute to the underperformance observed in certain samples and participants. The main reasons are not clear, but the analysis conditions in the participating laboratories and the lack of experience in this type of metrological exercise are possible causes. The standardisation of protocols and harmonisation of procedures emerged as critical components to ensure the accuracy and comparability of OP data across laboratories. This collaborative approach fosters a more robust OP science, facilitates data exchange and integration, and will ultimately contribute to a better understanding of the health impacts associated with PM exposure, allowing for more accurate exposure assessments and regulatory decisions.