Altitude profiles of reactivities and J-values averaged over 24 h, 5 days in August, and latitude blocks (50–20∘ S, 20∘ S–20∘ N and 20–50∘ N) are shown in Fig. 1 (6 models, 3 blocks, 18 profiles per panel). As expected for August, the 50–20∘ S values are very low, while the 20∘ S–20∘ N and 20–50∘ N ones are equally high. This basic latitude-season pattern holds across all models. The variability across the five separate days in the UCI model (Fig. S2) is primarily a smooth trend through August reflecting the changing solar declination from 18∘ to 12∘, but instances of highly variable cloud fields occur, even when averaged over 30∘ in latitude.

For J-O1D, five models (GFDL, GSFC, GC, NCAR, UCI) agree well over all pressures and latitude blocks, but NCAR is, unusually, 10 % higher only in the 20∘ S–20∘ N block. J-O1D from GISS is 80 % larger than other models for all pressure and latitude blocks, but this does not translate directly or simply into reactivities, where GISS L-O3 is higher (expected) but L-CH4 is lower (unexpected). For J-NO2, model differences are not so great and show largest values at 20–50∘ N consistent with the longer summer daytime hours. The spread in J-NO2 is partly understandable because of ambiguous choices in interpolating the temperature dependence of recommended NO2 cross sections and quantum yields (i.e., the absorption cross sections are given at 220 and 294 K; the quantum yields, at 248 and 298 K; and the choice of whether to interpolate linearly or logarithmically, or whether to extrapolate or not, affects J-NO2, especially in the upper troposphere). This ambiguity does not exist for J-O1D recommended cross section and quantum yields. J-O1D is strongly dependent on the overhead O3 column, and the zonal mean total O3 column from the models is compared with recent satellite measurements in Fig. S3. NCAR's O3 column is anomalously lower only in the 20∘ S–20∘ N region and likely explains their higher J-O1D noted above.

Reactivity profiles for the five non-GISS models show excellent agreement for P-O3 but noticeable differences for L-CH4 and even larger ones for L-O3 (Fig. 1). The altitude profiles are similar for the five models, indicating that the cause of the L-O3 spread is likely related to HOX. The GISS results are anomalous, with much higher P-O3 and an L-O3 vs. L-CH4 relationship that seems counter to known chemistry in which both L-O3 and L-CH4 maximize with the high HOX values in the warmer, wetter, lower troposphere of the tropical Pacific.

We examine the relationship between the three reactivities in each model with scatter plots of P-O3 and L-CH4 against L-O3 in Fig. 2. Each plot has 14 880 points (5-day parcel means) and is split by location: 60–20∘ S and 20–60∘ N (extra tropics, gray); tropics upper (20∘ S–20∘N, p < 600 hPa, cyan) and lower (p > 600 hPa, blue). Percentiles (10th, 50th, 90th) in each dimension are plotted as red dash-dot lines, and thus most points in the well correlated L-CH4 vs. L-O3 lie along the 3 quasi-diagonal intersections of red lines. The right-angle separation of high P-O3 and high L-O3 in the tropics reflects the high NOx (P-O3) in this data stream is in the upper troposphere and the largest L-O3 is from wet environments of the lower troposphere. GFDL has the most compact distribution of parcels and GISS, the most scattered. Four models (GSFC, GC, NCAR, UCI) have remarkably similar patterns in terms of the percentiles and structure, e.g., for L-CH4 vs. L-O3 they show the lower tropics dominating the upper part of the distribution and the extra-tropics, the lowermost points. GFDL has similar percentiles for P-O3 and L-CH4, but a much smaller spread for L-O3 that explains their compacted scatter plots. GISS is unique with much larger spread in both P-O3 and L-O3 but a compressed distribution in L-CH4. From these scatter plots, we can say that the four models are remarkably consistent, that GFDL is similar but should reexamine their L-O3 diagnostic, and that GISS has a “uniqueness” in its L-O3 vs. L-CH4 relationship as well as large scatter in both P-O3 and L-O3. While consistency does not guarantee correct implementation of the photochemical model (i.e., rate coefficients, cross sections), uniqueness is something that needs more investigation as it may be an error or may lead to fixes in the “consistent” models. Scatter plots of J-NO2 and J-O1D vs. L-O3 (Fig. S4) show similar J-value statistics for the five non-GISS models, and all models show a similar location of the three sets of points (extra-tropics, lower-tropics, upper-tropics) within their own percentiles.

On a parcel by parcel basis we compare in Fig. 3 the 5-day means from all six models against the reference case for the three reactivities and two J-values. If the models were all alike, they would fall tightly on the 1 : 1 line (black dashed). In each panel there are 89 280 points, with many overlapping. The order of plotting (shown by the legend) is important for visual impression since the latter points often overlie the earlier ones and the choice of order was based partly on the rms differences, with greatest first and smallest last. Here we can clearly see the type of scatter, the pattern of discrepancies across models, and at what levels of reactivity such discrepancy it occurs. It provides a focus for model development: UCI should reexamine its J-NO2 at the higher values and its P-O3 in the 1–3 ppb day-1 range; NCAR should examine why it has so much scatter in L-O3 and L-CH4 (see discussion of T and q later); GFDL has similar scatter (see T and q) but also has a low-bias in L-O3; and GISS has many differences that can be examined. As a cross-model question, are the above-the-line (UCI) and below-the-line (GSFC) differences in P-O3 and L-CH4 related to the same pattern in J-NO2?

A simple summary of these statistics – averages and rms differences relative to the reference case – is given in Table 3. We have selected (italics) those entries that seem anomalous as also found in Fig. 3. For example, average P-O3 ranges from 0.77 to 0.84 ppb day-1 for five models but is 1.40 ppb day-1 for GISS. Likewise, average L-O3 ranges from 1.44 to 1.54 ppb day-1 for 4 models, but is 0.83 for GFDL and 2.25 ppb day-1 for GISS. The rms differences with respect to the reference case favors the three models that define that case, but also shows that GFDL and NCAR are close to the reference case for P-O3, but farther away for L-O3 and L-CH4 probably caused by their T and q values (see later).

The variability of the five days in August tells us about the synoptic variability of clouds and possibly O3 columns in each model. The rms difference between the five individual days and the 5-day mean for each model (Table 3c) shows that GISS and NCAR have much larger variability in reactivities, caused by and mirrored by those in J-values. These rms differences in J-values for GISS and NCAR are surprising. Collectively, we should reexamine this variability in all the models to ascertain its cause. In general, the slopes of the individual vs. reference model for reactivities are close to 1 (Table S2) because the slope is determined by the large gradients with latitude and pressure that most models agree on. In comparing individual days vs. 5-day mean, it is encouraging that this slope averages 1 ± 0.04 for all reactivities and models (using each model's 5-day mean as its reference case, Table S3). Also, the slope decreases from about 1.01 to 0.96 through August as expected with declining photolysis rates in the north.

The rms difference across the five days is also a measure of how well the 5-day parcel mean can represent the true chemical model. Assuming that the cloud variability is random, the 5-day means with respect to other models are not really different unless that model–model rms exceeds some fraction of the day-to-day rms of the models involved. Using the UCI test with different model years, we find that the year-to-year rms differences are about two thirds of the day-to-day rms over 5-days. Thus, we cannot be sure that the rms differences between NCAR and the reference case are due to the inadequacy of the 5-day mean to represent the mean NCAR chemistry model (Table 3b, c). Conversely, some other source of model error is likely responsible for the large day-to-day rms.

Following the “which air matters” theme of P2017, we look at the more reactive air parcels to find out if the models agree on these. For each reactivity, we sort the 5-day parcel means in increasing order and integrate the cumulative reactivity. The value at 100 % (all 14 880 parcels) is equal to the average reactivity of the sample (Table 3a), and this is renormalized to 1 for comparison across models (Fig. 4, Table S4). With sorting, these curves must be monotonic and convex. The steeper the curve, the more important the top reactive parcels are in determining the total. For most models, these reactivity curves are remarkably similar and fall within the range seen for five different days with the same model (UCI, Fig. S5, Table S5). Focusing on the upper 10 %, the outliers are unusual and reactivity specific: for L-O3, GFDL is much steeper that the other models, consistent with the feature identified earlier in the scatter plots; and for L-CH4, GISS is much shallower. Surprisingly, with this diagnostic GISS is not an obvious outlier for P-O3 and L-O3 as seen in previous comparisons.

From this cumulative reactivity figure, one can see that the top 5 % of parcels comprise 15 % of the total reactivity, effectively a slope of 3 : 1. With the exceptions noted, total reactivity for the top 5, 10, 25, and 50 % of the parcels (Table S4) is similar across models and across days within a model (Table S5). Focusing on the top 10 % of parcels for each reactivity, we plot their latitude-by-pressure distribution for each model in Fig. 5. Top P-O3 are in the upper troposphere where NOx was highest in the specified data stream; and top L-O3 and L-CH4 are in the lower troposphere associated with warmer temperatures and higher water vapor, with L-CH4 being at lower altitude than L-O3 (all models except GISS). There is a region of top P-O3 parcels about 40∘ N that extends into the lower troposphere, although the shape varies across models. The vertical pattern of top-10 % parcels about 22∘ S clearly varies across models with GISS-GC-UCI not selecting these parcels.

Overlap of these three sets of parcels are quantified as Venn diagrams for each model in Fig. 6. Very few top-10 parcels are in the triple-overlap area (1–10 %); but when P-O3 parcels coincide with either L-O3 or L-CH4 parcels, they generally lie in this triple-overlap area. The only major exception to this pattern is GISS. In terms of L-O3 and L-CH4 overlap, 4 models are very consistent (76–80 %); but GISS is unusually low (49 %) and GFDL is unusually high (93 %). These patterns help identify distinctly different chemistries in these models that have been identified with other diagnostics. The Venn overlap diagrams will become more interesting with an observational data stream as they point to the co-occurrence of unusual atmospheric parcels.

At what level do the models agree on the hot, top-10 % parcels? We use the reference case defined above and sort each reactivity to identify the top-10 %, retain those parcel numbers and compare across models. Table S6 gives each model's overlap of their top-X % parcels in terms of the percent that also occur in the top-X % reference case. For a range of X, 5, 10, 25, and 50 %, the overlap increases successively with many models having 90 % overlap for the top-50 %. The exceptions are GFDL with lower than typical overlap for L-O3 at all top-X % levels, and GISS, with lower overlap for L-CH4. This new diagnostic is helpful in understanding these model differences because it implies that the L-O3 and L-CH4 differences identified previously are not caused by a systematic offset in all parcels, but rather by a selection of different parcels.

As expected, the three models GSFC-GC-UCI that define the reference case all have about 90 % overlap for the top-10 % parcels, and so we do not learn much with this. In terms of linking models with similar chemistries, probably 80 % overlap is a good mark, because we see that the different UCI years drop off to 85 % in L-O3 and L-CH4. Overlap in P-O3 is much easier to achieve as the few high-NOx parcels drive high P-O3 in all models: at the top-25 % parcels, the P-O3 overlap is about 84 % or better for all models.

On a day-to-day basis, we examine the top-10 % overlap for GC–GSFC–NCAR–UCI models, using their own 5-day mean as the reference (Table S7). Cloud variations across the five days lead to overlaps for the top-10 % parcels ranging from 78 to 92 % at best. NCAR has similar self-overlaps for P-O3 but only 58 to 72 % for L-O3 and L-CH4, because the modeled T and q changes with each day in August and greatly reduces the overlap of the hot parcels. This further supports T and q as being important drivers of L-O3 and L-CH4. The use of 5-day calculations with varying cloud fields is essential in identifying the top reactive parcels.

We plot the modeled reactivity of individual model 5-day mean parcels in ascending order based on the sorted top-10 % parcels in the reference case (Fig. 7). Hence the reference case (black line) is a monotonically increasing curve; while the individual models produce a scattered distribution of points. As expected, the three models defining the reference case have some scatter but mostly overlap with the reference case. UCI is typically higher and GSFC is lower. For J-NO2 in these most reactive parcels, UCI is notably higher as is GISS, a result seen in the average profiles (Fig. 1), but it does not affect the reactivities. The mean bias of models relative to the reference case is also seen in Fig. 7 with the offset of the points. The results here are similar to what has been identified earlier: GISS has unusual offsets for all reactivities and J-O1D; agreement for P-O3 is much better than for L-O3 and L-CH4; four models show the upward curve matching the top-1 % parcels; for L-O3 and L-CH4, GFDL-NCAR have a flat scatter of points and miss the upward curve because they reset the q of the data stream. Day-to-day scatter for the top-10 % (defined by the 5-day mean) is tested with the UCI model in Fig. S6. This one-model synoptic cloud variability has similar scatter to that seen for the more central models (Fig. 7) including the rapid increase in L-O3 at the top-1 % and the much greater scatter in J-NO2. The year-to-year variability in the top-10 % parcels is shown (Fig. S7) for the UCI model with year 2016 as the reference case (solid line) and years 1997 and 2015 as separate models. The patterns of scatter here are similar to but less than the day-to-day (Fig. S6), again showing the importance of 5-day averages, and identifying the lower limit of scatter at which this diagnostic can discern differences in model chemistry. Overall, the top-10 % J-O1D parcels (all in uppermost troposphere) have better agreement than the top-10 % J-NO2 which are more sensitive to clouds.

How interannual variability might affect the results is tested with the UCI CTM running the simulated data stream for five August days using years 1997 and 2015 meteorology to compare with year 2016 (see previous Tables, Table S8, and Fig. 8). The scatter plots in Fig. 8 do not look much different from those for the three models used in the reference case (Fig. 3). For the 5-day parcel means, the rms differences across any pairing of the three UCI years is about 8–10 % of the average reactivity, which is about half of that across the three models used in the reference case. Using this criterion (< 20 %) for distinctness, we effectively have only four independent distinctly different models here: GFDL, GISS, NCAR and the GSFC–GC–UCI group. The four models all differ from one another at the 30–100 % level of the UCI year-to-year variations. However, in terms of the overall average reactivities (Table 3), the different years of the UCI model are almost identical (< 1 %), while the differences across the three reference models are much larger (±5 %) and clearly distinguishable.

How the time-of-day of parcels in the data stream might affect reactivity is tested with the UCI model initializing the calculation at midnight (12:00 UT) instead of noon (see Fig. S8, Table S8). In this study, we chose parcels at 180∘ W and, since the global models begin each day at 00:00 UT, the photochemistry starts at local noon. A measurement data stream, such as from ATom (2017), will include measurements over a range of longitudes and taken with a wide range of local solar times. We need to ensure that the protocol here does not depend on when the 24 h integration of reactivity is initiated. The UCI model selected one day (16 August 2016) and shifted the local solar time by 12 h, thus initiating each parcel at local midnight. In addition, the cloud fields needed to be rearranged so that the pairing of clouds and solar zenith angles were the same in both cases. The start-at-midnight version has larger reactivities by at most 1 % with no changes in the J-values as expected for the protocol (e.g., keeping the morning clouds in the morning for both calculations). The rms differences between the two cases are 2–10 times less than the year-to-year differences. We conclude that the initiation time produces discernible differences but not at the level to affect the any of the results here, even with high levels of lightning-NO in daytime. The initiation time might affect highly polluted regions where the NOx reservoirs could be converted at night to less photolabile nitrates.

Two additional sensitivity tests included running the five days in August with a fixed solar declination (Fig. S9) and with different restart file (Fig. S10). As shown in these figures and Table S8, these two tests change the overall average in the fourth decimal place and have rms differences < 0.01 ppb day-1. For these choices, the protocol adopted here is adequate.

The GFDL and NCAR CCMs could not maintain the fixed, data-stream T and q values over the 24 h integration, which leads to larger rms differences because reactivities depend on both T and q. This explains in part why the GFDL and NCAR models in Fig. 3 have larger scatter for reactivities than the other non-GISS models, but similar scatter in J-values. This effect may also contribute to the larger day-to-day rms, for NCAR at least, and is examined more extensively with the UCI CTM running with the T and q's from both models (Sect. 3.5).

How overwriting of the data stream's T and q (with a CCM climate) impacts these results is tested with the UCI CTM re-running a one day (16 August 2016) data stream using T and q's reported from the GFDL and NCAR models. The rms reactivity differences for these two models are 2–3 times larger than those of the reference models (GSFC, GC, UCI, see Table 3); while J-value differences (much less affected by temperature) are similar.

For the five days, each with 14 880 parcels, the mean values of either GFDL or NCAR T and q's are similar to the data stream but their rms differences are large: about 3.6∘ K and 0.4 in log10(q), see Table S9. Both models have similar scatter patterns for T and for q (Fig. S11) with a number of parcels having log10(q) more than a factor of 10 different from the stream. In this sensitivity test, UCI CTM ran with just T from GFDL and NCAR, and then with both T and q (4 cases in all). The results are shown in Tables S8 and Fig. 9. For T alone, the reactivity differences were at the lower limit of detectable model–model differences but, with both T and q, the model showed surprisingly large shifts in L-O3 and L-CH4 along with standard deviations 2–10 times larger than the lower limit based on different UCI model years. In fact, the UCI model using GFDL and NCAR T and q has about the same rms reactivity differences with respect to the reference case as do the full models (Compare Tables S8 and 3, noting that Table 3 is a 5-day mean result and not 1-day result). Thus, without a model being able to use the specified T and q, we are unable to determine if its photochemical module is similar to another model. Moreover, with climate-varying T and q's the modeled reactivities from an observed data stream will also be too noisy for an analysis of the top-10 % parcels, i.e., which air matters.