First national intercomparison of solar ultraviolet radiometers in Italy

A blind intercomparison of ground-based ultraviolet (UV) instruments has been organized for the first time in Italy. The campaign was coordinated by the Environmental Protection Agency of Aosta Valley (ARPA Valle d’Aosta) and took place in Saint-Christophe (45.8 ◦ N, 7.4 E, 570 m a.s.l.), in the Alpine region, from 8 to 23 June 2010. It involved 8 institutions, 10 broadband radiometers, 2 filter radiometers and 2 spectroradiometers. Synchronized measurements of downward global solar UV irradiance at the ground were collected and the raw series were then individually processed by the respective operators on the base of their own procedures and calibration data. A radiative transfer model was successfully applied as an interpretative tool. The input parameters and output results are described in detail. The comparison was performed in terms of global solar UV Index and integrated UV-A irradiance against a wellcalibrated double monochromator spectroradiometer as reference. An improved algorithm for comparing broadband data and spectra has been developed and is discussed in detail. For some instruments, we found average deviations ranging from−16 % up to 20 % relative to the reference and diurnal variations as large as 15 % even in clear days. Remarkable deviations were found for the instruments calibrated in the manufacturers’ facilities and never involved in field intercomparison. Finally, some recommendations to the UV operators based on the campaign results are proposed. Correspondence to: H. Diémoz (h.diemoz@arpa.vda.it)

Introduction: "... health and the environment (UNEP, 2010).Well calibrated groundbased networks on a continental and national level are also essential in order to monitor the effective behaviour of the solar UV radiation in the next years on a small spatial scale (McKenzie et al., 2011).This will help to control whether the Montreal Protocol targets are fulfilled and the model predictions are correct.Moreover, quality ground-based instruments, traceable to a common reference, allow to validate satellite data mainly over polluted locations (Lee-Taylor et al., 2010) or at sites with a complex orography.";"... the campaign gives also additional information about the effectiveness of different correction procedures and the accuracy of calibration coefficients.Although this work involved instruments operated by Italian institutions, some results may be useful also for other operators.First, the variety of the radiometers participating to the campaign and the different calibration and processing procedures make the comparison representative of a wider community than the Italian one.Most radiometers (11 of 13) are commercialized worldwide.Thus, the comparison can provide a contribution to the knowledge about the general performance of narrowand broad-band UV radiometers, their characteristics, their operation and limits.Furthermore, some radiometers participating to the campaign were calibrated few months before the comparison by their respective manufacturers.Thus, the results of the comparison may be useful to obtain some information about the effectiveness and the consistency of the calibration procedures adopted by the manufacturers.Finally, the paper presents an in-depth analysis of the algorithm used to compare broadband and spectral UV data, that can improve the theoretical basis of the methods previously reported in literature".

C1172
Results and discussion: 5.5 Overall performances of the radiometers and recommendations.
Even within the uncertainty of the reference instrument, some inferences about the general performances of the radiometers can be drawn and some recommendations may be consequently formulated.
Figure 11 (1, in the Supplement) shows both the relative deviation from the reference during clear sky days and the IQR distance.Only results from the UV Index comparison are represented (UV-A irradiances have a similar behaviour and are omitted).Angular and spectral corrections employed by the operators are also reported in parentheses next to the instrument id.Some groups of instruments, which show similar performances, can be identified.
One of the most striking features of the graph is the importance of the traceability.Indeed, most of the instruments traceable to NTP (directly through QASUME or, with a further step, through Bentham 5541), with the only exception of instrument 12, show very low deviations with respect to the reference (blue markers).This points out the importance of a common and reliable reference scale and the effectiveness of a travelling standard such as QASUME.It is interesting to notice that the relative deviations from the reference among this group do not appreciably depend on the age of the last calibration.Broadband radiometers such as id 06 and 07 are stable, even after some years of operation after their last calibration.However, sudden changes may occur and calibrations on a more frequent basis are recommended.
Instruments which were calibrated by their respective manufacturers few months before the campaign may be included in a second group (red colour).This group is characterized by moderate to large deviations relative to the reference.Moreover, consistency among instruments calibrated by the same facility (e.g.id 05 and 04) is weirdly low.Furthermore, instrument 04 was recalibrated at the same laboratory after the comparison and is now only about 4% higher than the reference.Such large discrepancies C1173 remain unexplained.
Finally, a third group including two radiometers designed and built by ENEA and CNR-ISAC is highlighted (violet colour).Large relative deviations with respect to the reference are found.The intercomparison helped in identifying technical problems such as deterioration of internal components and drifts in calibrations.Thus, particular attention should be taken when operating with home-made radiometers and comparisons with reference instruments should be scheduled very often.
A further classification may be done according to the processing procedures and correction algorithms.Generally, radiometers processed using a matrix (which takes into account both spectral and angular corrections) show a lower IQR.On the contrary, other kinds of corrections, such that employed for radiometer 08, are not so effective.When a fixed calibration factor is used, the IQR may be even worse, as in the case of radiometer 07.Even though low daily variations may be obtained (as with instrument 10) with a fixed calibration factor, possibly because of a good cosine response, both spectral and angular corrections should be always employed.
Generally speaking, performances of the radiometers, especially of those with the highest deviations or IQRs, could be greatly improved by using state-of-the-art correction algorithm and planning frequent intercomparison for monitoring the instrumental stability.
Summary and conclusions: "... participated to the campaign.Most radiometers (11 of 13) are commercialized worldwide.The campaign also represented a chance of comparing several data processing algorithms employed by the participating institutions.";"... even for clear sky days were discovered and discussed.Three radiometers, which had been calibrated few months before the campaign by their respective manufacturers, showed significant deviations to the reference.The average deviations and daily variations were very large ...".

C1174
Comment #2: Furthermore, the analysis procedure used in this manuscript is different for particular instruments (Sections 4.1 and Sections 4.2) which renders the comparison between these subsets a questionable task.I recommend the authors to use a common analysis approach for all instruments in the campaign, even if the analysis of particular radiometers could be improved (e.g.Section 4.1).

Answer #2:
The following sentence will introduce Sect. 4 of the revised manuscript to explain the need of different analysis procedures: Since the instruments to be compared against the reference belong to very different classes (spectral, narrow-and broad-band radiometers), a common procedure of analysis to be applied to all instruments is not suitable.Most notably, since the sampling frequencies of the instruments are considerably different, a common approach for all instruments involved in the campaign should be based on the adoption of an identical time resolution (i.e.downscaling all data to the lowest resolution among the participating instruments) which is inappropriate and can lead to ambiguous results.Indeed, the deviations to the reference would increase due to the temporal interpolation, making arduous to discriminate the dispersion originating from other factors and the dispersion from the interpolation itself.
A reliable and rigorous analysis is essential to correctly compare the processing procedures used by each participating institution, which is one of the purposes of the campaign.Therefore, an appropriate algorithm must be employed for each class of instruments for comparing the UV data recorded by the various instruments against the reference and reducing the dispersion originating from the temporal interpolation.The methods are described below.

C1175
Comment #3: The analysis procedure described in Section 4.1 for a subset of the UV Radiometers of this comparison is very elaborate and reference is made to a procedure used in a previous campaign (COST726).A significant difference between the COST726 and the present campaign is that the former was also used to calibrate the radiometers while the one described here was limited to comparing the results from the UV radiometers applying the calibration from the home institutes.Thus I expect the analysis approach to be fundamentally different.In that sense I believe that Section 4.1 contains unnecessary relict information from the COST726 intercomparison and should be considerably simplifed.Lines 12-21, including equations 2-4 are unnecessary for the analysis of the present campaign and should be omitted.Answer #3: Section 4.1 has been rewritten omitting relict information from the COST726 intercomparison, according to the referee suggestion (text from line 17 p.2800 to line 18 p.2801 was omitted).The text has been revised in the following way: ... an appropriate algorithm was developed.
The downscaled irradiance from broadband radiometers was calculated as where I BB (t(λ)) are the CIE-weighted and cosine-corrected broadband irradiances reprocessed by the operators and I 0 SP (λ, t(λ)) is the clear-sky irradiance simulated by the radiative transfer code at the time at which the reference instrument is measuring the wavelength λ.

C1176
We also assume that clouds act as a grey filter...

Comment #4:
The main benefit of the elaborate analysis procedure in Section 4.1 is related to broken cloud conditions when the solar spectrum measured by the spectroradiometer is significantly affected.In that case however, also the broadband measurements are difficult to estimate due to the questionable way of correcting for angular response deviations (cosine corrections).As stated in 5.4, only a clear sky cosine correction was applied which is simplistic and in marked contrast to the elaborate comparison methodology.It would be interesting to determine the effect from that simplification on the performance of the radiometers during broken cloud conditions.Answer #4: Section 5.4 has been rewritten as follows in order to better explain the benefits of the algorithm and to show that the analysis of IQRs allows to estimate the effect of the clear-sky simplification on cosine corrections, at least for analog broadband radiometers: The effect of clouds on the ratios can be examined taking into account the series of measurements recorded in the whole campaign period (days 159 to 175), in both cloudy and clear days (Figs. 9 and 10).While median values do not change appreciably, the scatter of the ratios for each instrument and, in particular, the difference between the maximum and minimum values (i.e. the distance between the whiskers in the boxplots), increase noticeably and in many cases exceeds ±10 %.This is to ascribe to many factors, depending on the instruments.First, the analysis procedure which was applied to the analog broadband data allows to reduce the effect of time interpolations on the ratios.The dominant factor modulating the ratios must therefore be the influence of clouds on the cosine corrections, which were calculated under the hypothesis of clear sky conditions.This latter approach is, of course, simplistic, but it is adopted by most of the broadband operators worldwide.Also, more rigorous corrections would require cloud cover data which are not always available.For instance, a complete spectral correction in cloudy cases would strongly depend on both cloud cover and cloud optical thickness (Mateos et al., 2011).A simpler correction is possible in the borderline case of overcast sky in presence of thick clouds, taking into account only the fraction of scattered radiation while neglecting the direct beam.Nevertheless, this diffuse correction would require a complete radiometer characterization (angular and spectral responses), which was not available for all radiometers.However, the increase of the IQRs for analog broadband radiometers during cloudy conditions with respect to clear-sky conditions is a good estimate of the error on the cosine correction originating from the clear-sky simplification.It is interesting to notice that radiometers with a good cosine response (i.e. with a smooth matrix), for example id 01, 02, 04 and 05, show slightly better performances compared to the others, e.g.id 03 and 06.
In the case of instruments with coarser time resolutions (e.g.09 to 11 and, to a lesser degree, 12 and 13), time interpolations between measurements may produce large deviations.However, a more in-depth analysis of cloud effects on the cosine corrections for these instruments is beyond the purpose of this paper and may be studied in a future work.
Finally, it is interesting to notice that the error is amplified by the processing algorithms applied on radiometers 07 and 08.In those cases, the full range of the ratios may exceed ±20 % or even ±30 %.We also assume that clouds act as a grey filter, i.e. the spectral transmittance of the cloud at all wavelengths can be approximated by the transmittance integrated over a wavelength band (e.g.UV-A or the erythemal range).Marking the modeled clear-sky irradiance over the spectral band of interest with I 0 BB (t), we obtain: As previously stated, a spectral scan of the reference instrument takes about 3 minutes.During this time, the variation of I 0 BB (t) depends uniquely on the change of the solar zenith angle and can be considered negligible with respect to the fluctuations of the measured irradiance, I BB (t), which is induced by the transit of broken clouds.Based on radiative transfer calculations, the error in considering I 0 BB (t) as a constant is estimated to be lower than 1%.Thus, we obtain: Comment #6: Section 4.2.The use of cubic splines is very handy, but it can produce nasty surprises when applied automatically.Furthermore, how do you know that the radiation changes like a cubic spline in between the missing measurements?I would recommend the use of a simple linear interpolation.
Answer #6: The authors intended just to point out the risk of an indiscriminate use of time interpolations (including cubic splines).Introduction to Sect. 5 has been rewritten as follows: C1179 ... Figure 4 presents a comparison of several methods to downscale the data from broadband radiometers during three cloudy days.The aim is to draw attention to the risk of an indiscriminate use of time interpolations.First, the ratios derived by interpolating the broadband values to the most representative time of the reference spectra using a cubic spline and a simple linear interpolation are shown.Cubic splines are chosen since they were applied to depict the results of the COST726 campaign (G.Hülsen, personal communication).As can be seen from the figure, the results obtained in cloudy conditions are not optimal and present some fictitious fluctuations.The linear interpolation gives results similar to the cubic spline.The new algorithm developed in this study ... Figure 2 of the Supplement will replace the original in the paper.
Comment #7: Section 5.1, lines 11,12: Please provide a description of how the radiometers were calibrated relative to the Bentham, or a reference where that method is described.How does this "exercise" provide information on the reliability of the angular correction?Answer #7: A description of how the radiometers were calibrated relative to the Bentham is already reported in Sect.2.3.1, which has been expanded as follows: In such a way, spectral and angular corrections could be easily introduced.More precisely, data from radiometers 01-03 were processed with a matrix calculated by the owner agency: the spectral and angular characterization of those radiometers was performed by the PMOD-WRC, then the spectral and angular corrections were calculated by ARPA Valle d'Aosta using the libRadtran model.Finally, an absolute C1180 calibration factor was determined some weeks before the comparison campaign with reference to the Bentham spectroradiometer, following the procedure described by Gröbner (2007).
Reference to Sect.2.3.1 was added in 5.1.Also, the next paragraph has been included in Sect.5.1 follwing the suggestion of the referee: Since the IQRs of those radiometers are very low (ranging from 1.4 to 3%), this exercise also provides information on the radiometers stability and the reliability of the angular corrections.
Comment #8: Page 8, line 24.Was that statement verified by measuring the angular response of these radiometers?As far as I know this information is not supplied by the manufacturers.
Answer #8: Text in Sect.5.2 has been rewritten in the following way: Figures 5 and 6 clearly show that the series of measurements of every instrument have a different IQR during clear sky days.Even though the angular response of the radiometers was not measured during the campaign, most of the total variability for clear sky conditions, as we will see later, is clearly ascribable to the daily (not day-to-day) variability... Comment #9: page 9, line 16.What does mean "the full range of ratios increases" ?Could you try to rewrite this sentence?

C1181
Answer #9: The sentence has been rewritten in the following way: The scatter of the ratios for each instrument and, in particular, the difference between the maximum and minimum values (i.e. the distance between the whiskers in the boxplots), increase...

Comment #10:
The acronyms used in this manuscript are not consistently used throughout the manuscript and should be spelled out at first use: Examples are : ARPA, IBIMET, APPA, PMOD (identical to PMOD-WRC?), QASUME.Answer #10: A table containing the acronyms (see Table 1, Supplement) of all participating agencies has been added to the manuscript.The latter was completely revised for a consistent use of the agency names.Answer #12: The table has been modified according to the suggestion of the referee (see Table 2, Supplement).

C1183
Comment #13: tables 2 and 3 should state clearly the use of expanded or simple uncertainties (coverage interval etc...).
Answer #13: The tables were updated with the coverage factors.
Comment #14: Was the linear drift of the responsivity of the reference spectroradiometer (see Figure 1) of 1% taken into account in that uncertainty estimate?
Answer #14: The text in Sect.2.1 has been expanded following the suggestion of the referee: Figure 1 presents the variations of the spectral responsivity, which are not strictly a drift (the responsivity measured during the first calibration, used as a reference for the following, is lower than the second and similar to the third).The changes are to ascribe to the instrumental instability (0.4% uncertainty, calculated assuming a rectangular probability distribution) and to the heating of the diffuser during the calibration (1% uncertainty).The combined uncertainty originating from the two factors is higher than the observed variability.In order to reduce the errors ... The radiative transfer model is used not only for the analysis, but also as an independent radiometer.Thus the use of particular parameters is worthwhile to be discussed.Indeed, can the authors comment on the following aspects.

Comment #15:
Answer #16: The model description in Sect.2.5 has been expanded following the suggestion of the referee: A radiative transfer model, the libRadtran package (Mayer and Kylling, 2005), was used in the campaign for comparing different kinds of data as explained in Sect. 4 and as a further quality control.Table 4 summarizes the data set entered as input to the model.
The solar spectrum was set to the recommended value following the model documentation (Atlas-3, shifted to air wavelengths).Default summer atmospheric profiles were used.Pseudo-spherical discrete-ordinate method (DISORT) with double precision was chosen as the solver, since a simple plane-parallel DISORT solver showed relevant deviations from the reference instrument even for low zenith angles.The effective ground albedo was set to 3% (Degünther et al., 1998).Rural aerosol properties, background stratospheric aerosols and the default Shettle aerosol profile were given as inputs to the model.Since independent measurements of the aerosol single scattering albedo (SSA) were not available during the campaign, the SSA value was chosen in order to best reproduce several spectral measurements recorded with Bentham 5541 during clear-sky days in summers 2008 to 2010 at Saint-Christophe (wintertime measurements were not considered because of changes of effective ground albedo due to the snow) as explained by Ialongo et al. (2010).Therefore, the single scattering albedo (SSA) was reduced by 10% relative to the default model value (i.e.0.90 to 0.95 depending on the wavelength).Similarly, the Molina&Molina ozone cross sections were Incidentally, the cosine-weighted fraction of sky above the horizon in Saint-Christophe C1188 is similar to that obtained at the Davos station during COST726 campaign.For that comparison, the authors concluded that "The largest influence [of the horizon on the cosine correction of radiometers] is of the order of 0.8% for a total overcast sky and even less for a clear sky where this error would be reduced by the direct to diffuse radiation ratio".
The matrices relative to the instruments belonging to ARPA Valle d'Aosta have been calculated taking into account the real horizon.
Comment #21: Figure 3: I would not call the periods before sunrise and after sunset as missing data.If really necessary, a possible quantification could be the total number of possible measurements versus the actual measurements).
Answer #21: The caption has been modified as follows: Data for rainy periods, before sunrise, after sunset, during calibration of the reference and dome cleaning were not included in the graph.
The comparison has not been performed during these periods, since most of the matrices were calculated for clear-sky and visible sun.

Comment # 5 :C1178Answer # 5 :
line 1, page 7 I do not understand this sentence (How can the integrated clear irradiance I0BB (please define it in the text) NOT change during a spectral scan?What is meant by appreciably?Please quantify?The text has been modified as follows:

Comment # 11 :
The list of references are extensive (maybe too much for the type of manuscript).I think some redundant references could be omitted if the original citation is kept.Reference on page 11, line 30 seems incomplete.Is there a web-link?Answer #11: The following redundant references were omitted: • Lucas et al. 2010 (p.2791) • McKenzie 2003 (p.2791) • Bernard et al. 1998 (p.2792) • Webb et al. 1998 (p.2792) C1182 • Webb et al. 2003 (p.2792) • Gröbner et al. 2002 (p.2792) • Gröbner et al. 2007 (replaced by Hülsen and Gröbner 2007) • Bais et al. 2001 (p.2792) • Lantz et al. 2002 (p.2792) • Ialongo et al. 2008 (p.2792) • Gröbner and Blumthaler 2007 (p.2793) • Kerr 2010 (p.2794) Reference on line 13, p. 2810 has been rewritten as BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML 2008 Evaluation of Measurement Data-Guide to the Expression of Uncertainty in Measurement GUM 1995 with minor corrections.Joint Committee for Guides in Metrology, JCGM 100 http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdfComment #12: I am uncomfortable with the content of the reference scale column, as I wonder how a specific instrument can represent a reference scale.I would recommend stating the Institute or laboratory to which the measurements are traceable to and modify the title from "reference scale" to "Traceability".

Figures 4
Figures 4 and 5 in the Supplement show the (spectral) ratios between the model and the reference, using Bass&Paur and Molina&Molina cross-sections, respectively.The calculations take into account the different Rayleigh scattering coefficients used in the Brewer algorithm and the model, which otherwise originate a fictious difference of about 3 Dobson Units (J.Gröbner, personal communication).As can be noticed in the figures, major differences arise at wavelengths lower than the range spanned by the Brewer (blue lines in the figure).

Table 3 : Intuitively, I would expect the wavelength uncertainty to increase with increasing SZA. This does not seem to be the case for the range 310-400 (last line of the table). Can the authors confirm these values? Answer #15:
It is a typo (as one might see from the estimate of the total uncertainty, which is correctly calculated).The wavelength uncertainty in the third column should be 0.9% (according to Gröbner 2005).This table has been corrected.