Review for ‚Aerosol effects on day-ahead solar radiation forecasting’

General comments

Overall, this is a very nice study, looking into the day ahead forecast accuracy of AOD and DNI for solar energy production in cloud-free conditions. It is re-doing the study of Schroedter-Homscheidt et al. (2013 and 2016), but now for the recent CAMS forecast generation instead of the previously used AERONET-only assessment for cloud-free conditions (2013 paper) and for the precursor dataset from MACC but for all-sky conditions (2016 paper, DOI 10.1127/metz/2016/0676).

Furthermore, additional statistics are provided, but on the other hand the intra-day variability is assumed to be neglectable and only daily means of AOD or daily sums of DNI are compared. This is counter-intuitive as power markets require hourly resolved forecasts or even 15 min resolved forecasts. It may be very well justified by AOD variability, but requires clearer justification to serve reader/users requirements in the solar energy sector.

Furthermore, power markets require day ahead forecasts to be provided in the late morning (e.g. 10 or 11 am). Therefore, a 1-day persistence as evaluated here (FC method 1) is not of relevance for the solar energy sector in day-ahead-markets, it only may be of relevance for intra-day-markets for the afternoon hours (where there is the smallest forecast value). Only the AERONET observations from yesterday can be used in the persistence approach for the day-ahead due to the timing constraints of energy markets. The typical persistence approach as used in meteorology is not applicable in the solar energy community. To make the study relevant for the solar energy sector in day-ahead markets, this ‘2-day-persistence’ approach needs to be added or the current persistence approach results need to be replaced by the ‘2-day-persistence’.

Especially, the 2^nd point requires a major revision in the sense of re-running the results.

Furthermore, I would like to suggest to think about a more specific title for the paper. The current title is very general and a bit misleading. The paper is more oriented on assessing the forecast accuracy of CAMS AOD forecasts versus AERONET-based persistence and climatology as naïve forecasts.

Comment to the editor: The scope of ACP may fit a bit better than AMT, but I understand that this is a manuscript for a joint special issue of ACP and AMT. 

Specific comments

Abstract, line 11/12: Please specify the term ‘5% deviation’ better already at the beginning. Do you mean in daily means of AOD/daily sums of DNI or in hourly day ahead DNI forecasts? This should be clear already when reading only the abstract.



Also, after reading the introduction, it is still not defined what temporal resolution you look at in the study. This only gets clear in the method section, but also this section starts with an un-justified assumption of no intra-day variability. Is there any knowledge available in the literature to justify this assumption via a citation?

Later, a sensitivity analysis of this assumption is done for the only site Beijing. I wonder if this is a good choice. Intra-day variability of AOD can be expected e.g. in areas with dust fronts in deserts. I’m not sure if Beijing with its urban AOD sources is a very good place to expect intra-day variability. It may be wise to repeat this assessment for more stations with more event-like meteorological phenomena?

Furthermore, in table 4 it is not clear if these are statistics in hourly temporal resolution or again in daily resolution but with hourly input (which would be not relevant for the reader). For users it is relevant if the hourly DNI is predicted well, daily DNI is of no interest in the forecasting of solar energy as there are (at least to my knowledge) no energy markets working on daily sums of tomorrow. Therefore, it would be needed to prove that your approach of looking into daily data is of value for the solar energy community with their hourly DNI forecast requirement. I’m not saying that your assumption is wrong, only it is not clearly discussed and justified to the users.

Also, I would spend an extra sub-chapter with a clear and easy-to-find heading on the question if daily assessment is sufficient. Knowing that power markets need at least hourly resolved radiation day ahead forecasts, the gap between your results and the user community needs to be filled with a good and (for the reader) easy to find explanation.  Currently this part is very much split and therefore hidden separate statement in various chapters.

Introduction:

Line 14 onwards: Authors may also want to discuss the relevance of DNI for any tracked system, e.g. tracked PV or the correct diffuse/direct split needed for Agri-PV with their various tilted surface options.

The paper correctly discusses two impacts of aerosols on DNI, the extinction and the soiling of surfaces. I’d appreciate if the references in line 20 onwards would be better sorted in references on the extinction and references of the soiling effect. It is ok to name the soiling issue, but as the paper does not deal with this further, the two groups of references should not be mixed up as done at the moment.

It may be worth to ensure that the reader understands that all DNI comparisons assume cloud-free conditions during the whole day. This study design is ok for an assessment of aerosols, but solar energy community readers may not be aware of this constraint of the study. This should be mentioned very clearly in abstract and introduction, that cloudy cases are not treated in the assessment. Therefore, all results are not to misinterpreted as ‘all-sky forecast accuracy’ which users will search for.   

It may be very helpful for the reader to understand in the introduction, that this study is revisiting work by Schroedter-Homscheidt et al (2013 and 2016) with a similar approach and metrics as suggested by them, but for more recent CAMS AOD forecast cycles. There is no doubt, that an assessment of more recent CAMS AOD forecasts after more than 10 years is a very valuable scientific contribution, and it is welcome to see that the authors are adding further detailed assessment at some aspects/locations. Nevertheless, it may be wise to cite both papers for the reader’s orientation.

Data section: CAMS forecasts and CAMS reanalysis are run with different software versions. Strictly speaking, citing a reanalysis assessment is not a proper reference without further description of the differences of both products. Regular CAMS forecast assessment reports may be a good alternative.

Method section:

As discussed above: Please add a ‘2-day persistence’ for the day-ahead application case or replace the ‘1-day persistence’ (which is only valid for the intra-day application case) by the ‘2-day persistence’ to fulfill your goal of assessing day-ahead forecast capability.  All assessments need to be re-run to either show the 2-day persistence (as the one needed primarily by the solar community) or to show both persistence approaches.

Results:

Table 3 is strange. As a reader I want to know the performance quantitatively. I want to decide if differences are significant. Why do you only give the method name which performs best? Why is this table not quantitative?

Are the quantitative results given later in fig. 5 and 6? If yes, then table 3 is superfluous perhaps? You may want to order the stations in Fig 5 to 6 in the groups you introduce in table 3 and then delete table 3?

Talking about grouping of stations, you already introduced a grouping in Fig 2 and 3 according to geographical area. Perhaps you want to use only the grouping introduced in table 3 in the whole paper as it is the aerosol type related grouping (which has more physical meaning than a pure geographical area)?

What is the value of Fig. 7? Isn’t this just the expected behavior? Please tell the message of this plot better, or omit it?

Also fig. 8 looks rather as expected. In case of no variation from day to day, the persistence will do the job. What is the ‘news’ in this plot? Why is it worth to show this plot? Can you elaborate on that?

Technical corrections

Abstract: Sentence ‘AERONET persistence…’, line 10 seems to be incomplete.

Line 66 & table 2: ‘what is meant by are available from 2015’. CAMS ADS catalogue entry states: “EAC4 is only available from 2003 onwards.” Why do you state ‘since 2015’?

Fig 5 and 6: This is a daily mean AOD forecast? Perhaps add this?

Fig 8: what is relative deviation of DNI forecasts? Which unit? How is it normalized (mean of all values, mean of all daytime values, if the latter - how is daytime defined)?  

Fig 9/10/11: better clarify that DNI is daily sum of DNI deviation due to daily mean AOD variation ?

Review of the manuscript “Aerosol effects on day-ahead solar radiation forecasting”

This work compares the use of aerosol data from AERONET and CAMS for the day-ahead forecast of direct normal irradiance (DNI). Three forecast methods of AOD are used together with a radiative transfer model: a monthly mean and the next day persistence method based on AERONET surface data and the day-ahead forecasts based on CAMS products. The paper is well organized and the topic is worthy of investigation. However, there are some aspects that should be additionally considered/clarified.

1. The work compared the use of AERONET and CAMS data for the day-ahead forecast of DNI, taking as a reference the AERONET data for that day. The use of other ground-based measurements of DNI, for example, from reference stations collocated to AERONET sites, is not mentioned (neither in the abstract nor in the data section), and this should become clearer. This must be clarified in the abstract and in the introduction and possible deviations in relation to reference DNI measurements must be also mentioned. Furthermore, the abstract starts by referring that the spatiotemporal variations of AOD at a global level are examined, and only later does it become clear that one of main objectives is to compare different data/method combos for day-ahead forecasts of DNI.

2. The assumption that AOD is invariant during the day must be further discussed and justified and that it can be calculated from three records only. To what extent is this assumption valid in view of the deviations observed in the persistence method under evaluation? Or, how this assumption can affect (positively or negatively) the performance of the forecast methods, namely the persistence method? Also, please clarify if any interpolation method is used to obtain the AOD forecasts from CAMS for the AERONET sites (L.81).

3. How exactly the daily values/simulations of DNI are determined (e.g. how many values are used), for example to produce the relative deviation graphs of DNI in Figure 7? How these values are representative of the true daily DNI values when only three AERONET records are available?

4. The sentence starting “Pre-calculated look-up tables …” (L.98-100) is not clear. Please clarify if a complete batch of simulations was carried out for generating this tables for the indicated ranges and then deviations were determined. If so, which interpolation method was used?

5. In Table 1., a uniform resolution for lat. and lon. values may be used.

6. The construction of the graphs shown in Fig. 4 should be better explained, namely which reference value is used for plotting the deviations (positive or negative) of the absolute day-to-day differences.

7. The order of the AERONET sites on the horizontal axis of Figure 5 appears to be the one that produces a decreasing order of the correlation coefficient for CAMS. This criterion was not followed for the order in Figure 6. In the latter case, and in the absence of a table with values, using the same order in the top and bottom graphs would help to compare the RMSE and MAE values ​​of the different sites.

8. The analysis at the beginning of section 4.3 can be further improved (see also comment 2.). The way how the day-to-day variation in the number of valid AERONET records affects the metrics of the forecast methods should be further addressed. The number of valid AERONET records does not depend on the variation of the AOD.

9. L15. “… in regions with high direct normal irradiance (DNI>200 W/m2) …” is not clear. Is this a filter to only consider records in which DNI under clear-sky conditions (already screened by AERONET) higher that 200 W/m2 are considered. Please clarify.

10. Other statistical indicators can be used in this analysis, which have been used more recently in similar works in this area, such as fractional bias, fractional gross error, global performance index and skill score.