This study focuses on the assessment of surface solar radiation (SSR) based
on operational neural network (NN) and multi-regression function (MRF)
modelling techniques that produce instantaneous (in less than 1 min)
outputs. Using real-time cloud and aerosol optical properties inputs from the
Spinning Enhanced Visible and Infrared Imager (SEVIRI) on board the Meteosat
Second Generation (MSG) satellite and the Copernicus Atmosphere Monitoring
Service (CAMS), respectively, these models are capable of calculating SSR in
high resolution (1 nm, 0.05
Solar energy exploitation is a cornerstone for sustainable development, through efficient energy planning, towards the goal of gradual independence from fossil fuels. In this direction, the European Union (EU), the Middle East and North Africa (MENA) and numerous neighbouring regions and countries have laid out specific technology roadmaps aiming at the integration of low carbon energy technologies linked with the deployment of photovoltaic (PV) installations in the energy market (IPCC, 2012; NREL, 2016; IRENA, 2016; Jager-Waldau, 2016; REN21, 2017; UN, 2017). In addition, the United Nations (2017) has set as its main sustainable development goal by 2030 to ensure universal access to affordable, reliable, and modern energy services. The International Energy Agency (2007) has estimated that the global primary energy demand will increase by 40–50 % from 2003 to 2030. Since energy production, transportation and consumption put considerable pressure on the environment, there is serious concern regarding the sustainability of energy consumption.
Earth observation (EO)-based systems and relevant services already play an important role in the solar energy industry, as well as in human-health-related emerging technologies, but there is still significant potential in increasing their efficiency and exploitation (Schroedter-Homscheidt et al., 2006; Wald et al., 2011; Lefevre et al., 2014). EO from space is already triggering services and applications that can deliver benefits throughout all the phases of energy production and supply. Their contribution ranges from identifying reservoirs and locations with solar energy potential to controlling and monitoring of the distribution networks across Europe, Africa and the Middle East, while providing support to energy policy formulation and enforcement (EU, 2011; IEA, 2010).
The need for improved EO-based surface solar irradiance assessment is increasing as more solar farms are included in national electricity grids worldwide (EC, 2013). Solar-energy-related installations have been increasing their share on the total energy demand as defined by the distribution and transmission system operators (DSOs and TSOs, respectively). As a result, accurate, real-time and short-term forecasting estimations of surface solar radiation (SSR) and, more specifically, global horizontal irradiance (GHI) related to the operation principles of PV installations are vital. The real-time GHI estimations are required at local and regional scales, as well as high temporal frequency (every 5–15 min), in order to be used for near-real-time decisions, linked with the PV-related contribution to the electricity grid.
Since the launch of EO satellites, such as Meteosat Second Generation (MSG), and Sentinel satellite series, real-time image processing techniques have been developed (Suárez and Nesmachnow, 2012). The main advantage of these techniques is the possibility to monitor numerous meteorological variables in almost real time (Derrien et al., 2005; MeteoFrance, 2013). A comprehensive intercomparison of radiation products, codes, algorithms, models and independent databases has been performed by many researchers (Oreopoulos and Mlawer, 2010; Oreopoulos et al., 2012; Ellingson et al., 1991; Ineichen, 2006; Beyer et al., 2009; Cahalan, et al., 2005). Solid steps in estimating the surface GHI were taken by Deneke and Feijt (2008), Schulz et al. (2009), Mueller et al. (2009), Huang et al. (2011) and Qu et al. (2017), who developed GHI retrieval methodologies based on the use of discrete pre-calculated look-up tables (LUTs), while Dorvlo et al. (2002), Zarzalejo et al. (2005), Lopez et al. (2001) and Takenaka et al. (2011) developed solutions based on neural network (NN) models. The validation of most of the above mentioned methodologies was performed against radiative transfer model (RTM) simulations and ground-based measurements, from various networks around the globe. However, from the validation results it was shown that accuracy was inversely proportional to calculation speed under all-sky and terrain conditions. The magnitude of the GHI uncertainty due to the effect of aerosols and clouds is significant and has motivated numerous related studies (Federico et al., 2017; Kosmopoulos et al., 2015; Lara-Fanego et al., 2012; Tegen et al., 1996; Lindfors et al., 2013). Under high aerosol loads the SSR can be reduced by 20–50 % (Eck et al., 1998; Gleeson et al., 2016; Kosmopoulos et al., 2017), while under cloudy conditions the impact was up to 60–90 % for overcast conditions and cloud coverage of 8 octas (Aebi, et al., 2017; Kosmopoulos et al., 2015; Zygmuntowska et al., 2012), highlighting the significant effect of these atmospheric parameters (clouds and aerosols) on the GHI calculations and in the performance of PV installations and energy production.
In the present study, we report on (i) the assessment of the surface solar irradiance calculated in real time, which is defined as the product with a time delay of 1 min or less from an actual atmospheric situation, by developing and using two NN-based techniques and a multi-regression-function-based technique and (ii) the validation of these techniques against ground-based measurements from the Baseline Surface Radiation Network (BSRN). Section 2 presents data, methods and techniques used. Section 3 describes the validation results including a sensitivity analysis of related atmospheric parameters and in Sect. 4 we present our conclusions on the proposed techniques.
Coordinates (degrees) and height (metres above sea level) of the BSRN stations used for the validation.
Flowchart of the modelling technique scheme. The initial pixel classification followed by the clear- or cloudy-sky inputs to the real-time solver result the spectral (NNS) and integrated (MRF and NN) SSR-related outputs.
The verification of the applied SSR real-time modelling techniques was performed against ground-based measurements from nine stations (Table 1) of the Baseline Surface Radiation Network (BSRN; Hegner et al., 1998) equipped with Kipp and Zonen pyranometers (GHI measurements) and a precision filter radiometer (PFR) at Izaña, Spain. BSRN consists of high-quality ground-based measurements of SSR and for the purposes of the comparison we used the dataset from July 2014 to June 2015. Table 1 presents the location and description of the nine BSRN stations used for the validation of the SSR estimations calculated with the modelling techniques. The temporal resolution of the ground-based measurements is 1 min, so in order to match the 15 min resolution of the MSG cloud data (and hence the SSR outputs) we used 15 min averages of all the BSRN and PFR measurements used. The selected BSRN stations represent a variety of different climates, altitudes and aerosol sources in the field of view of MSG and thus provide an opportunity to study the models' performance under various atmospheric conditions.
The most important inputs to our real-time modelling techniques were the
satellite cloud data products from the Spinning Enhanced Visible and
Infrared Imager (SEVIRI) on board the Meteosat Second Generation (MSG)
satellite. We obtained the cloud type (CT), the cloud phase (CP) and the
cloud optical thickness (COT) products so as to efficiently quantify the
effect of clouds on SSR. COT depends on the moisture density as well as the
vertical thickness of the cloud. The cloud reflectance at channel at 0.6
For the real-time assessment of the SSR we additionally incorporated the aerosol 1-day forecast data from the Copernicus
Atmospheric Monitoring Service (CAMS) as the
basic input parameter. These forecasts are based on the
Monitoring Atmospheric Composition and Climate (MACC) reanalysis tools, and
include validated modelling of aerosol and satellite data assimilation
(Eskes et al., 2015). They are able to provide operationally accurate data
of aerosol optical depth (AOD) at 550 nm, at 1 h time steps and 0.4
In this section we present the SSR real-time modelling techniques, the
methodology used for developing operational products and the validation
statistics against ground-based measurements. The techniques are the
multi-regression function (MRF), the neural network that produces spectral
irradiances (NNS) and which is presented in detail in Taylor et al. (2015),
and a variant version of the NN that produces integrated irradiances. All
three techniques have been optimized based on LUTs that are described in the
Sect. 2.2.1 and produce instantaneous (with less than 1 min delay
from the time that the MSG image is produced) SSR. The number of outputs
depends on the region under study and can be of the order of 10
An example of the output maps based on the real-time SSR techniques. Here is the GHI for 15 April 2015 at 12:00 UTC together with the BSRN station locations.
The performance of the real-time techniques was evaluated by comparing the
GHI outputs with (i) the initial RTM simulation LUTs and (ii) the BSRN
ground-based measurements and with respect to the aerosol and cloud effects.
The evaluation was based on the bias and mean bias error (MBE), the root
mean square error (RMSE) and their relative components (rMBE and rRMSE,
respectively):
RTM-simulated GHI at 15 min time intervals as compared to
the BSRN ground-based measurements in terms of correlation coefficient (
Percentage difference (%) of the real-time modelling techniques as compared to the RTM simulations for all ground stations. The box charts highlight the more precise estimation approach of the MRF technique as compared to the NN-based techniques.
The altitude correction of GHI for various SZAs as a
function of the SSR ratio (SSR at height
All modelling techniques presented in this paper for the real-time
assessment of the SSR are based on LUTs, calculated with the radiative
transfer model (RTM) libRadtran (Mayer and Kylling, 2005; Emde et al.,
2016). These LUTs are described in detail in Taylor et al. (2015) and
consist of more than 2.5 million RTM simulations with atmospheric inputs and
1 nm spectral resolution GHI outputs. The interoperable exchange of similar
GHI databases is studied by Ménard et al. (2015), highlighting the
usefulness and necessity of such LUT-based approaches (Lefevre et al.,
2014). Under clear-sky conditions the simulated by libRadtran input
parameters were the solar zenith angle (SZA), the AOD, the Ångström
exponent (AE), the single-scattering albedo (SSA), TOC and the columnar
water vapour (WV), while under cloudy conditions except from SZA and TOC, we
also used the optical thicknesses of water and ice clouds (WCOT and ICOT,
respectively) as inputs. The AOD is not used for cloudy conditions when
COT > 1, as the effects of aerosols are much weaker compared to
thick clouds. For the model versus BSRN station comparison, in order to take
into account the station altitude, an altitude correction on the solar
energy output of the different model simulations has been applied based on
RTM (Libradtran) calculations. The outputs are high-resolution spectral
irradiances (1 nm) covering the wavelength region between 285 and 2700 nm.
In brief, we used the SDISORT radiative transfer solver (Dahlback and
Stamnes, 1991) with pseudospherical approximation to produce valid outputs
from 0 to 90
The mean GHI in W m
Mean GHI differences in W m
The multi-regression function (MRF) technique was developed as an analytical
methodology using the RTM outputs, with the aim to provide results as close
as possible to the initial (training set) RTM outputs. The advantage in the
use of these functions is that they can be executed very rapidly and can be
used for real-time SSR determination. In order to achieve that, analytical
functions for the SSR should be constructed. In general, SSR is a function of SZA, COT, AOD, AE, SSA, WV and TOC (Appendix A presents the complete list of nomenclature and abbreviations). For the AE and SSA we used monthly
climatological values in order to bridge the gap between the operational
input availability and the SSR accuracy. However, a preliminary
investigation has been performed for the sensitivity of GHI to WV column and
TOC. We compared integrated spectral GHI over the entire spectrum for
different TOC values and we found a mean difference of only 0.5 % for TOC
ranging between 300 and 400 DU. For WV columns ranging between 0.5 and 2 cm
we found a mean difference of 3.2 %, although for SZA < 15
Values of parameters used for the polynomial function Eq. (3) of the MRF technique for GHI calculations under clear-sky and cloudy-sky conditions.
Then, we constructed different polynomial functions according to Gasca and
Sauer (2000) for cloudy and clear-sky conditions, to be applied into the
scheme presented in Fig. 1. For cloudy cases the irradiance is expressed as
As presented in Taylor et al. (2015), the LUT approach, despite its large
size, still provides estimates at discrete input values. The interpolation
techniques to correct the input-output parameter intervals are
computationally more costly than a continuous function-approximating model,
or a NN model, which is more preferable for producing real-time outputs
(Hornik et al., 1989). Indicatively, using a test set of 1000 RTM
simulations from the developed LUT, we applied an interpolating function to
adjacent/nearest value and found that each interpolation calculation
required a time in excess (in total
In the first NN set, we produced instantaneous SSR spectra of the order of 1
million in less than 1 min, using as operational inputs the CAMS AOD
1-day forecasts, the MSG COT and real-time calculations of SZA. The output
resolution is high in terms of spectral (1 nm), spatial (0.05
Since the proposed modelling techniques (MRF, NN and NNS) operate in
real time, the potential applicability for short-term forecasting purposes
for the next few hours is feasible. In this direction, the CAMS AOD is
already an operational forecast input (Benedetti et al., 2009), with accurate
predictions every 1 h even under high aerosol load conditions
(Kosmopoulos et al., 2017). On the other hand, the MSG COT short-term
forecasting requires the employment of a cloud motion vector analysis (e.g.
Hammer et al., 1999) in high spatial and temporal resolution (5 km
This section initially summarizes the performance of all the real-time
modelling techniques against the RTM simulations for all BSRN stations.
Figure 3 presents the percentage difference between the RTM simulations and
the MRF, NN and NNS techniques. All data presented here are GHI model
outputs with a 15 min temporal resolution. The box plots represent the
interquartile range between the 25th and 75th percentiles with the in-box line
to show the median and the upper and lower whiskers to represent the maximum
and minimum error values that are within 1.5
Seasonal relative RMSE values of the GHI estimations produced by the real-time techniques as compared to the BSRN measurements.
The NN and NNS approaches showed a systematic underestimation for the NN of
The model accuracy was verified against nine BSRN stations. We calculated the regression of the mean GHI between the ground measurements and the model outputs, shown in Fig. 5. We also show the intra-model regression compared to the initial RTM simulations (Fig. 5a) in order to assess the NN and NNS included interpolations of the LUT outputs and the MRF performance. We found that the MRF technique presents identical values to the RTM, for all ground stations and under all climatological conditions. The NN and NNS show quite good agreement too in terms of absolute values, as under all conditions mean GHIs are less than 5 % different from the BSRN measurements. In Fig. 5b we confirmed the similarity of MRF with RTM and in some cases with the NN models, indicating the overall efficiency of all interpolation and multi-function techniques used. A slightly better performance was observed for higher mean GHIs, proving the usefulness under high solar energy potential conditions.
GHI evaluation results as a function of season and real-time techniques for all stations. The model MBE and RMSE statistical
scores are shown in absolute units (W m
Scatterplots of real-time (MRF and NN) and RTM-simulated
GHI in W m
Mean percentage difference (blue) and standard deviation (red) of the 15 min GHI produced from the MRF technique as compared to ground-based measurements from all stations as a function of the COT.
Figure 6 shows the accuracy of MRF, being the most reliable technique as
presented in Fig. 3, with respect to the ground-based measurements for
various temporal integrations, starting from the (actual derived) 15 min to
hourly, daily and monthly averages. The uncertainty range of the MRF
simulations given as mean interquartile GHI differences is highest (from
The cloud effect via the radiative transfer of solar radiation in the
atmosphere represents the greatest source of uncertainty in the simulation
of SSR, while several models do not have the capability to deal with clouds
coexisting with a radiatively active atmosphere (Cahalan et al., 2005).
Small changes in cloudiness and its optical properties can impact on GHI.
The magnitude of the cloud effects on the model to BSRN comparison can be
seen in Fig. 8. Under clear-sky conditions (Fig. 8a), the
regression of the 15 min modelled GHI values, in terms of coefficient of
determination (
Figure 9 illustrates the mean percentage difference and standard deviation
of the 15 min GHI produced by the MRF and the measured values by the BSRN
stations (only instances with cloudy conditions were used for all stations)
as a function of COT. For COT < 2, the MRF technique results in higher
GHI values than those actually measured, 1–12 %, while as the COT
values increase the MRF underestimates the measurements by up to
In addition to the clouds, aerosols play an important role in the solar
radiation transfer in the atmosphere. Especially in places with high solar
energy potential, where cloud-free conditions prevail during the greater
part of the year, significant aerosol sources could exist (Gkikas et al.,
2012). The aerosols effect is closely related to the aerosol optical
properties and mostly AOD, and as a consequence the uncertainty in the model
AOD input could result in significant errors in the assessment of SSR (Oumbe
et al., 2015; Kosmopoulos et al., 2017). For the purposes of this study we
used the Global Atmosphere Watch (GAW-PFR) station of IZA, which is an
internationally recognized test bed for aerosol remote sensing instruments
(Cuevas et al., 2016), to quantify the AOD difference between the
operational input from CAMS and a PFR instrument, under high-altitude
conditions (Garcia et al., 2013). In Fig. 10 we present the yearly frequency
distribution of the differences between CAMS and PFR values for cloudless
sky conditions. The majority of the AOD differences are lower than 0.2, with
the maximum frequency encountered at zero AOD differences, indicating the
overall good accuracy of CAMS-derived 1-day forecasts of AOD. The mean
absolute difference was found equal to 0.1075
Frequency histogram of differences between the CAMS and the PFR AOD at the Izaña station together with the mean absolute difference and standard deviation metrics.
Absolute differences in GHI (in W m
This study proposed state-of-the-art modelling techniques (NNS, NN, MRF) for the real-time estimation of SSR, which have been validated against ground-based BSRN measurements. The determination and understanding of the input parameter effects on radiative transfer revealed that the accuracy of simulations depends on the quality and resolution of the atmospheric inputs to the models (mostly COT and AOD), while increasing the calculation speed and including spectral GHI information decreases the model accuracy.
We firstly described the developed modelling techniques, which are based on
large LUTs for clear-sky and cloudy conditions. Verification of these models
was performed for the GHI against ground measurements at nine stations, with
variable geographical, atmospheric and altitudinal conditions. The
comparison showed a dependence on seasonal variability, with summer rRMSE
values below 30 % for all models and under all conditions, and revealed
largest errors for the NNS technique because of the spectral special
characteristics, as well as for LER and TOR stations. The NN presented a
slight underestimation of 8 % against its training RTM simulations, while against BSRN stations it achieved MBE and RMSE values less than 30 and 80 W m
The results presented here show the potential use of such techniques for
solar-energy-related applications and electricity grid support services
(IRENA, 2015). Comparison of the proposed real-time models with existing
databases (e.g. SARAH), which in most cases are post-processed data using
past data series, showed similar results. Finally, we tested the impact of
cloud and aerosol inputs to the models in order to reveal the AOD forecast
accuracy of CAMS, which turned out to be
All data sets used and produced for the purposes of this paper are freely available and can be requested from the corresponding author. The model codes developed (NN, NNS and MRF) can be used for various applications after consultation with the corresponding author.
The authors declare that they have no conflict of interest.
This article is part of the special issue “SKYNET – the international network for aerosol, clouds, and solar radiation studies and their applications”. It is not associated with a conference.
This research was partly funded by the H2020 GEO-CRADLE project under grant agreement no. 690133, the IERSD/NOA's action THESPIA with grant number PDE2013SE01380031 under the call KRIPIS, and the project Aristotelis-SOLAR. Edited by: M. Campanelli Reviewed by: three anonymous referees