Articles | Volume 16, issue 18
https://doi.org/10.5194/amt-16-4165-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/amt-16-4165-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Retrieval of surface solar irradiance from satellite imagery using machine learning: pitfalls and perspectives
Hadrien Verbois
CORRESPONDING AUTHOR
Mines Paris, Université PSL, Centre Observation Impacts Energie (O.I.E.), 06904 Sophia Antipolis, France
Yves-Marie Saint-Drenan
Mines Paris, Université PSL, Centre Observation Impacts Energie (O.I.E.), 06904 Sophia Antipolis, France
Vadim Becquet
Mines Paris, Université PSL, Centre Observation Impacts Energie (O.I.E.), 06904 Sophia Antipolis, France
Benoit Gschwind
Mines Paris, Université PSL, Centre Observation Impacts Energie (O.I.E.), 06904 Sophia Antipolis, France
Philippe Blanc
Mines Paris, Université PSL, Centre Observation Impacts Energie (O.I.E.), 06904 Sophia Antipolis, France
Related authors
No articles found.
Yun Chen, Dazhi Yang, Chunlin Huang, Hongrong Shi, Adam Jensen, Xiang'ao Xia, Yves-Marie Saint-Drenan, Christian Gueymard, Martin Mayer, and Yanbo Shen
EGUsphere, https://doi.org/10.5194/egusphere-2025-3414, https://doi.org/10.5194/egusphere-2025-3414, 2025
This preprint is open for discussion and under review for Atmospheric Measurement Techniques (AMT).
Short summary
Short summary
We tested two satellite-based irradiance datasets against both high- and low-accuracy ground-based measurements. The dataset is unique: it includes irradiance measurements from a new research-grade monitoring station in a rare climate, along with new satellite data from China’s Fengyun-4B geostationary satellite. Findings suggest that using low-accuracy measurements as a reference for validation can be risky.
Job I. Wiltink, Hartwig Deneke, Yves-Marie Saint-Drenan, Chiel C. van Heerwaarden, and Jan Fokke Meirink
Atmos. Meas. Tech., 17, 6003–6024, https://doi.org/10.5194/amt-17-6003-2024, https://doi.org/10.5194/amt-17-6003-2024, 2024
Short summary
Short summary
Meteosat Spinning Enhanced Visible and Infrared Imager (SEVIRI) global horizontal irradiance (GHI) retrievals are validated at standard and increased spatial resolution against a network of 99 pyranometers. GHI accuracy is strongly dependent on the cloud regime. Days with variable cloud conditions show significant accuracy improvements when retrieved at higher resolution. We highlight the benefits of dense network observations and a cloud-regime-resolved approach in validating GHI retrievals.
Benoît Tournadre, Benoît Gschwind, Yves-Marie Saint-Drenan, Xuemei Chen, Rodrigo Amaro E Silva, and Philippe Blanc
Atmos. Meas. Tech., 15, 3683–3704, https://doi.org/10.5194/amt-15-3683-2022, https://doi.org/10.5194/amt-15-3683-2022, 2022
Short summary
Short summary
Solar radiation received by the Earth's surface is valuable information for various fields like the photovoltaic industry or climate research. Pictures taken from satellites can be used to estimate the solar radiation from cloud reflectivity. Two issues for a good estimation are different instrumentations and orbits. We modify a widely used method that is today only used on geostationary satellites, so it can be applied on instruments on different orbits and with different sensitivities.
Cited articles
Ball, J. E., Anderson, D. T., and Chan, C. S.: Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community, J. Appl. Remote Sens., 11, 042609, https://doi.org/10.1117/1.jrs.11.042609, 2017. a
Blanc, P. and Wald, L.: The SG2 algorithm for a fast and accurate computation of the position of the Sun for multi-decadal time period, Sol. Energy, 86, 3072–3083, https://doi.org/10.1016/j.solener.2012.07.018, 2012. a, b
Blanc, P., Gschwind, B., Lefèvre, M., and Wald, L.: The HelioClim
project: Surface solar irradiance data for climate applications, Remote
Sens., 3, 343–361, https://doi.org/10.3390/rs3020343, 2011a. a
Blanc, P., Gschwind, B., Lefèvre, M., and Wald, L.: The HelioClim
project: Surface solar irradiance data for climate applications, Remote
Sens., 3, 343–361, https://doi.org/10.3390/rs3020343, 2011b. a
Blanc, P., Gschwind, B., Lefevre, M., and Wald, L.: Twelve monthly maps of
ground Albedo parameters derived from MODIS data sets, International
Geoscience and Remote Sensing Symposium (IGARSS), 13–18 July 2014, Quebec City, QC, Canada, IEEE, 3270–3272, https://doi.org/10.1109/IGARSS.2014.6947177, 2014. a, b
Blanc, P., Remund, J., and Vallance, L.: Short-term solar power forecasting based on satellite images, Elsevier Ltd, https://doi.org/10.1016/B978-0-08-100504-0.00006-8, 2017. a
Boukabara, S.-A., Krasnopolsky, V., Stewart, J. Q., Maddy, E. S., Shahroudi, N., and Hoffman, R. N.: Leveraging Modern Artificial Intelligence for Remote Sensing and NWP: Benefits and Challenges, B. Am. Meteorol. Soc., 100, ES473–ES491, https://doi.org/10.1175/BAMS-D-18-0324.1, 2019. a
Brenowitz, N. D. and Bretherton, C. S.: Prognostic Validation of a Neural Network Unified Physics Parameterization, Geophys. Res. Lett., 45,
6289–6298, https://doi.org/10.1029/2018GL078510, 2018. a
Brunier, L.: Basic measurements of radiation at station Carpentras (2018-10), Centre Radiometrique, PANGAEA [data set], https://doi.org/10.1594/PANGAEA.896713, 2018. a
Cano, D., Monget, J., Albuisson, M., Guillard, H., Regas, N., and Wald, L.: A method for the determination of the global solar radiation from
meteorological satellite data, Sol. Energy, 37, 31–39,
https://doi.org/10.1016/0038-092X(86)90104-0, 1986. a
Copernicus Atmospheric Monitoring Service: CAMS Radiation Service, Copernicus Atmospheric Monitoring Service, http://www.soda-pro.com/web-services/radiation/cams-radiation-service (last access: 12 September 2023), 2020. a
Eissa, Y., Korany, M., Aoun, Y., Boraiy, M., Abdel Wahab, M. M., Alfaro, S. C., Blanc, P., El-Metwally, M., Ghedira, H., Hungershoefer, K., and Wald, L.: Validation of the Surface Downwelling Solar Irradiance Estimates of the
HelioClim-3 Database in Egypt, Remote Sens., 7, 9269–9291,
https://doi.org/10.3390/rs70709269, 2015. a
EUMETSAT: High Rate SEVIRI Level 1.5 Image Data – MSG – 0 degree, EUMETSAT [data set], https://navigator.eumetsat.int/product/EO:EUM:DAT:MSG:HRSEVIRI (last access: 12 September 2023), 2009. a
EUMETSAT: MSG Level 1.5 Image Data Format Description, EUMETSAT, Tech. rep., https://www.eumetsat.int/media/45126 (last access: 5 September 2023), 2017. a
Forstinger, A., Wilbert, S., Jensen, A. R., Kraas, B., Fernández Peruchena, C., Gueymard, C. A., Ronzio, D., Yang, D., Collino, E., Polo Martinez, J., Ruiz-Arias, J. A., Hanrieder, N., Blanc, P., and Saint-Drenan, Y.-M.: Worldwide solar radiation benchmark of modelled surface irradiance, EMS Annual Meeting 2022, Bonn, Germany, 5–9 September 2022, EMS2022-547, https://doi.org/10.5194/ems2022-547, 2023. a, b
Giles, D. M., Sinyuk, A., Sorokin, M. G., Schafer, J. S., Smirnov, A., Slutsker, I., Eck, T. F., Holben, B. N., Lewis, J. R., Campbell, J. R., Welton, E. J., Korkin, S. V., and Lyapustin, A. I.: Advancements in the Aerosol Robotic Network (AERONET) Version 3 database – automated near-real-time quality control algorithm with improved cloud screening for Sun photometer aerosol optical depth (AOD) measurements, Atmos. Meas. Tech., 12, 169–209, https://doi.org/10.5194/amt-12-169-2019, 2019. a
Gschwind, B., Wald, L., Blanc, P., Lefèvre, M., Schroedter-Homscheidt,
M., and Arola, A.: Improving the McClear model estimating the downwelling
solar radiation at ground level in cloud-free conditions – McClear-v3,
Meteorol. Z., 28, 147–163, https://doi.org/10.1127/metz/2019/0946, 2019. a
Hao, D., Asrar, G. R., Zeng, Y., Zhu, Q., Wen, J., Xiao, Q., and Chen, M.:
Estimating hourly land surface downward shortwave and photosynthetically
active radiation from DSCOVR/EPIC observations, Remote Sens. Environ., 232, 111320, https://doi.org/10.1016/j.rse.2019.111320, 2019. a, b
Hao, D., Asrar, G. R., Zeng, Y., Zhu, Q., Wen, J., Xiao, Q., and Chen, M.: DSCOVR/EPIC-derived global hourly and daily downward shortwave and photosynthetically active radiation data at 0.1∘ × 0.1∘ resolution, Earth Syst. Sci. Data, 12, 2209–2221, https://doi.org/10.5194/essd-12-2209-2020, 2020. a
Hastie, T., Tibshirani, R., and Friedman, J.: The Elements of Statistical
LearningData Mining, Inference, and Prediction, Second Edition, Springer
Series in Statistics, Springer New York, New York, NY,
https://doi.org/10.1007/978-0-387-84858-7, 2009. a
Holben, B. N., Eck, T. F., Slutsker, I., Tanré, D., Buis, J. P., Setzer, A., Vermote, E., Reagan, J. A., Kaufman, Y. J., Nakajima, T., Lavenu, F., Jankowiak, I., and Smirnov, A.: AERONET – A federated instrument network and data archive for aerosol characterization, Remote Sens. Environ., 66, 1–16, https://doi.org/10.1016/S0034-4257(98)00031-5, 1998. a
Holmgren, W. F., Hansen, C. W., and Mikofski, M. A.: pvlib python: a python package for modeling solar energy systems, Journal of Open Source Software, 3, 884, https://doi.org/10.21105/joss.00884, 2018 (code available at: https://pvlib-python.readthedocs.io/en/stable/reference/generated/pvlib.iotools.get_cams.html, last access: 7 September 2023). a, b
Huang, G., Li, Z., Li, X., Liang, S., Yang, K., Wang, D., and Zhang, Y.:
Estimating surface solar irradiance from satellites: Past, present, and
future perspectives, Remote Sens. Environ., 233, 111371,
https://doi.org/10.1016/j.rse.2019.111371, 2019. a
Jiang, H., Lu, N., Qin, J., Tang, W., and Yao, L.: A deep learning algorithm to estimate hourly global solar radiation from geostationary satellite data, Renew. Sust. Energ. Rev., 114, 109327, https://doi.org/10.1016/j.rser.2019.109327, 2019. a
Lefèvre, M., Oumbe, A., Blanc, P., Espinar, B., Gschwind, B., Qu, Z., Wald, L., Schroedter-Homscheidt, M., Hoyer-Klick, C., Arola, A., Benedetti, A., Kaiser, J. W., and Morcrette, J.-J.: McClear: a new model estimating downwelling solar radiation at ground level in clear-sky conditions, Atmos. Meas. Tech., 6, 2403–2418, https://doi.org/10.5194/amt-6-2403-2013, 2013. a
Long, C. N. and Dutton, E. G.: BSRN Global Network recommended QC tests, V2.x, https://hdl.handle.net/10013/epic.38770 (last access: 12 September 2023), 2010. a
McGovern, A., Elmore, K. L., Gagne, D. J., Haupt, S. E., Karstens, C. D., Lagerquist, R., Smith, T., and Williams, J. K.: Using artificial intelligence to improve real-time decision-making for high-impact weather, B. Am. Meteorol. Soc., 98, 2073–2090, https://doi.org/10.1175/BAMS-D-16-0123.1, 2017. a
Müller, R. and Pfeifroth, U.: Remote sensing of solar surface radiation – a reflection of concepts, applications and input data based on experience with the effective cloud albedo, Atmos. Meas. Tech., 15, 1537–1561, https://doi.org/10.5194/amt-15-1537-2022, 2022. a
Ohmura, A., Dutton, E. G., Forgan, B., Frohlich, C., Gilgen, H., Hegner, H.,
Heimo, A., Konig-Langlo, G., Mcarthur, B., Miiller, G., Philipona, R.,
Pinker, R., Whitlock, C. H., Dehne, K., and Wild, M.: Baseline Surface Radiation Network (BSRN/WCRP): New Precision Radiometry for Climate Research, B. Am. Meteorol. Soc., 79, 2115–2136, http://www.jstor.org/stable/26214877 (last access: 7 September 2023), 1998. a
Polo, J., Wilbert, S., Ruiz-Arias, J. A., Meyer, R., Gueymard, C., Súri, M., Martín, L., Mieslinger, T., Blanc, P., Grant, I., Boland, J., Ineichen, P., Remund, J., Escobar, R., Troccoli, A., Sengupta, M., Nielsen, K. P., Renne, D., Geuder, N., and Cebecauer, T.: Preliminary survey on site-adaptation techniques for satellite-derived and reanalysis solar radiation datasets, Sol. Energy, 132, 25–37,
https://doi.org/10.1016/j.solener.2016.03.001, 2016. a
Polo, J., Fernández-Peruchena, C., Salamalikis, V., Mazorra-Aguiar, L., Turpin, M., Martín-Pomares, L., Kazantzidis, A., Blanc, P., and Remund,
J.: Benchmarking on improvement and site-adaptation techniques for modeled
solar radiation datasets, Sol. Energy, 201, 469–479,
https://doi.org/10.1016/j.solener.2020.03.040, 2020. a
Qu, Z., Oumbe, A., Blanc, P., Espinar, B., Gesell, G., Gschwind, B., Klüser, L., Lefèvre, M., Saboret, L., Schroedter-Homscheidt, M.,
and Wald, L.: Fast radiative transfer parameterisation for assessing the
surface solar irradiance: The Heliosat-4 method, Meteorol. Z., 26, 33–57, https://doi.org/10.1127/metz/2016/0781, 2017. a, b, c
Racah, E., Beckham, C., Maharaj, T., Kahou, S. E., Prabhat, and Pal, C.: ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events, Adv. Neur. In., 2017, 3403–3414, 2017. a
Ranalli, J. and Zech, M.: Generalizability of Neural Network-based
Identification of PV in Aerial Images, 50th IEEE PV Specialists Conference,
11–16 June 2023, San Juan, Puerto Rico, IEEE, https://scholarsphere.psu.edu/resources/b8ff0491-2820-4328-a399-c3f80d911cd7 (last access: 12 September 2023), 2023. a
Rasp, S., Pritchard, M. S., and Gentine, P.: Deep learning to represent
subgrid processes in climate models, P. Natl. Acad. Sci. USA, 115, 9684–9689, https://doi.org/10.1073/pnas.1810286115, 2018. a
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J.,
Carvalhais, N., and Prabhat: Deep learning and process understanding for
data-driven Earth system science, Nature, 566, 195–204,
https://doi.org/10.1038/s41586-019-0912-1, 2019. a
Reno, M. J. and Hansen, C. W.: Identification of periods of clear sky
irradiance in time series of GHI measurements, Renew. Energ., 90,
520–531, https://doi.org/10.1016/j.renene.2015.12.031, 2016. a
Rigollier, C. and Wald, L.: Towards operational mapping of solar radiation from Meteosat images, in: Proceedings, EARSeL Symposium 1998 “operational remote sensing for sustainable development”, 11–14 May 1998, Enschede, the Netherlands, edited by: Nieuwenhuis, J. A., Vaughan, R. A., and Molenaar, M., CRC Press, 385–391, ISBN 9058090299, 1998. a
Scheck, L., Frèrebeau, P., Buras-Schnell, R., and Mayer, B.: A fast
radiative transfer method for the simulation of visible satellite imagery,
J. Quant. Spectrosc. Ra., 175, 54–67, https://doi.org/10.1016/j.jqsrt.2016.02.008, 2016.
a
Schroedter-Homscheidt, M., Arola, A., Killius, N., Lefèvre, M., Saboret,
L., Wandji, W., Wald, L., and Wey, E.: The Copernicus atmosphere monitoring
service (CAMS) radiation service in a nutshell, Proc. SolarPACES16, 22nd SolarPACES Conference 2016, 11–14 October 2016, AbuDhabi, United Arab Emirates, Centre pour la communication scientifique directe, 11–14, https://minesparis-psl.hal.science/hal-01386187 (last access: 7 September 2023), 2016. a
Sengupta, M., Xie, Y., Lopez, A., Habte, A., Maclaurin, G., and Shelby, J.:
The National Solar Radiation Data Base (NSRDB), Renew. Sust. Energ. Rev., 89, 51–60, https://doi.org/10.1016/j.rser.2018.03.003, 2018. a
Spearman, C.: The Proof and Measurement of Association between Two Things,
Am. J. Psychol., 100, 441–471, https://doi.org/10.2307/1422689, 1987. a
Tieleman, T. and Hinton, G.: Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude, COURSERA: Neural networks for machine learning, 4, 26–31, 2012. a
Tournadre, B.: Heliosat-V: une méthode polyvalente d'estimation du rayonnement solaire par satellite, Génie des procédés, Université Paris sciences et lettres, Tech. rep., https://pastel.archives-ouvertes.fr/tel-03227271 (last access: 5 September 2023), 2020. a
Verbois, H., Blanc, P., Huva, R., Saint-Drenan, Y.-M. Y. M., Rusydi, A., and
Thiery, A.: Beyond quadratic error: Case-study of a multiple criteria
approach to the performance assessment of numerical forecasts of solar
irradiance in the tropics, Renew. Sust. Energ. Rev., 117,
109471, https://doi.org/10.1016/j.rser.2019.109471, 2020. a
Verbois, H., Saint-Drenan, Y. M., Thiery, A., and Blanc, P.: Statistical learning for NWP post-processing: A benchmark for solar irradiance forecasting, Sol. Energy, 238, 132–149, https://doi.org/10.1016/j.solener.2022.03.017, 2022. a, b
Verbois, H., Saint-Drenan, Y.-M., Libois, Q., Michel, Y., Cassas, M., Dubus, L., and Blanc, P.: Improvement of satellite-derived surface solar irradiance estimations using spatio-temporal extrapolation with statistical learning, Sol. Energy, 258, 175–193, https://doi.org/10.1016/j.solener.2023.04.037, 2023. a, b, c
Wang, R., Camilo, J., Collins, L. M., Bradbury, K., and Malof, J. M.: The poor generalization of deep convolutional networks to aerial imagery from new geographic locations: an empirical study with solar array detection, in: 2017 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), 10–12 October 2017, Washington, DC, USA, IEEE, 1–8, https://doi.org/10.1109/AIPR.2017.8457960, 2017. a
Xie, Y., Sengupta, M., and Dudhia, J.: A Fast All-sky Radiation Model for
Solar applications (FARMS): Algorithm and performance evaluation, Sol.
Energy, 135, 435–445, https://doi.org/10.1016/j.solener.2016.06.003, 2016. a
Yang, X., Bright, J. M., Gueymard, C. A., Acord, B., and Wang, P.: Worldwide validation of an Earth Polychromatic Imaging Camera (EPIC) derived radiation product and comparison with recent reanalyses, Sol. Energy, 243, 421–430,
https://doi.org/10.1016/j.solener.2022.08.013, 2022. a
Short summary
Solar surface irradiance (SSI) estimations inferred from satellite images are essential to gain a comprehensive understanding of the solar resource, which is crucial in many fields. This study examines the recent data-driven methods for inferring SSI from satellite images and explores their strengths and weaknesses. The results suggest that while these methods show great promise, they sometimes dramatically underperform and should probably be used in conjunction with physical approaches.
Solar surface irradiance (SSI) estimations inferred from satellite images are essential to gain...