IDEAS home Printed from https://ideas.repec.org/a/eee/rensus/v117y2020ics1364032119306793.html
   My bibliography  Save this article

Beyond quadratic error: Case-study of a multiple criteria approach to the performance assessment of numerical forecasts of solar irradiance in the tropics

Author

Listed:
  • Verbois, Hadrien
  • Blanc, Philippe
  • Huva, Robert
  • Saint-Drenan, Yves-Marie
  • Rusydi, Andrivo
  • Thiery, Alexandre

Abstract

As the penetration of photovoltaic power in the electrical grid increases, reliable forecasts of solar irradiance are becoming ever more critical. For forecasting horizons beyond a few hours, Numerical Weather Prediction models (NWP) are stapled to yield the best results. For potential users of NWP forecasts, choosing which model to use can be challenging, as a large palette of NWP models and configurations is available. To tackle this issue, several studies have conducted systematic comparisons between available models and configurations. Most of these studies, however, only evaluate forecast skills with a small number of metrics, such as the Root Mean Square Error (RMSE) or the Mean Average Error (MAE). To better understand the nature and specificities of various NWP models, we developed an evaluation approach that combines multiple-criteria analysis, Fourier analysis and classification of daily profiles of irradiance. In this paper, we present this novel approach and demonstrate its strengths by applying it to a practical use-case, using 108 different NWP forecasts models based on different parametrizations of the mesoscale NWP Weather Research and Forecasting model (WRF) in Singapore. Our approach allowed a clearer overview of the forecasts’ skills, as well as better discrimination between models. It also significantly improved our understanding of the nature of the NWP errors. In particular, WRF forecasts in Singapore were found unfit to timely resolve irradiance at an hourly scale, but better adapted to predict daily profiles of irradiance. Furthermore, the proposed multiple criteria approach was applied to a sensitivity analysis of WRF physical schemes, providing a better insight into the impact of each model. This work shows that it is critical to go beyond RMSE when evaluating forecasts and that more holistic approaches such as the one proposed here should be preferred.

Suggested Citation

  • Verbois, Hadrien & Blanc, Philippe & Huva, Robert & Saint-Drenan, Yves-Marie & Rusydi, Andrivo & Thiery, Alexandre, 2020. "Beyond quadratic error: Case-study of a multiple criteria approach to the performance assessment of numerical forecasts of solar irradiance in the tropics," Renewable and Sustainable Energy Reviews, Elsevier, vol. 117(C).
    • Handle: RePEc:eee:rensus:v:117:y:2020:i:c:s1364032119306793
    DOI: 10.1016/j.rser.2019.109471
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1364032119306793
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.rser.2019.109471?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Philippe Lauret & Mathieu David & Hugo T. C. Pedro, 2017. "Probabilistic Solar Forecasting Using Quantile Regression Models," Energies, MDPI, vol. 10(10), pages 1-17, October.
    2. Zempila, Melina-Maria & Giannaros, Theodore M. & Bais, Alkiviadis & Melas, Dimitris & Kazantzidis, Andreas, 2016. "Evaluation of WRF shortwave radiation parameterizations in predicting Global Horizontal Irradiance in Greece," Renewable Energy, Elsevier, vol. 86(C), pages 831-840.
    3. Engeland, Kolbjørn & Borga, Marco & Creutin, Jean-Dominique & François, Baptiste & Ramos, Maria-Helena & Vidal, Jean-Philippe, 2017. "Space-time variability of climate variables and intermittent renewable electricity production – A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 79(C), pages 600-617.
    4. Stram, Bruce N., 2016. "Key challenges to expanding renewable energy," Energy Policy, Elsevier, vol. 96(C), pages 728-734.
    5. Sen, Souvik & Ganguly, Sourav, 2017. "Opportunities, barriers and issues with renewable energy development – A discussion," Renewable and Sustainable Energy Reviews, Elsevier, vol. 69(C), pages 1170-1181.
    6. Kaur, Amanpreet & Nonnenmacher, Lukas & Pedro, Hugo T.C. & Coimbra, Carlos F.M., 2016. "Benefits of solar forecasting for energy imbalance markets," Renewable Energy, Elsevier, vol. 86(C), pages 819-830.
    7. Diagne, Maimouna & David, Mathieu & Lauret, Philippe & Boland, John & Schmutz, Nicolas, 2013. "Review of solar irradiance forecasting methods and a proposition for small-scale insular grids," Renewable and Sustainable Energy Reviews, Elsevier, vol. 27(C), pages 65-76.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Panagiotis Kosmopoulos & Dimitris Kouroutsidis & Kyriakoula Papachristopoulou & Panagiotis Ioannis Raptis & Akriti Masoom & Yves-Marie Saint-Drenan & Philippe Blanc & Charalampos Kontoes & Stelios Kaz, 2020. "Short-Term Forecasting of Large-Scale Clouds Impact on Downwelling Surface Solar Irradiation," Energies, MDPI, vol. 13(24), pages 1-22, December.
    2. Thomas Carrière & Rodrigo Amaro e Silva & Fuqiang Zhuang & Yves-Marie Saint-Drenan & Philippe Blanc, 2021. "A New Approach for Satellite-Based Probabilistic Solar Forecasting with Cloud Motion Vectors," Energies, MDPI, vol. 14(16), pages 1-19, August.
    3. Luerssen, Christoph & Verbois, Hadrien & Gandhi, Oktoviano & Reindl, Thomas & Sekhar, Chandra & Cheong, David, 2021. "Global sensitivity and uncertainty analysis of the levelised cost of storage (LCOS) for solar-PV-powered cooling," Applied Energy, Elsevier, vol. 286(C).
    4. Abhnil Amtesh Prasad & Merlinde Kay, 2020. "Assessment of Simulated Solar Irradiance on Days of High Intermittency Using WRF-Solar," Energies, MDPI, vol. 13(2), pages 1-22, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Julián Urrego-Ortiz & J. Alejandro Martínez & Paola A. Arias & Álvaro Jaramillo-Duque, 2019. "Assessment and Day-Ahead Forecasting of Hourly Solar Radiation in Medellín, Colombia," Energies, MDPI, vol. 12(22), pages 1-29, November.
    2. Takahiro Takamatsu & Hideaki Ohtake & Takashi Oozeki, 2022. "Support Vector Quantile Regression for the Post-Processing of Meso-Scale Ensemble Prediction System Data in the Kanto Region: Solar Power Forecast Reducing Overestimation," Energies, MDPI, vol. 15(4), pages 1-18, February.
    3. Rodríguez, Fermín & Galarza, Ainhoa & Vasquez, Juan C. & Guerrero, Josep M., 2022. "Using deep learning and meteorological parameters to forecast the photovoltaic generators intra-hour output power interval for smart grid control," Energy, Elsevier, vol. 239(PB).
    4. Ahmed, Adil & Khalid, Muhammad, 2019. "A review on the selected applications of forecasting models in renewable power systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 100(C), pages 9-21.
    5. Michel Fliess & Cédric Join & Cyril Voyant, 2018. "Prediction bands for solar energy: New short-term time series forecasting techniques," Post-Print hal-01736518, HAL.
    6. Elena Collino & Dario Ronzio, 2021. "Exploitation of a New Short-Term Multimodel Photovoltaic Power Forecasting Method in the Very Short-Term Horizon to Derive a Multi-Time Scale Forecasting System," Energies, MDPI, vol. 14(3), pages 1-30, February.
    7. Elsinga, Boudewijn & van Sark, Wilfried G.J.H.M., 2017. "Short-term peer-to-peer solar forecasting in a network of photovoltaic systems," Applied Energy, Elsevier, vol. 206(C), pages 1464-1483.
    8. Alonso-Suárez, R. & David, M. & Branco, V. & Lauret, P., 2020. "Intra-day solar probabilistic forecasts including local short-term variability and satellite information," Renewable Energy, Elsevier, vol. 158(C), pages 554-573.
    9. Benali, L. & Notton, G. & Fouilloy, A. & Voyant, C. & Dizene, R., 2019. "Solar radiation forecasting using artificial neural network and random forest methods: Application to normal beam, horizontal diffuse and global components," Renewable Energy, Elsevier, vol. 132(C), pages 871-884.
    10. Zheng, Hong-Hao & Wang, Zheng-Xin, 2019. "Measurement and comparison of export sophistication of the new energy industry in 30 countries during 2000–2015," Renewable and Sustainable Energy Reviews, Elsevier, vol. 108(C), pages 140-158.
    11. Notton, Gilles & Nivet, Marie-Laure & Voyant, Cyril & Paoli, Christophe & Darras, Christophe & Motte, Fabrice & Fouilloy, Alexis, 2018. "Intermittent and stochastic character of renewable energy sources: Consequences, cost of intermittence and benefit of forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 87(C), pages 96-105.
    12. Javier Borquez & Hector Chavez & Karina A. Barbosa & Marcela Jamett & Rodrigo Acuna, 2020. "A Simple Distribution Energy Tariff under the Penetration of DG," Energies, MDPI, vol. 13(8), pages 1-17, April.
    13. Cheng-Yih Hong & Hsiu-Ching Chang, 2019. "Comparing the Impact of Wind Power and Solar Power Investment on Industrial Development: Application of Dynamic Energy Industry-related Models," International Journal of Energy Economics and Policy, Econjournals, vol. 9(6), pages 38-44.
    14. Ahmed, R. & Sreeram, V. & Mishra, Y. & Arif, M.D., 2020. "A review and evaluation of the state-of-the-art in PV solar power forecasting: Techniques and optimization," Renewable and Sustainable Energy Reviews, Elsevier, vol. 124(C).
    15. Bracken, Cameron & Voisin, Nathalie & Burleyson, Casey D. & Campbell, Allison M. & Hou, Z. Jason & Broman, Daniel, 2024. "Standardized benchmark of historical compound wind and solar energy droughts across the Continental United States," Renewable Energy, Elsevier, vol. 220(C).
    16. Silva, Ana R. & Pousinho, H.M.I. & Estanqueiro, Ana, 2022. "A multistage stochastic approach for the optimal bidding of variable renewable energy in the day-ahead, intraday and balancing markets," Energy, Elsevier, vol. 258(C).
    17. John Boland & Sleiman Farah & Lei Bai, 2022. "Forecasting of Wind and Solar Farm Output in the Australian National Electricity Market: A Review," Energies, MDPI, vol. 15(1), pages 1-18, January.
    18. Ping-Huan Kuo & Chiou-Jye Huang, 2018. "A Green Energy Application in Energy Management Systems by an Artificial Intelligence-Based Solar Radiation Forecasting Model," Energies, MDPI, vol. 11(4), pages 1-15, April.
    19. Mohamed Massaoudi & Ines Chihi & Lilia Sidhom & Mohamed Trabelsi & Shady S. Refaat & Fakhreddine S. Oueslati, 2021. "Enhanced Random Forest Model for Robust Short-Term Photovoltaic Power Forecasting Using Weather Measurements," Energies, MDPI, vol. 14(13), pages 1-20, July.
    20. Jean-Maurice Cadet & Hassan Bencherif & Thierry Portafaix & Kévin Lamy & Katlego Ncongwane & Gerrie J. R. Coetzee & Caradee Y. Wright, 2017. "Comparison of Ground-Based and Satellite-Derived Solar UV Index Levels at Six South African Sites," IJERPH, MDPI, vol. 14(11), pages 1-15, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:rensus:v:117:y:2020:i:c:s1364032119306793. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600126/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.