IDEAS home Printed from https://ideas.repec.org/a/eee/rensus/v138y2021ics1364032120308005.html
   My bibliography  Save this article

An overview of performance evaluation metrics for short-term statistical wind power forecasting

Author

Listed:
  • González-Sopeña, J.M.
  • Pakrashi, V.
  • Ghosh, B.

Abstract

Wind power forecasting has become an essential tool for energy trading and the operation of the grid due to the increasing importance of wind energy. Therefore, estimating the forecast accuracy of a WPF model and understanding how the accuracy is calculated are necessary steps to appropriately validate WPF models. The present study gives an extensive overview of the performance evaluation methods used for assessing the forecast accuracy of short-term statistical wind power forecast estimates, and the concept of robustness is introduced to determine the validity of a model over different wind power generation scenarios over the testing set. Finally, a numerical study using decomposition-based hybrid models is presented to analyse the robustness of the performance evaluation metrics under different conditions in the context of wind power forecasting. Data from Ireland are employed using two different resolutions to examine its influence on the forecast accuracy.

Suggested Citation

  • González-Sopeña, J.M. & Pakrashi, V. & Ghosh, B., 2021. "An overview of performance evaluation metrics for short-term statistical wind power forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 138(C).
    • Handle: RePEc:eee:rensus:v:138:y:2021:i:c:s1364032120308005
    DOI: 10.1016/j.rser.2020.110515
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1364032120308005
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.rser.2020.110515?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tascikaraoglu, A. & Uzunoglu, M., 2014. "A review of combined approaches for prediction of short-term wind speed and power," Renewable and Sustainable Energy Reviews, Elsevier, vol. 34(C), pages 243-254.
    2. Mahmoud, Tawfek & Dong, Z.Y. & Ma, Jin, 2018. "An advanced approach for optimal wind power generation prediction intervals by using self-adaptive evolutionary extreme learning machine," Renewable Energy, Elsevier, vol. 126(C), pages 254-269.
    3. Conor Sweeney & Ricardo J. Bessa & Jethro Browell & Pierre Pinson, 2020. "The future of forecasting for renewable energy," Wiley Interdisciplinary Reviews: Energy and Environment, Wiley Blackwell, vol. 9(2), March.
    4. Zhang, Yao & Wang, Jianxue & Wang, Xifan, 2014. "Review on probabilistic forecasting of wind power generation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 32(C), pages 255-270.
    5. Colak, Ilhami & Sagiroglu, Seref & Yesilbudak, Mehmet, 2012. "Data mining and wind power prediction: A literature review," Renewable Energy, Elsevier, vol. 46(C), pages 241-247.
    6. P. Pinson, 2012. "Very-short-term probabilistic forecasting of wind power with generalized logit–normal distributions," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 61(4), pages 555-576, August.
    7. Koenker, Roger W & Bassett, Gilbert, Jr, 1978. "Regression Quantiles," Econometrica, Econometric Society, vol. 46(1), pages 33-50, January.
    8. Hong, Tao & Pinson, Pierre & Fan, Shu, 2014. "Global Energy Forecasting Competition 2012," International Journal of Forecasting, Elsevier, vol. 30(2), pages 357-363.
    9. Hong, Tao & Pinson, Pierre & Fan, Shu & Zareipour, Hamidreza & Troccoli, Alberto & Hyndman, Rob J., 2016. "Probabilistic energy forecasting: Global Energy Forecasting Competition 2014 and beyond," International Journal of Forecasting, Elsevier, vol. 32(3), pages 896-913.
    10. Gallego-Castillo, Cristobal & Bessa, Ricardo & Cavalcante, Laura & Lopez-Garcia, Oscar, 2016. "On-line quantile regression in the RKHS (Reproducing Kernel Hilbert Space) for operational probabilistic forecasting of wind power," Energy, Elsevier, vol. 113(C), pages 355-365.
    11. Spyros Makridakis & Evangelos Spiliotis & Vassilios Assimakopoulos, 2018. "Statistical and Machine Learning forecasting methods: Concerns and ways forward," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-26, March.
    12. Yuan, Xiaohui & Tan, Qingxiong & Lei, Xiaohui & Yuan, Yanbin & Wu, Xiaotao, 2017. "Wind power prediction using hybrid autoregressive fractionally integrated moving average and least square support vector machine," Energy, Elsevier, vol. 129(C), pages 122-137.
    13. Zhang, Yao & Wang, Jianxue, 2016. "K-nearest neighbors and a kernel density estimator for GEFCom2014 probabilistic wind power forecasting," International Journal of Forecasting, Elsevier, vol. 32(3), pages 1074-1080.
    14. Alessandrini, S. & Delle Monache, L. & Sperati, S. & Nissen, J.N., 2015. "A novel application of an analog ensemble for short-term wind power forecasting," Renewable Energy, Elsevier, vol. 76(C), pages 768-781.
    15. Yan, Jie & Liu, Yongqian & Han, Shuang & Wang, Yimei & Feng, Shuanglei, 2015. "Reviews on uncertainty analysis of wind power forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 52(C), pages 1322-1330.
    16. Qian, Zheng & Pei, Yan & Zareipour, Hamidreza & Chen, Niya, 2019. "A review and discussion of decomposition-based hybrid models for wind energy forecasting applications," Applied Energy, Elsevier, vol. 235(C), pages 939-953.
    17. Wang, Jianzhou & Song, Yiliao & Liu, Feng & Hou, Ru, 2016. "Analysis and application of forecasting models in wind power integration: A review of multi-step-ahead wind speed forecasting models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 60(C), pages 960-981.
    18. Heinermann, Justin & Kramer, Oliver, 2016. "Machine learning ensembles for wind power prediction," Renewable Energy, Elsevier, vol. 89(C), pages 671-679.
    19. Kim, Sungil & Kim, Heeyoung, 2016. "A new metric of absolute percentage error for intermittent demand forecasts," International Journal of Forecasting, Elsevier, vol. 32(3), pages 669-679.
    20. Wang, Huai-zhi & Li, Gang-qiang & Wang, Gui-bin & Peng, Jian-chun & Jiang, Hui & Liu, Yi-tao, 2017. "Deep learning based ensemble approach for probabilistic wind power forecasting," Applied Energy, Elsevier, vol. 188(C), pages 56-70.
    21. Zhang, Jinhua & Meng, Hang & Gu, Bo & Li, Pin, 2020. "Research on short-term wind power combined forecasting and its Gaussian cloud uncertainty to support the integration of renewables and EVs," Renewable Energy, Elsevier, vol. 153(C), pages 884-899.
    22. Yan, Jie & Liu, Yongqian & Han, Shuang & Qiu, Meng, 2013. "Wind power grouping forecasts and its uncertainty analysis using optimized relevance vector machine," Renewable and Sustainable Energy Reviews, Elsevier, vol. 27(C), pages 613-621.
    23. Gallego-Castillo, Cristobal & Cuerva-Tejero, Alvaro & Lopez-Garcia, Oscar, 2015. "A review on the recent history of wind power ramp forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 52(C), pages 1148-1157.
    24. Foley, Aoife M. & Leahy, Paul G. & Marvuglia, Antonino & McKeogh, Eamon J., 2012. "Current methods and advances in forecasting of wind power generation," Renewable Energy, Elsevier, vol. 37(1), pages 1-8.
    25. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    26. Costa, Alexandre & Crespo, Antonio & Navarro, Jorge & Lizcano, Gil & Madsen, Henrik & Feitosa, Everaldo, 2008. "A review on the young history of the wind power short-term prediction," Renewable and Sustainable Energy Reviews, Elsevier, vol. 12(6), pages 1725-1744, August.
    27. Chang, G.W. & Lu, H.J. & Chang, Y.R. & Lee, Y.D., 2017. "An improved neural network-based approach for short-term wind speed and power forecast," Renewable Energy, Elsevier, vol. 105(C), pages 301-311.
    28. Liu, Jinqiang & Wang, Xiaoru & Lu, Yun, 2017. "A novel hybrid methodology for short-term wind power forecasting based on adaptive neuro-fuzzy inference system," Renewable Energy, Elsevier, vol. 103(C), pages 620-629.
    29. Messner, Jakob W. & Pinson, Pierre, 2019. "Online adaptive lasso estimation in vector autoregressive models for high dimensional wind power forecasting," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1485-1498.
    30. Hao, Yan & Tian, Chengshi, 2019. "A novel two-stage forecasting model based on error factor and ensemble method for multi-step wind power forecasting," Applied Energy, Elsevier, vol. 238(C), pages 368-383.
    31. Han, Li & Romero, Carlos E. & Yao, Zheng, 2015. "Wind power forecasting based on principle component phase space reconstruction," Renewable Energy, Elsevier, vol. 81(C), pages 737-744.
    32. C. W. J. Granger & Roselyne Joyeux, 1980. "An Introduction To Long‐Memory Time Series Models And Fractional Differencing," Journal of Time Series Analysis, Wiley Blackwell, vol. 1(1), pages 15-29, January.
    33. Zhao, Yongning & Ye, Lin & Li, Zhi & Song, Xuri & Lang, Yansheng & Su, Jian, 2016. "A novel bidirectional mechanism based on time series model for wind power forecasting," Applied Energy, Elsevier, vol. 177(C), pages 793-803.
    34. Lahouar, A. & Ben Hadj Slama, J., 2017. "Hour-ahead wind power forecast based on random forests," Renewable Energy, Elsevier, vol. 109(C), pages 529-541.
    35. Naik, Jyotirmayee & Dash, Sujit & Dash, P.K. & Bisoi, Ranjeeta, 2018. "Short term wind power forecasting using hybrid variational mode decomposition and multi-kernel regularized pseudo inverse neural network," Renewable Energy, Elsevier, vol. 118(C), pages 180-212.
    36. Chen, Chunhang, 1997. "Robustness properties of some forecasting methods for seasonal time series: A Monte Carlo study," International Journal of Forecasting, Elsevier, vol. 13(2), pages 269-280, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yitian Xing & Fue-Sang Lien & William Melek & Eugene Yee, 2022. "A Multi-Hour Ahead Wind Power Forecasting System Based on a WRF-TOPSIS-ANFIS Model," Energies, MDPI, vol. 15(15), pages 1-35, July.
    2. Cheng, Lilin & Zang, Haixiang & Wei, Zhinong & Zhang, Fengchun & Sun, Guoqiang, 2022. "Evaluation of opaque deep-learning solar power forecast models towards power-grid applications," Renewable Energy, Elsevier, vol. 198(C), pages 960-972.
    3. Zhong, Lingshu & Wu, Pan & Pei, Mingyang, 2024. "Wind power generation prediction during the COVID-19 epidemic based on novel hybrid deep learning techniques," Renewable Energy, Elsevier, vol. 222(C).
    4. Hu, Yue & Liu, Hanjing & Wu, Senzhen & Zhao, Yuan & Wang, Zhijin & Liu, Xiufeng, 2024. "Temporal collaborative attention for wind power forecasting," Applied Energy, Elsevier, vol. 357(C).
    5. Lu, Peng & Ye, Lin & Zhao, Yongning & Dai, Binhua & Pei, Ming & Tang, Yong, 2021. "Review of meta-heuristic algorithms for wind power prediction: Methodologies, applications and challenges," Applied Energy, Elsevier, vol. 301(C).
    6. Zhong, Mingwei & Xu, Cancheng & Xian, Zikang & He, Guanglin & Zhai, Yanpeng & Zhou, Yongwang & Fan, Jingmin, 2024. "DTTM: A deep temporal transfer model for ultra-short-term online wind power forecasting," Energy, Elsevier, vol. 286(C).
    7. Heng, Jiani & Hong, Yongmiao & Hu, Jianming & Wang, Shouyang, 2022. "Probabilistic and deterministic wind speed forecasting based on non-parametric approaches and wind characteristics information," Applied Energy, Elsevier, vol. 306(PA).
    8. Hou, Guolian & Wang, Junjie & Fan, Yuzhen, 2024. "Multistep short-term wind power forecasting model based on secondary decomposition, the kernel principal component analysis, an enhanced arithmetic optimization algorithm, and error correction," Energy, Elsevier, vol. 286(C).
    9. Yang, Ting & Yang, Zhenning & Li, Fei & Wang, Hengyu, 2024. "A short-term wind power forecasting method based on multivariate signal decomposition and variable selection," Applied Energy, Elsevier, vol. 360(C).
    10. Lu, Peng & Yang, Jianbin & Ye, Lin & Zhang, Ning & Wang, Yaqing & Di, Jingyi & Gao, Ze & Wang, Cheng & Liu, Mingyang, 2024. "A novel adaptively combined model based on induced ordered weighted averaging for wind power forecasting," Renewable Energy, Elsevier, vol. 226(C).
    11. Niu, Dongxiao & Sun, Lijie & Yu, Min & Wang, Keke, 2022. "Point and interval forecasting of ultra-short-term wind power based on a data-driven method and hybrid deep learning model," Energy, Elsevier, vol. 254(PA).
    12. Zhang, Jiaan & Liu, Dong & Li, Zhijun & Han, Xu & Liu, Hui & Dong, Cun & Wang, Junyan & Liu, Chenyu & Xia, Yunpeng, 2021. "Power prediction of a wind farm cluster based on spatiotemporal correlations," Applied Energy, Elsevier, vol. 302(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jin, Huaiping & Shi, Lixian & Chen, Xiangguang & Qian, Bin & Yang, Biao & Jin, Huaikang, 2021. "Probabilistic wind power forecasting using selective ensemble of finite mixture Gaussian process regression models," Renewable Energy, Elsevier, vol. 174(C), pages 1-18.
    2. Gensler, André & Sick, Bernhard & Vogt, Stephan, 2018. "A review of uncertainty representations and metaverification of uncertainty assessment techniques for renewable energies," Renewable and Sustainable Energy Reviews, Elsevier, vol. 96(C), pages 352-379.
    3. Kim, Deockho & Hur, Jin, 2018. "Short-term probabilistic forecasting of wind energy resources using the enhanced ensemble method," Energy, Elsevier, vol. 157(C), pages 211-226.
    4. Lahouar, A. & Ben Hadj Slama, J., 2017. "Hour-ahead wind power forecast based on random forests," Renewable Energy, Elsevier, vol. 109(C), pages 529-541.
    5. Gallego-Castillo, Cristobal & Bessa, Ricardo & Cavalcante, Laura & Lopez-Garcia, Oscar, 2016. "On-line quantile regression in the RKHS (Reproducing Kernel Hilbert Space) for operational probabilistic forecasting of wind power," Energy, Elsevier, vol. 113(C), pages 355-365.
    6. Tawn, R. & Browell, J., 2022. "A review of very short-term wind and solar power forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 153(C).
    7. Fabrizio De Caro & Jacopo De Stefani & Gianluca Bontempi & Alfredo A. Vaccaro & Domenico D. Villacci, 2020. "Robust Assessment of Short-Term Wind Power Forecasting Models on Multiple Time Horizons," ULB Institutional Repository 2013/314435, ULB -- Universite Libre de Bruxelles.
    8. Liu, Yanli & Wang, Junyi, 2022. "Transfer learning based multi-layer extreme learning machine for probabilistic wind power forecasting," Applied Energy, Elsevier, vol. 312(C).
    9. Wang, Yun & Zou, Runmin & Liu, Fang & Zhang, Lingjun & Liu, Qianyi, 2021. "A review of wind speed and wind power forecasting with deep neural networks," Applied Energy, Elsevier, vol. 304(C).
    10. Costa, Marcelo Azevedo & Ruiz-Cárdenas, Ramiro & Mineti, Leandro Brioschi & Prates, Marcos Oliveira, 2021. "Dynamic time scan forecasting for multi-step wind speed prediction," Renewable Energy, Elsevier, vol. 177(C), pages 584-595.
    11. Jha, Sunil Kr. & Bilalovic, Jasmin & Jha, Anju & Patel, Nilesh & Zhang, Han, 2017. "Renewable energy: Present research and future scope of Artificial Intelligence," Renewable and Sustainable Energy Reviews, Elsevier, vol. 77(C), pages 297-317.
    12. Lv, Jiaqing & Zheng, Xiaodong & Pawlak, Mirosław & Mo, Weike & Miśkowicz, Marek, 2021. "Very short-term probabilistic wind power prediction using sparse machine learning and nonparametric density estimation algorithms," Renewable Energy, Elsevier, vol. 177(C), pages 181-192.
    13. Yan, Jie & Möhrlen, Corinna & Göçmen, Tuhfe & Kelly, Mark & Wessel, Arne & Giebel, Gregor, 2022. "Uncovering wind power forecasting uncertainty sources and their propagation through the whole modelling chain," Renewable and Sustainable Energy Reviews, Elsevier, vol. 165(C).
    14. Shahriari, M. & Cervone, G. & Clemente-Harding, L. & Delle Monache, L., 2020. "Using the analog ensemble method as a proxy measurement for wind power predictability," Renewable Energy, Elsevier, vol. 146(C), pages 789-801.
    15. Zhong, Mingwei & Xu, Cancheng & Xian, Zikang & He, Guanglin & Zhai, Yanpeng & Zhou, Yongwang & Fan, Jingmin, 2024. "DTTM: A deep temporal transfer model for ultra-short-term online wind power forecasting," Energy, Elsevier, vol. 286(C).
    16. Liu, Hui & Chen, Chao, 2019. "Data processing strategies in wind energy forecasting models and applications: A comprehensive review," Applied Energy, Elsevier, vol. 249(C), pages 392-408.
    17. Sharifzadeh, Mahdi & Sikinioti-Lock, Alexandra & Shah, Nilay, 2019. "Machine-learning methods for integrated renewable power generation: A comparative study of artificial neural networks, support vector regression, and Gaussian Process Regression," Renewable and Sustainable Energy Reviews, Elsevier, vol. 108(C), pages 513-538.
    18. Liu, Xin & Yang, Luoxiao & Zhang, Zijun, 2022. "The attention-assisted ordinary differential equation networks for short-term probabilistic wind power predictions," Applied Energy, Elsevier, vol. 324(C).
    19. Jannik Schütz Roungkvist & Peter Enevoldsen, 2020. "Timescale classification in wind forecasting: A review of the state‐of‐the‐art," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(5), pages 757-768, August.
    20. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:rensus:v:138:y:2021:i:c:s1364032120308005. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600126/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.