IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v285y2021ics0306261920317748.html
   My bibliography  Save this article

Assessing the performance of deep learning models for multivariate probabilistic energy forecasting

Author

Listed:
  • Mashlakov, Aleksei
  • Kuronen, Toni
  • Lensu, Lasse
  • Kaarna, Arto
  • Honkapuro, Samuli

Abstract

Deep learning models have the potential to advance the short-term decision-making of electricity market participants and system operators by capturing the complex dependences and uncertainties of power system operation. Currently, however, the adoption of global deep learning models for multivariate energy forecasting in power systems is far behind the developments in the deep learning research field. In this context, the objectives of this study are to review recent developments in the field of probabilistic, multivariate, and multihorizon time series forecasting and empirically evaluate the performance of novel global deep learning models for forecasting wind and solar generation, electricity load, and wholesale electricity price for intraday and day-ahead time horizons. Two forecast types, deterministic and probabilistic forecasts, are studied. The evaluation data consist of real-world datasets with hourly resolution at the levels of an individual customer and regional and national electricity market bidding zones. The model evaluation criteria include achievable levels of forecasting accuracy and uncertainty risks, hyperparameter sensitivity, the effect of exogenous variables and fieldwise dataset split, and run-time efficiency factors, such as memory utilization, simulation time, electricity consumption, and convergence rate. We conclude that the performance of the global models is more beneficial for intraday forecasts of heterogeneous datasets with nonuniform patterns of time series, but can be affected by the hyperparameter sensitivity and hardware limitations with the growth of dataset dimensionality. The results can serve as a reference point for the quantitative evaluation of deep learning models for probabilistic multivariate energy forecasting in power systems.

Suggested Citation

  • Mashlakov, Aleksei & Kuronen, Toni & Lensu, Lasse & Kaarna, Arto & Honkapuro, Samuli, 2021. "Assessing the performance of deep learning models for multivariate probabilistic energy forecasting," Applied Energy, Elsevier, vol. 285(C).
  • Handle: RePEc:eee:appene:v:285:y:2021:i:c:s0306261920317748
    DOI: 10.1016/j.apenergy.2020.116405
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261920317748
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2020.116405?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Brusaferri, Alessandro & Matteucci, Matteo & Portolani, Pietro & Vitali, Andrea, 2019. "Bayesian deep learning based method for probabilistic forecast of day-ahead electricity prices," Applied Energy, Elsevier, vol. 250(C), pages 1158-1175.
    2. Ricardo J. Bessa & Corinna Möhrlen & Vanessa Fundel & Malte Siefert & Jethro Browell & Sebastian Haglund El Gaidi & Bri-Mathias Hodge & Umit Cali & George Kariniotakis, 2017. "Towards Improved Understanding of the Applicability of Uncertainty Forecasts in the Electric Power Industry," Energies, MDPI, vol. 10(9), pages 1-48, September.
    3. Conor Sweeney & Ricardo J. Bessa & Jethro Browell & Pierre Pinson, 2020. "The future of forecasting for renewable energy," Wiley Interdisciplinary Reviews: Energy and Environment, Wiley Blackwell, vol. 9(2), March.
    4. Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
    5. De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.
    6. Rafal Weron & Florian Ziel, 2018. "Electricity price forecasting," HSC Research Reports HSC/18/08, Hugo Steinhaus Center, Wroclaw University of Technology.
    7. Tilmann Gneiting & Fadoua Balabdaoui & Adrian E. Raftery, 2007. "Probabilistic forecasts, calibration and sharpness," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(2), pages 243-268, April.
    8. Luc Bauwens & Sébastien Laurent & Jeroen V. K. Rombouts, 2006. "Multivariate GARCH models: a survey," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(1), pages 79-109, January.
    9. Zang, Haixiang & Liu, Ling & Sun, Li & Cheng, Lilin & Wei, Zhinong & Sun, Guoqiang, 2020. "Short-term global horizontal irradiance forecasting based on a hybrid CNN-LSTM model with spatiotemporal correlations," Renewable Energy, Elsevier, vol. 160(C), pages 26-41.
    10. Montero-Manso, Pablo & Hyndman, Rob J., 2021. "Principles and algorithms for forecasting groups of time series: Locality and globality," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1632-1653.
    11. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    12. Croonenbroeck, Carsten & Stadtmann, Georg, 2019. "Renewable generation forecast studies – Review and good practice guidance," Renewable and Sustainable Energy Reviews, Elsevier, vol. 108(C), pages 312-322.
    13. Yang, Yandong & Hong, Weijun & Li, Shufang, 2019. "Deep ensemble learning based probabilistic load forecasting in smart grids," Energy, Elsevier, vol. 189(C).
    14. van der Meer, D.W. & Widén, J. & Munkhammar, J., 2018. "Review on probabilistic forecasting of photovoltaic power production and electricity consumption," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1484-1512.
    15. Wang, Huai-zhi & Li, Gang-qiang & Wang, Gui-bin & Peng, Jian-chun & Jiang, Hui & Liu, Yi-tao, 2017. "Deep learning based ensemble approach for probabilistic wind power forecasting," Applied Energy, Elsevier, vol. 188(C), pages 56-70.
    16. Ricardo Bessa & Carlos Moreira & Bernardo Silva & Manuel Matos, 2014. "Handling renewable energy variability and uncertainty in power systems operation," Wiley Interdisciplinary Reviews: Energy and Environment, Wiley Blackwell, vol. 3(2), pages 156-178, March.
    17. Hong, Tao & Fan, Shu, 2016. "Probabilistic electric load forecasting: A tutorial review," International Journal of Forecasting, Elsevier, vol. 32(3), pages 914-938.
    18. Liu, Yongqi & Qin, Hui & Zhang, Zhendong & Pei, Shaoqian & Jiang, Zhiqiang & Feng, Zhongkai & Zhou, Jianzhong, 2020. "Probabilistic spatiotemporal wind speed forecasting based on a variational Bayesian deep learning model," Applied Energy, Elsevier, vol. 260(C).
    19. Amanda Lenzi & Ingelin Steinsland & Pierre Pinson, 2018. "Benefits of spatiotemporal modeling for short‐term wind power forecasting at both individual and aggregated levels," Environmetrics, John Wiley & Sons, Ltd., vol. 29(3), May.
    20. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    21. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    22. Wang, Kejun & Qi, Xiaoxia & Liu, Hongda, 2019. "A comparison of day-ahead photovoltaic power forecasting models based on deep learning neural network," Applied Energy, Elsevier, vol. 251(C), pages 1-1.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dumas, Jonathan & Wehenkel, Antoine & Lanaspeze, Damien & Cornélusse, Bertrand & Sutera, Antonio, 2022. "A deep generative model for probabilistic energy forecasting in power systems: normalizing flows," Applied Energy, Elsevier, vol. 305(C).
    2. Hammerschmitt, Bruno Knevitz & Guarda, Fernando Guilherme Kaehler & Lucchese, Felipe Cirolini & Abaide, Alzenira da Rosa, 2022. "Complementary thermal energy generation associated with renewable energies using Artificial Intelligence," Energy, Elsevier, vol. 254(PB).
    3. Jozef Barunik & Lubos Hanus, 2023. "Learning Probability Distributions of Day-Ahead Electricity Prices," Papers 2310.02867, arXiv.org, revised Oct 2023.
    4. Jeong, Jaeik & Kim, Hongseok, 2021. "DeepComp: Deep reinforcement learning based renewable energy error compensable forecasting," Applied Energy, Elsevier, vol. 294(C).
    5. Hu, Yusha & Man, Yi, 2023. "Energy consumption and carbon emissions forecasting for industrial processes: Status, challenges and perspectives," Renewable and Sustainable Energy Reviews, Elsevier, vol. 182(C).
    6. Haokun Su & Xiangang Peng & Hanyu Liu & Huan Quan & Kaitong Wu & Zhiwen Chen, 2022. "Multi-Step-Ahead Electricity Price Forecasting Based on Temporal Graph Convolutional Network," Mathematics, MDPI, vol. 10(14), pages 1-16, July.
    7. Park, Jungyeon & Alvarenga, Estêvão & Jeon, Jooyoung & Li, Ran & Petropoulos, Fotios & Kim, Hokyun & Ahn, Kwangwon, 2024. "Probabilistic forecast-based portfolio optimization of electricity demand at low aggregation levels," Applied Energy, Elsevier, vol. 353(PB).
    8. Farid Moazzen & M. J. Hossain, 2024. "Multivariate Deep Learning Long Short-Term Memory-Based Forecasting for Microgrid Energy Management Systems," Energies, MDPI, vol. 17(17), pages 1-16, August.
    9. Yin, Linfei & Cao, Xinghui & Liu, Dongduan, 2023. "Weighted fully-connected regression networks for one-day-ahead hourly photovoltaic power forecasting," Applied Energy, Elsevier, vol. 332(C).
    10. Perera, Maneesha & De Hoog, Julian & Bandara, Kasun & Senanayake, Damith & Halgamuge, Saman, 2024. "Day-ahead regional solar power forecasting with hierarchical temporal convolutional neural networks using historical power generation and weather data," Applied Energy, Elsevier, vol. 361(C).
    11. Marcjasz, Grzegorz & Narajewski, Michał & Weron, Rafał & Ziel, Florian, 2023. "Distributional neural networks for electricity price forecasting," Energy Economics, Elsevier, vol. 125(C).
    12. Eikeland, Odin Foldvik & Kelsall, Colin C. & Buznitsky, Kyle & Verma, Shomik & Bianchi, Filippo Maria & Chiesa, Matteo & Henry, Asegun, 2023. "Power availability of PV plus thermal batteries in real-world electric power grids," Applied Energy, Elsevier, vol. 348(C).
    13. Mayer, Martin János & Biró, Bence & Szücs, Botond & Aszódi, Attila, 2023. "Probabilistic modeling of future electricity systems with high renewable energy penetration using machine learning," Applied Energy, Elsevier, vol. 336(C).
    14. Hernandez-Matheus, Alejandro & Löschenbrand, Markus & Berg, Kjersti & Fuchs, Ida & Aragüés-Peñalba, Mònica & Bullich-Massagué, Eduard & Sumper, Andreas, 2022. "A systematic review of machine learning techniques related to local energy communities," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    15. Baggio, Roberta & Muzy, Jean-François, 2024. "Improving probabilistic wind speed forecasting using M-Rice distribution and spatial data integration," Applied Energy, Elsevier, vol. 360(C).
    16. Stover, Oliver & Nath, Paromita & Karve, Pranav & Mahadevan, Sankaran & Baroud, Hiba, 2024. "Dependence structure learning and joint probabilistic forecasting of stochastic power grid variables," Applied Energy, Elsevier, vol. 357(C).
    17. Chen, Xiaoyang & Du, Yang & Lim, Enggee & Fang, Lurui & Yan, Ke, 2022. "Towards the applicability of solar nowcasting: A practice on predictive PV power ramp-rate control," Renewable Energy, Elsevier, vol. 195(C), pages 147-166.
    18. Katarzyna Maciejowska & Bartosz Uniejewski & Rafa{l} Weron, 2022. "Forecasting Electricity Prices," Papers 2204.11735, arXiv.org.
    19. Shao, Zhen & Yang, Yudie & Zheng, Qingru & Zhou, Kaile & Liu, Chen & Yang, Shanlin, 2022. "A pattern classification methodology for interval forecasts of short-term electricity prices based on hybrid deep neural networks: A comparative analysis," Applied Energy, Elsevier, vol. 327(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Croonenbroeck, Carsten & Stadtmann, Georg, 2019. "Renewable generation forecast studies – Review and good practice guidance," Renewable and Sustainable Energy Reviews, Elsevier, vol. 108(C), pages 312-322.
    3. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    4. Derek W. Bunn & Angelica Gianfreda & Stefan Kermer, 2018. "A Trading-Based Evaluation of Density Forecasts in a Real-Time Electricity Market," Energies, MDPI, vol. 11(10), pages 1-13, October.
    5. Roberto Baviera & Giuseppe Messuti, 2020. "Daily Middle-Term Probabilistic Forecasting of Power Consumption in North-East England," Papers 2005.13005, arXiv.org, revised Oct 2020.
    6. Katarzyna Maciejowska & Bartosz Uniejewski & Rafa{l} Weron, 2022. "Forecasting Electricity Prices," Papers 2204.11735, arXiv.org.
    7. Rafal Weron & Florian Ziel, 2018. "Electricity price forecasting," HSC Research Reports HSC/18/08, Hugo Steinhaus Center, Wroclaw University of Technology.
    8. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    9. Sharifzadeh, Mahdi & Sikinioti-Lock, Alexandra & Shah, Nilay, 2019. "Machine-learning methods for integrated renewable power generation: A comparative study of artificial neural networks, support vector regression, and Gaussian Process Regression," Renewable and Sustainable Energy Reviews, Elsevier, vol. 108(C), pages 513-538.
    10. Barja-Martinez, Sara & Aragüés-Peñalba, Mònica & Munné-Collado, Íngrid & Lloret-Gallego, Pau & Bullich-Massagué, Eduard & Villafafila-Robles, Roberto, 2021. "Artificial intelligence techniques for enabling Big Data services in distribution networks: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 150(C).
    11. Stavros Degiannakis & Apostolos Kiohos, 2014. "Multivariate modelling of 10-day-ahead VaR and dynamic correlation for worldwide real estate and stock indices," Journal of Economic Studies, Emerald Group Publishing Limited, vol. 41(2), pages 216-232, March.
    12. Uniejewski, Bartosz & Marcjasz, Grzegorz & Weron, Rafał, 2019. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting: Part II — Probabilistic forecasting," Energy Economics, Elsevier, vol. 79(C), pages 171-182.
    13. Weng, Haijie & Trück, Stefan, 2011. "Style analysis and Value-at-Risk of Asia-focused hedge funds," Pacific-Basin Finance Journal, Elsevier, vol. 19(5), pages 491-510, November.
    14. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    15. Uniejewski, Bartosz & Marcjasz, Grzegorz & Weron, Rafał, 2019. "Understanding intraday electricity markets: Variable selection and very short-term price forecasting using LASSO," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1533-1547.
    16. Marcjasz, Grzegorz & Narajewski, Michał & Weron, Rafał & Ziel, Florian, 2023. "Distributional neural networks for electricity price forecasting," Energy Economics, Elsevier, vol. 125(C).
    17. Tsyplakov, Alexander, 2014. "Theoretical guidelines for a partially informed forecast examiner," MPRA Paper 55017, University Library of Munich, Germany.
    18. Erik Kole & Thijs Markwat & Anne Opschoor & Dick van Dijk, 2017. "Forecasting Value-at-Risk under Temporal and Portfolio Aggregation," Journal of Financial Econometrics, Oxford University Press, vol. 15(4), pages 649-677.
    19. Nadja Klein & Michael Stanley Smith & David J. Nott, 2020. "Deep Distributional Time Series Models and the Probabilistic Forecasting of Intraday Electricity Prices," Papers 2010.01844, arXiv.org, revised May 2021.
    20. Arne Vogler & Florian Ziel, "undated". "On The Evaluation Of Binary Event Probability Predictions In Electricity Price Forecasting," EWL Working Papers 1911, University of Duisburg-Essen, Chair for Management Science and Energy Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:285:y:2021:i:c:s0306261920317748. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.