IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i23p6131-d1537393.html
   My bibliography  Save this article

New Forecasting Metrics Evaluated in Prophet, Random Forest, and Long Short-Term Memory Models for Load Forecasting

Author

Listed:
  • Prajowal Manandhar

    (DEWA R&D Centre, Dubai Electricity and Water Authority, Dubai P.O. Box 564, United Arab Emirates)

  • Hasan Rafiq

    (DEWA R&D Centre, Dubai Electricity and Water Authority, Dubai P.O. Box 564, United Arab Emirates)

  • Edwin Rodriguez-Ubinas

    (DEWA R&D Centre, Dubai Electricity and Water Authority, Dubai P.O. Box 564, United Arab Emirates)

  • Themis Palpanas

    (LIPADE, Université de Paris, 45 Rue Des Saints-Peres, 75006 Paris, France)

Abstract

Data mining is vital for smart grids because it enhances overall grid efficiency, enabling the analysis of large volumes of data, the optimization of energy distribution, the identification of patterns, and demand forecasting. Several performance metrics, such as the MAPE and RMSE, have been created to assess these forecasts. This paper presents new performance metrics called Evaluation Metrics for Performance Quantification (EMPQ), designed to evaluate forecasting models in a more comprehensive and detailed manner. These metrics fill the gap left by established metrics by assessing the likelihood of over- and under-forecasting. The proposed metrics quantify forecast bias through maximum and minimum deviation percentages, assessing the proximity of predicted values to actual consumption and differentiating between over- and under-forecasts. The effectiveness of these metrics is demonstrated through a comparative analysis of short-term load forecasting for residential customers in Dubai. This study was based on high-resolution smart meter data, weather data, and voluntary survey data of household characteristics, which permitted the subdivision of the customers into several groups. The new metrics were demonstrated on the Prophet, Random Forest (RF), and Long Short-term Memory (LSTM) models. EMPQ help to determine that the LSTM model exhibited a superior performance with a maximum deviation of approximately 10% for day-ahead and 20% for week-ahead forecasts in the “AC-included” category, outperforming the Prophet model, which had deviation rates of approximately 44% and 42%, respectively. EMPQ also help to determine that the RF excelled over LSTM for the ‘bedroom-number’ subcategory. The findings highlight the value of the proposed metrics in assessing model performance across diverse subcategories. This study demonstrates the value of tailored forecasting models for accurate load prediction and underscores the importance of enhanced performance metrics in informing model selection and supporting energy management strategies.

Suggested Citation

  • Prajowal Manandhar & Hasan Rafiq & Edwin Rodriguez-Ubinas & Themis Palpanas, 2024. "New Forecasting Metrics Evaluated in Prophet, Random Forest, and Long Short-Term Memory Models for Load Forecasting," Energies, MDPI, vol. 17(23), pages 1-30, December.
    • Handle: RePEc:gam:jeners:v:17:y:2024:i:23:p:6131-:d:1537393
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/23/6131/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/23/6131/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sean J. Taylor & Benjamin Letham, 2018. "Forecasting at Scale," The American Statistician, Taylor & Francis Journals, vol. 72(1), pages 37-45, January.
    2. Prajowal Manandhar & Hasan Rafiq & Edwin Rodriguez-Ubinas & Juan David Barbosa & Omer Ahmed Qureshi & Mahmoud Tarek & Sgouris Sgouridis, 2022. "Understanding Energy Behavioral Changes Due to COVID-19 in the Residents of Dubai Using Electricity Consumption Data and Their Impacts," Energies, MDPI, vol. 16(1), pages 1-23, December.
    3. He, Feifei & Zhou, Jianzhong & Feng, Zhong-kai & Liu, Guangbiao & Yang, Yuqi, 2019. "A hybrid short-term load forecasting model based on variational mode decomposition and long short-term memory networks considering relevant factors with Bayesian optimization algorithm," Applied Energy, Elsevier, vol. 237(C), pages 103-116.
    4. Wan, Anping & Chang, Qing & AL-Bukhaiti, Khalil & He, Jiabo, 2023. "Short-term power load forecasting for combined heat and power using CNN-LSTM enhanced by attention mechanism," Energy, Elsevier, vol. 282(C).
    5. Sadaei, Hossein Javedani & de Lima e Silva, Petrônio Cândido & Guimarães, Frederico Gadelha & Lee, Muhammad Hisyam, 2019. "Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series," Energy, Elsevier, vol. 175(C), pages 365-377.
    6. Wei, Nan & Yin, Lihua & Li, Chao & Wang, Wei & Qiao, Weibiao & Li, Changjun & Zeng, Fanhua & Fu, Lingdi, 2022. "Short-term load forecasting using detrend singular spectrum fluctuation analysis," Energy, Elsevier, vol. 256(C).
    7. Behm, Christian & Nolting, Lars & Praktiknjo, Aaron, 2020. "How to model European electricity load profiles using artificial neural networks," Applied Energy, Elsevier, vol. 277(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vasileios Laitsos & Georgios Vontzos & Paschalis Paraschoudis & Eleftherios Tsampasis & Dimitrios Bargiotas & Lefteri H. Tsoukalas, 2024. "The State of the Art Electricity Load and Price Forecasting for the Modern Wholesale Electricity Market," Energies, MDPI, vol. 17(22), pages 1-37, November.
    2. Huiqun Yu & Haoyi Sun & Yueze Li & Chunmei Xu & Chenkun Du, 2024. "Enhanced Short-Term Load Forecasting: Error-Weighted and Hybrid Model Approach," Energies, MDPI, vol. 17(21), pages 1-22, October.
    3. Sharma, Abhishek & Jain, Sachin Kumar, 2022. "A novel seasonal segmentation approach for day-ahead load forecasting," Energy, Elsevier, vol. 257(C).
    4. Grzegorz Dudek, 2021. "Short-Term Load Forecasting Using Neural Networks with Pattern Similarity-Based Error Weights," Energies, MDPI, vol. 14(11), pages 1-18, May.
    5. Zang, Haixiang & Xu, Ruiqi & Cheng, Lilin & Ding, Tao & Liu, Ling & Wei, Zhinong & Sun, Guoqiang, 2021. "Residential load forecasting based on LSTM fusing self-attention mechanism with pooling," Energy, Elsevier, vol. 229(C).
    6. Tavassoli-Hojati, Z. & Ghaderi, S.F. & Iranmanesh, H. & Hilber, P. & Shayesteh, E., 2020. "A self-partitioning local neuro fuzzy model for short-term load forecasting in smart grids," Energy, Elsevier, vol. 199(C).
    7. Thangjam, Aditya & Jaipuria, Sanjita & Dadabada, Pradeep Kumar, 2023. "Time-Varying approaches for Long-Term Electric Load Forecasting under economic shocks," Applied Energy, Elsevier, vol. 333(C).
    8. Mingping Liu & Xihao Sun & Qingnian Wang & Suhui Deng, 2022. "Short-Term Load Forecasting Using EMD with Feature Selection and TCN-Based Deep Learning Model," Energies, MDPI, vol. 15(19), pages 1-22, September.
    9. Bashiri Behmiri, Niaz & Fezzi, Carlo & Ravazzolo, Francesco, 2023. "Incorporating air temperature into mid-term electricity load forecasting models using time-series regressions and neural networks," Energy, Elsevier, vol. 278(C).
    10. Jorge-Eusebio Velasco-López & Ramón-Alberto Carrasco & Jesús Serrano-Guerrero & Francisco Chiclana, 2024. "Profiling Social Sentiment in Times of Health Emergencies with Information from Social Networks and Official Statistics," Mathematics, MDPI, vol. 12(6), pages 1-23, March.
    11. Monika Zimmermann & Florian Ziel, 2024. "Efficient mid-term forecasting of hourly electricity load using generalized additive models," Papers 2405.17070, arXiv.org.
    12. Fang, Ping & Fu, Wenlong & Wang, Kai & Xiong, Dongzhen & Zhang, Kai, 2022. "A compositive architecture coupling outlier correction, EWT, nonlinear Volterra multi-model fusion with multi-objective optimization for short-term wind speed forecasting," Applied Energy, Elsevier, vol. 307(C).
    13. Fadaki, Masih & Asadikia, Atie, 2024. "Augmenting Monte Carlo Tree Search for managing service level agreements," International Journal of Production Economics, Elsevier, vol. 271(C).
    14. Michael Vössing & Niklas Kühl & Matteo Lind & Gerhard Satzger, 2022. "Designing Transparency for Effective Human-AI Collaboration," Information Systems Frontiers, Springer, vol. 24(3), pages 877-895, June.
    15. Zhang, Ning & Li, Zhiying & Zou, Xun & Quiring, Steven M., 2019. "Comparison of three short-term load forecast models in Southern California," Energy, Elsevier, vol. 189(C).
    16. Maghsoodi, Abtin Ijadi, 2023. "Cryptocurrency portfolio allocation using a novel hybrid and predictive big data decision support system," Omega, Elsevier, vol. 115(C).
    17. Miroslav Navratil & Andrea Kolkova, 2019. "Decomposition and Forecasting Time Series in the Business Economy Using Prophet Forecasting Model," Central European Business Review, Prague University of Economics and Business, vol. 2019(4), pages 26-39.
    18. Saima Akhtar & Sulman Shahzad & Asad Zaheer & Hafiz Sami Ullah & Heybet Kilic & Radomir Gono & Michał Jasiński & Zbigniew Leonowicz, 2023. "Short-Term Load Forecasting Models: A Review of Challenges, Progress, and the Road Ahead," Energies, MDPI, vol. 16(10), pages 1-29, May.
    19. Yang, Ting & Zhao, Liyuan & Li, Wei & Zomaya, Albert Y., 2021. "Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning," Energy, Elsevier, vol. 235(C).
    20. Omar Farhan Al-Hardanee & Hüseyin Demirel, 2024. "Hydropower Station Status Prediction Using RNN and LSTM Algorithms for Fault Detection," Energies, MDPI, vol. 17(22), pages 1-23, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:23:p:6131-:d:1537393. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.