IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i14p4993-d858451.html
   My bibliography  Save this article

Transformer-Based Model for Electrical Load Forecasting

Author

Listed:
  • Alexandra L’Heureux

    (Department of Electrical and Computer Engineering, The University of Western Ontario, London, ON N6A 5B9, Canada)

  • Katarina Grolinger

    (Department of Electrical and Computer Engineering, The University of Western Ontario, London, ON N6A 5B9, Canada)

  • Miriam A. M. Capretz

    (Department of Electrical and Computer Engineering, The University of Western Ontario, London, ON N6A 5B9, Canada)

Abstract

Amongst energy-related CO 2 emissions, electricity is the largest single contributor, and with the proliferation of electric vehicles and other developments, energy use is expected to increase. Load forecasting is essential for combating these issues as it balances demand and production and contributes to energy management. Current state-of-the-art solutions such as recurrent neural networks (RNNs) and sequence-to-sequence algorithms (Seq2Seq) are highly accurate, but most studies examine them on a single data stream. On the other hand, in natural language processing (NLP), transformer architecture has become the dominant technique, outperforming RNN and Seq2Seq algorithms while also allowing parallelization. Consequently, this paper proposes a transformer-based architecture for load forecasting by modifying the NLP transformer workflow, adding N-space transformation, and designing a novel technique for handling contextual features. Moreover, in contrast to most load forecasting studies, we evaluate the proposed solution on different data streams under various forecasting horizons and input window lengths in order to ensure result reproducibility. Results show that the proposed approach successfully handles time series with contextual data and outperforms the state-of-the-art Seq2Seq models.

Suggested Citation

  • Alexandra L’Heureux & Katarina Grolinger & Miriam A. M. Capretz, 2022. "Transformer-Based Model for Electrical Load Forecasting," Energies, MDPI, vol. 15(14), pages 1-23, July.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:14:p:4993-:d:858451
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/14/4993/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/14/4993/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zhao, Huiru & Guo, Sen, 2016. "An optimized grey model for annual power load forecasting," Energy, Elsevier, vol. 107(C), pages 272-286.
    2. Jessica Walther & Matthias Weigold, 2021. "A Systematic Review on Predicting and Forecasting the Electrical Energy Consumption in the Manufacturing Industry," Energies, MDPI, vol. 14(4), pages 1-24, February.
    3. Hu, Yi & Qu, Boyang & Wang, Jie & Liang, Jing & Wang, Yanli & Yu, Kunjie & Li, Yaxin & Qiao, Kangjia, 2021. "Short-term load forecasting using multimodal evolutionary algorithm and random vector functional link network based ensemble learning," Applied Energy, Elsevier, vol. 285(C).
    4. Fekri, Mohammad Navid & Patel, Harsh & Grolinger, Katarina & Sharma, Vinay, 2021. "Deep learning for load forecasting with smart meter data: Online Adaptive Recurrent Neural Network," Applied Energy, Elsevier, vol. 282(PA).
    5. Yang, Wangwang & Shi, Jing & Li, Shujian & Song, Zhaofang & Zhang, Zitong & Chen, Zexu, 2022. "A combined deep learning load forecasting model of single household resident user considering multi-time scale electricity consumption behavior," Applied Energy, Elsevier, vol. 307(C).
    6. Cai, Mengmeng & Pipattanasomporn, Manisa & Rahman, Saifur, 2019. "Day-ahead building-level load forecasts using deep learning vs. traditional time-series techniques," Applied Energy, Elsevier, vol. 236(C), pages 1078-1088.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jakubek, Dariusz & Ocłoń, Paweł & Nowak-Ocłoń, Marzena & Sułowicz, Maciej & Varbanov, Petar Sabev & Klemeš, Jiří Jaromír, 2023. "Mathematical modelling and model validation of the heat losses in district heating networks," Energy, Elsevier, vol. 267(C).
    2. Aoqi Xu & Man-Wen Tian & Behnam Firouzi & Khalid A. Alattas & Ardashir Mohammadzadeh & Ebrahim Ghaderpour, 2022. "A New Deep Learning Restricted Boltzmann Machine for Energy Consumption Forecasting," Sustainability, MDPI, vol. 14(16), pages 1-12, August.
    3. Raiden Skala & Mohamed Ahmed T. A. Elgalhud & Katarina Grolinger & Syed Mir, 2023. "Interval Load Forecasting for Individual Households in the Presence of Electric Vehicle Charging," Energies, MDPI, vol. 16(10), pages 1-21, May.
    4. Songjiang Li & Wenxin Zhang & Peng Wang, 2023. "TS2ARCformer: A Multi-Dimensional Time Series Forecasting Framework for Short-Term Load Prediction," Energies, MDPI, vol. 16(15), pages 1-22, August.
    5. Ze Wu & Feifan Pan & Dandan Li & Hao He & Tiancheng Zhang & Shuyun Yang, 2022. "Prediction of Photovoltaic Power by the Informer Model Based on Convolutional Neural Network," Sustainability, MDPI, vol. 14(20), pages 1-16, October.
    6. Pilotti, L. & Colombari, M. & Castelli, A.F. & Binotti, M. & Giaconia, A. & Martelli, E., 2023. "Simultaneous design and operational optimization of hybrid CSP-PV plants," Applied Energy, Elsevier, vol. 331(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eren, Yavuz & Küçükdemiral, İbrahim, 2024. "A comprehensive review on deep learning approaches for short-term load forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PB).
    2. Ping Ma & Shuhui Cui & Mingshuai Chen & Shengzhe Zhou & Kai Wang, 2023. "Review of Family-Level Short-Term Load Forecasting and Its Application in Household Energy Management System," Energies, MDPI, vol. 16(15), pages 1-17, August.
    3. Li, Yiyan & Zhang, Si & Hu, Rongxing & Lu, Ning, 2021. "A meta-learning based distribution system load forecasting model selection framework," Applied Energy, Elsevier, vol. 294(C).
    4. Diogo M. F. Izidio & Paulo S. G. de Mattos Neto & Luciano Barbosa & João F. L. de Oliveira & Manoel Henrique da Nóbrega Marinho & Guilherme Ferretti Rissi, 2021. "Evolutionary Hybrid System for Energy Consumption Forecasting for Smart Meters," Energies, MDPI, vol. 14(7), pages 1-19, March.
    5. Stefan Ungureanu & Vasile Topa & Andrei Cristinel Cziker, 2021. "Analysis for Non-Residential Short-Term Load Forecasting Using Machine Learning and Statistical Methods with Financial Impact on the Power Market," Energies, MDPI, vol. 14(21), pages 1-26, October.
    6. Jihoon Moon & Junhong Kim & Pilsung Kang & Eenjun Hwang, 2020. "Solving the Cold-Start Problem in Short-Term Load Forecasting Using Tree-Based Methods," Energies, MDPI, vol. 13(4), pages 1-37, February.
    7. Kong, Xiangyu & Li, Chuang & Wang, Chengshan & Zhang, Yusen & Zhang, Jian, 2020. "Short-term electrical load forecasting based on error correction using dynamic mode decomposition," Applied Energy, Elsevier, vol. 261(C).
    8. Sen Guo & Haoran Zhao & Huiru Zhao, 2017. "A New Hybrid Wind Power Forecaster Using the Beveridge-Nelson Decomposition Method and a Relevance Vector Machine Optimized by the Ant Lion Optimizer," Energies, MDPI, vol. 10(7), pages 1-20, July.
    9. Wang, Kejun & Qi, Xiaoxia & Liu, Hongda, 2019. "A comparison of day-ahead photovoltaic power forecasting models based on deep learning neural network," Applied Energy, Elsevier, vol. 251(C), pages 1-1.
    10. Türkoğlu, A. Selim & Erkmen, Burcu & Eren, Yavuz & Erdinç, Ozan & Küçükdemiral, İbrahim, 2024. "Integrated Approaches in Resilient Hierarchical Load Forecasting via TCN and Optimal Valley Filling Based Demand Response Application," Applied Energy, Elsevier, vol. 360(C).
    11. Ahmad, Tanveer & Chen, Huanxin, 2019. "Deep learning for multi-scale smart energy forecasting," Energy, Elsevier, vol. 175(C), pages 98-112.
    12. Fang, Lei & He, Bin, 2023. "A deep learning framework using multi-feature fusion recurrent neural networks for energy consumption forecasting," Applied Energy, Elsevier, vol. 348(C).
    13. Wang, Qiang & Li, Shuyu & Li, Rongrong & Ma, Minglu, 2018. "Forecasting U.S. shale gas monthly production using a hybrid ARIMA and metabolic nonlinear grey model," Energy, Elsevier, vol. 160(C), pages 378-387.
    14. Jonathan Berrisch & Micha{l} Narajewski & Florian Ziel, 2022. "High-Resolution Peak Demand Estimation Using Generalized Additive Models and Deep Neural Networks," Papers 2203.03342, arXiv.org, revised Nov 2022.
    15. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    16. Wei, Ziqing & Zhang, Tingwei & Yue, Bao & Ding, Yunxiao & Xiao, Ran & Wang, Ruzhu & Zhai, Xiaoqiang, 2021. "Prediction of residential district heating load based on machine learning: A case study," Energy, Elsevier, vol. 231(C).
    17. He, Yan & Zhang, Hongli & Dong, Yingchao & Wang, Cong & Ma, Ping, 2024. "Residential net load interval prediction based on stacking ensemble learning," Energy, Elsevier, vol. 296(C).
    18. Hyunsoo Kim & Jiseok Jeong & Changwan Kim, 2022. "Daily Peak-Electricity-Demand Forecasting Based on Residual Long Short-Term Network," Mathematics, MDPI, vol. 10(23), pages 1-17, November.
    19. Song, Wanqing & Cattani, Carlo & Chi, Chi-Hung, 2020. "Multifractional Brownian motion and quantum-behaved particle swarm optimization for short term power load forecasting: An integrated approach," Energy, Elsevier, vol. 194(C).
    20. Wang, Qiang & Jiang, Feng, 2019. "Integrating linear and nonlinear forecasting techniques based on grey theory and artificial intelligence to forecast shale gas monthly production in Pennsylvania and Texas of the United States," Energy, Elsevier, vol. 178(C), pages 781-803.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:14:p:4993-:d:858451. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.