IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i24p6420-d1548214.html
   My bibliography  Save this article

Applications of Deep Reinforcement Learning for Home Energy Management Systems: A Review

Author

Listed:
  • Dominik Latoń

    (Department of Power Electronics and Energy Control Systems, Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, AGH University of Krakow, 30-059 Krakow, Poland)

  • Jakub Grela

    (Department of Power Electronics and Energy Control Systems, Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, AGH University of Krakow, 30-059 Krakow, Poland)

  • Andrzej Ożadowicz

    (Department of Power Electronics and Energy Control Systems, Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, AGH University of Krakow, 30-059 Krakow, Poland)

Abstract

In the context of the increasing integration of renewable energy sources (RES) and smart devices in domestic applications, the implementation of Home Energy Management Systems (HEMS) is becoming a pivotal factor in optimizing energy usage and reducing costs. This review examines the role of reinforcement learning (RL) in the advancement of HEMS, presenting it as a powerful tool for the adaptive management of complex, real-time energy demands. This review is notable for its comprehensive examination of the applications of RL-based methods and tools in HEMS, which encompasses demand response, load scheduling, and renewable energy integration. Furthermore, the integration of RL within distributed automation and Internet of Things (IoT) frameworks is emphasized in the review as a means of facilitating autonomous, data-driven control. Despite the considerable potential of this approach, the authors identify a number of challenges that require further investigation, including the need for robust data security and scalable solutions. It is recommended that future research place greater emphasis on real applications and case studies, with the objective of bridging the gap between theoretical models and practical implementations. The objective is to achieve resilient and secure energy management in residential and prosumer buildings, particularly within local microgrids.

Suggested Citation

  • Dominik Latoń & Jakub Grela & Andrzej Ożadowicz, 2024. "Applications of Deep Reinforcement Learning for Home Energy Management Systems: A Review," Energies, MDPI, vol. 17(24), pages 1-30, December.
    • Handle: RePEc:gam:jeners:v:17:y:2024:i:24:p:6420-:d:1548214
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/24/6420/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/24/6420/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Anwar, Ghazanfar Ali & Zhang, Xiaoge, 2024. "Deep reinforcement learning for intelligent risk optimization of buildings under hazard," Reliability Engineering and System Safety, Elsevier, vol. 247(C).
    2. Xu, Gaoyuan & Shi, Jian & Wu, Jiaman & Lu, Chenbei & Wu, Chenye & Wang, Dan & Han, Zhu, 2024. "An optimal solutions-guided deep reinforcement learning approach for online energy storage control," Applied Energy, Elsevier, vol. 361(C).
    3. Mohammad Mansour & Amal Gamal & Ahmed I. Ahmed & Lobna A. Said & Abdelmoniem Elbaz & Norbert Herencsar & Ahmed Soltan, 2023. "Internet of Things: A Comprehensive Overview on Protocols, Architectures, Technologies, Simulation Tools, and Future Directions," Energies, MDPI, vol. 16(8), pages 1-39, April.
    4. Langer, Lissy & Volling, Thomas, 2022. "A reinforcement learning approach to home energy management for modulating heat pumps and photovoltaic systems," Applied Energy, Elsevier, vol. 327(C).
    5. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.
    6. Wang, Zixuan & Xiao, Fu & Ran, Yi & Li, Yanxue & Xu, Yang, 2024. "Scalable energy management approach of residential hybrid energy system using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 367(C).
    7. Wang, Zhe & Hong, Tianzhen, 2020. "Reinforcement learning for building controls: The opportunities and challenges," Applied Energy, Elsevier, vol. 269(C).
    8. Michał Markiewicz & Aleksander Skała & Jakub Grela & Szymon Janusz & Tadeusz Stasiak & Dominik Latoń & Andrzej Bielecki & Katarzyna Bańczyk, 2023. "The Architecture for Testing Central Heating Control Algorithms with Feedback from Wireless Temperature Sensors," Energies, MDPI, vol. 16(14), pages 1-15, July.
    9. Clara Ceccolini & Roozbeh Sangi, 2022. "Benchmarking Approaches for Assessing the Performance of Building Control Strategies: A Review," Energies, MDPI, vol. 15(4), pages 1-30, February.
    10. Jendoubi, Imen & Bouffard, François, 2023. "Multi-agent hierarchical reinforcement learning for energy management," Applied Energy, Elsevier, vol. 332(C).
    11. Salvatore Favuzza & Mariano Giuseppe Ippolito & Fabio Massaro & Rossano Musca & Eleonora Riva Sanseverino & Giuseppe Schillaci & Gaetano Zizzo, 2018. "Building Automation and Control Systems and Electrical Distribution Grids: A Study on the Effects of Loads Control Logics on Power Losses and Peaks," Energies, MDPI, vol. 11(3), pages 1-15, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liu, Mingzhe & Guo, Mingyue & Fu, Yangyang & O’Neill, Zheng & Gao, Yuan, 2024. "Expert-guided imitation learning for energy management: Evaluating GAIL’s performance in building control applications," Applied Energy, Elsevier, vol. 372(C).
    2. Wu, Haochi & Qiu, Dawei & Zhang, Liyu & Sun, Mingyang, 2024. "Adaptive multi-agent reinforcement learning for flexible resource management in a virtual power plant with dynamic participating multi-energy buildings," Applied Energy, Elsevier, vol. 374(C).
    3. Michael Bachseitz & Muhammad Sheryar & David Schmitt & Thorsten Summ & Christoph Trinkl & Wilfried Zörner, 2024. "PV-Optimized Heat Pump Control in Multi-Family Buildings Using a Reinforcement Learning Approach," Energies, MDPI, vol. 17(8), pages 1-16, April.
    4. Schmitz, Simon & Brucke, Karoline & Kasturi, Pranay & Ansari, Esmail & Klement, Peter, 2024. "Forecast-based and data-driven reinforcement learning for residential heat pump operation," Applied Energy, Elsevier, vol. 371(C).
    5. Panagiotis Michailidis & Iakovos Michailidis & Dimitrios Vamvakas & Elias Kosmatopoulos, 2023. "Model-Free HVAC Control in Buildings: A Review," Energies, MDPI, vol. 16(20), pages 1-45, October.
    6. Yassine Chemingui & Adel Gastli & Omar Ellabban, 2020. "Reinforcement Learning-Based School Energy Management System," Energies, MDPI, vol. 13(23), pages 1-21, December.
    7. Evelina Di Corso & Tania Cerquitelli & Daniele Apiletti, 2018. "METATECH: METeorological Data Analysis for Thermal Energy CHaracterization by Means of Self-Learning Transparent Models," Energies, MDPI, vol. 11(6), pages 1-24, May.
    8. Gokhale, Gargya & Claessens, Bert & Develder, Chris, 2022. "Physics informed neural networks for control oriented thermal modeling of buildings," Applied Energy, Elsevier, vol. 314(C).
    9. Davide Coraci & Silvio Brandi & Marco Savino Piscitelli & Alfonso Capozzoli, 2021. "Online Implementation of a Soft Actor-Critic Agent to Enhance Indoor Temperature Control and Energy Efficiency in Buildings," Energies, MDPI, vol. 14(4), pages 1-26, February.
    10. Langer, Lissy & Volling, Thomas, 2022. "A reinforcement learning approach to home energy management for modulating heat pumps and photovoltaic systems," Applied Energy, Elsevier, vol. 327(C).
    11. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    12. Pinto, Giuseppe & Piscitelli, Marco Savino & Vázquez-Canteli, José Ramón & Nagy, Zoltán & Capozzoli, Alfonso, 2021. "Coordinated energy management for a cluster of buildings through deep reinforcement learning," Energy, Elsevier, vol. 229(C).
    13. Li, Yanxue & Wang, Zixuan & Xu, Wenya & Gao, Weijun & Xu, Yang & Xiao, Fu, 2023. "Modeling and energy dynamic control for a ZEH via hybrid model-based deep reinforcement learning," Energy, Elsevier, vol. 277(C).
    14. Dai, Mingkun & Li, Hangxin & Wang, Shengwei, 2023. "A reinforcement learning-enabled iterative learning control strategy of air-conditioning systems for building energy saving by shortening the morning start period," Applied Energy, Elsevier, vol. 334(C).
    15. Charalampos Rafail Lazaridis & Iakovos Michailidis & Georgios Karatzinis & Panagiotis Michailidis & Elias Kosmatopoulos, 2024. "Evaluating Reinforcement Learning Algorithms in Residential Energy Saving and Comfort Management," Energies, MDPI, vol. 17(3), pages 1-33, January.
    16. Nweye, Kingsley & Sankaranarayanan, Siva & Nagy, Zoltan, 2023. "MERLIN: Multi-agent offline and transfer learning for occupant-centric operation of grid-interactive communities," Applied Energy, Elsevier, vol. 346(C).
    17. Zhang, Yiwen & Lin, Rui & Mei, Zhen & Lyu, Minghao & Jiang, Huaiguang & Xue, Ying & Zhang, Jun & Gao, David Wenzhong, 2024. "Interior-point policy optimization based multi-agent deep reinforcement learning method for secure home energy management under various uncertainties," Applied Energy, Elsevier, vol. 376(PA).
    18. Alexandra Brintrup & George Baryannis & Ashutosh Tiwari & Svetan Ratchev & Giovanna Martinez-Arellano & Jatinder Singh, 2023. "Trustworthy, responsible, ethical AI in manufacturing and supply chains: synthesis and emerging research questions," Papers 2305.11581, arXiv.org.
    19. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    20. Tomasz Cholewa & Agnieszka Malec & Alicja Siuta-Olcha & Andrzej Smolarz & Piotr Muryjas & Piotr Wolszczak & Łukasz Guz & Marzenna R. Dudzińska & Krystian Łygas, 2021. "On the Influence of Solar Radiation on Heat Delivered to Buildings for Heating," Energies, MDPI, vol. 14(4), pages 1-16, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:24:p:6420-:d:1548214. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.