IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i20p5211-d1502573.html
   My bibliography  Save this article

Multi-Agent Reinforcement Learning for Smart Community Energy Management

Author

Listed:
  • Patrick Wilk

    (Department of Electrical and Computer Engineering, Rowan University, Glassboro, NJ 08028, USA)

  • Ning Wang

    (Department of Computer Science, Rowan University, Glassboro, NJ 08028, USA)

  • Jie Li

    (Department of Electrical and Computer Engineering, Rowan University, Glassboro, NJ 08028, USA)

Abstract

This paper investigates a Local Strategy-Driven Multi-Agent Deep Deterministic Policy Gradient (LSD-MADDPG) method for demand-side energy management systems (EMS) in smart communities. LSD-MADDPG modifies the conventional MADDPG framework by limiting data sharing during centralized training to only discretized strategic information. During execution, it relies solely on local information, eliminating post-training data exchange. This approach addresses critical challenges commonly faced by EMS solutions serving dynamic, increasing-scale communities, such as communication delays, single-point failures, scalability, and nonstationary environments. By leveraging and sharing only strategic information among agents, LSD-MADDPG optimizes decision-making while enhancing training efficiency and safeguarding data privacy—a critical concern in the community EMS. The proposed LSD-MADDPG has proven to be capable of reducing energy costs and flattening the community demand curve by coordinating indoor temperature control and electric vehicle charging schedules across multiple buildings. Comparative case studies reveal that LSD-MADDPG excels in both cooperative and competitive settings by ensuring fair alignment between individual buildings’ energy management actions and community-wide goals, highlighting its potential for advancing future smart community energy management.

Suggested Citation

  • Patrick Wilk & Ning Wang & Jie Li, 2024. "Multi-Agent Reinforcement Learning for Smart Community Energy Management," Energies, MDPI, vol. 17(20), pages 1-21, October.
    • Handle: RePEc:gam:jeners:v:17:y:2024:i:20:p:5211-:d:1502573
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/20/5211/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/20/5211/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lu, Renzhi & Li, Yi-Chang & Li, Yuting & Jiang, Junhui & Ding, Yuemin, 2020. "Multi-agent deep reinforcement learning based demand response for discrete manufacturing systems energy management," Applied Energy, Elsevier, vol. 276(C).
    2. Zahra Foroozandeh & Sérgio Ramos & João Soares & Fernando Lezama & Zita Vale & António Gomes & Rodrigo L. Joench, 2020. "A Mixed Binary Linear Programming Model for Optimal Energy Management of Smart Buildings," Energies, MDPI, vol. 13(7), pages 1-16, April.
    3. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.
    4. Kumar, R. Seshu & Raghav, L. Phani & Raju, D. Koteswara & Singh, Arvind R., 2021. "Intelligent demand side management for optimal energy scheduling of grid connected microgrids," Applied Energy, Elsevier, vol. 285(C).
    5. Omu, Akomeno & Choudhary, Ruchi & Boies, Adam, 2013. "Distributed energy resource system optimisation using mixed integer linear programming," Energy Policy, Elsevier, vol. 61(C), pages 249-266.
    6. Arroyo, Javier & Manna, Carlo & Spiessens, Fred & Helsen, Lieve, 2022. "Reinforced model predictive control (RL-MPC) for building energy management," Applied Energy, Elsevier, vol. 309(C).
    7. Babonneau, Frédéric & Caramanis, Michael & Haurie, Alain, 2016. "A linear programming model for power distribution with demand response and variable renewable energy," Applied Energy, Elsevier, vol. 181(C), pages 83-95.
    8. Fernandez, Edstan & Hossain, M.J. & Nizami, M.S.H., 2018. "Game-theoretic approach to demand-side energy management for a smart neighbourhood in Sydney incorporating renewable resources," Applied Energy, Elsevier, vol. 232(C), pages 245-257.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Panagiotis Michailidis & Iakovos Michailidis & Socratis Gkelios & Elias Kosmatopoulos, 2024. "Artificial Neural Network Applications for Energy Management in Buildings: Current Trends and Future Directions," Energies, MDPI, vol. 17(3), pages 1-47, January.
    2. Ajagekar, Akshay & Decardi-Nelson, Benjamin & You, Fengqi, 2024. "Energy management for demand response in networked greenhouses with multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 355(C).
    3. Alabi, Tobi Michael & Aghimien, Emmanuel I. & Agbajor, Favour D. & Yang, Zaiyue & Lu, Lin & Adeoye, Adebusola R. & Gopaluni, Bhushan, 2022. "A review on the integrated optimization techniques and machine learning approaches for modeling, prediction, and decision making on integrated energy systems," Renewable Energy, Elsevier, vol. 194(C), pages 822-849.
    4. Panagiotis Michailidis & Iakovos Michailidis & Dimitrios Vamvakas & Elias Kosmatopoulos, 2023. "Model-Free HVAC Control in Buildings: A Review," Energies, MDPI, vol. 16(20), pages 1-45, October.
    5. Waibel, Christoph & Evins, Ralph & Carmeliet, Jan, 2019. "Co-simulation and optimization of building geometry and multi-energy systems: Interdependencies in energy supply, energy demand and solar potentials," Applied Energy, Elsevier, vol. 242(C), pages 1661-1682.
    6. Aliakbari Sani, Sajad & Bahn, Olivier & Delage, Erick, 2022. "Affine decision rule approximation to address demand response uncertainty in smart Grids’ capacity planning," European Journal of Operational Research, Elsevier, vol. 303(1), pages 438-455.
    7. Cindy Paola Guzman & Nataly Bañol Arias & John Fredy Franco & Marcos J. Rider & Rubén Romero, 2020. "Enhanced Coordination Strategy for an Aggregator of Distributed Energy Resources Participating in the Day-Ahead Reserve Market," Energies, MDPI, vol. 13(8), pages 1-22, April.
    8. Woltmann, Stefan & Kittel, Julia, 2022. "Development and implementation of multi-agent systems for demand response aggregators in an industrial context," Applied Energy, Elsevier, vol. 314(C).
    9. Nweye, Kingsley & Sankaranarayanan, Siva & Nagy, Zoltan, 2023. "MERLIN: Multi-agent offline and transfer learning for occupant-centric operation of grid-interactive communities," Applied Energy, Elsevier, vol. 346(C).
    10. Nizami, M.S.H. & Hossain, M.J. & Amin, B.M. Ruhul & Fernandez, Edstan, 2020. "A residential energy management system with bi-level optimization-based bidding strategy for day-ahead bi-directional electricity trading," Applied Energy, Elsevier, vol. 261(C).
    11. Nguyen, Hai-Tra & Safder, Usman & Loy-Benitez, Jorge & Yoo, ChangKyoo, 2022. "Optimal demand side management scheduling-based bidirectional regulation of energy distribution network for multi-residential demand response with self-produced renewable energy," Applied Energy, Elsevier, vol. 322(C).
    12. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    13. He, Wangli & Li, Chengyuan & Cai, Chenhao & Qing, Xiangyun & Du, Wenli, 2024. "Suppressing active power fluctuations at PCC in grid-connection microgrids via multiple BESSs: A collaborative multi-agent reinforcement learning approach," Applied Energy, Elsevier, vol. 373(C).
    14. Lu, Renzhi & Bai, Ruichang & Ding, Yuemin & Wei, Min & Jiang, Junhui & Sun, Mingyang & Xiao, Feng & Zhang, Hai-Tao, 2021. "A hybrid deep learning-based online energy management scheme for industrial microgrid," Applied Energy, Elsevier, vol. 304(C).
    15. Chenhao Sun & Xiwei Jiang & Zhiwei Jia & Kun Yu & Sheng Xiang & Jianhong Su, 2023. "An Optimization Ensemble for Integrated Energy System Configuration Strategy Incorporating Demand–Supply Coordination," Sustainability, MDPI, vol. 15(21), pages 1-17, October.
    16. Huang, Yan & Ju, Yuntao & Ma, Kang & Short, Michael & Chen, Tao & Zhang, Ruosi & Lin, Yi, 2022. "Three-phase optimal power flow for networked microgrids based on semidefinite programming convex relaxation," Applied Energy, Elsevier, vol. 305(C).
    17. Cai, Qiran & Xu, Qingyang & Qing, Jing & Shi, Gang & Liang, Qiao-Mei, 2022. "Promoting wind and photovoltaics renewable energy integration through demand response: Dynamic pricing mechanism design and economic analysis for smart residential communities," Energy, Elsevier, vol. 261(PB).
    18. Chao-Chung Hsu & Bi-Hai Jiang & Chun-Cheng Lin, 2023. "A Survey on Recent Applications of Artificial Intelligence and Optimization for Smart Grids in Smart Manufacturing," Energies, MDPI, vol. 16(22), pages 1-15, November.
    19. Fu, Yangyang & Xu, Shichao & Zhu, Qi & O’Neill, Zheng & Adetola, Veronica, 2023. "How good are learning-based control v.s. model-based control for load shifting? Investigations on a single zone building energy system," Energy, Elsevier, vol. 273(C).
    20. Marquant, Julien F. & Evins, Ralph & Bollinger, L. Andrew & Carmeliet, Jan, 2017. "A holarchic approach for multi-scale distributed energy system optimisation," Applied Energy, Elsevier, vol. 208(C), pages 935-953.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:20:p:5211-:d:1502573. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.