IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i20p5211-d1502573.html
   My bibliography  Save this article

Multi-Agent Reinforcement Learning for Smart Community Energy Management

Author

Listed:
  • Patrick Wilk

    (Department of Electrical and Computer Engineering, Rowan University, Glassboro, NJ 08028, USA)

  • Ning Wang

    (Department of Computer Science, Rowan University, Glassboro, NJ 08028, USA)

  • Jie Li

    (Department of Electrical and Computer Engineering, Rowan University, Glassboro, NJ 08028, USA)

Abstract

This paper investigates a Local Strategy-Driven Multi-Agent Deep Deterministic Policy Gradient (LSD-MADDPG) method for demand-side energy management systems (EMS) in smart communities. LSD-MADDPG modifies the conventional MADDPG framework by limiting data sharing during centralized training to only discretized strategic information. During execution, it relies solely on local information, eliminating post-training data exchange. This approach addresses critical challenges commonly faced by EMS solutions serving dynamic, increasing-scale communities, such as communication delays, single-point failures, scalability, and nonstationary environments. By leveraging and sharing only strategic information among agents, LSD-MADDPG optimizes decision-making while enhancing training efficiency and safeguarding data privacy—a critical concern in the community EMS. The proposed LSD-MADDPG has proven to be capable of reducing energy costs and flattening the community demand curve by coordinating indoor temperature control and electric vehicle charging schedules across multiple buildings. Comparative case studies reveal that LSD-MADDPG excels in both cooperative and competitive settings by ensuring fair alignment between individual buildings’ energy management actions and community-wide goals, highlighting its potential for advancing future smart community energy management.

Suggested Citation

  • Patrick Wilk & Ning Wang & Jie Li, 2024. "Multi-Agent Reinforcement Learning for Smart Community Energy Management," Energies, MDPI, vol. 17(20), pages 1-21, October.
    • Handle: RePEc:gam:jeners:v:17:y:2024:i:20:p:5211-:d:1502573
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/20/5211/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/20/5211/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lu, Renzhi & Li, Yi-Chang & Li, Yuting & Jiang, Junhui & Ding, Yuemin, 2020. "Multi-agent deep reinforcement learning based demand response for discrete manufacturing systems energy management," Applied Energy, Elsevier, vol. 276(C).
    2. Zahra Foroozandeh & Sérgio Ramos & João Soares & Fernando Lezama & Zita Vale & António Gomes & Rodrigo L. Joench, 2020. "A Mixed Binary Linear Programming Model for Optimal Energy Management of Smart Buildings," Energies, MDPI, vol. 13(7), pages 1-16, April.
    3. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.
    4. Kumar, R. Seshu & Raghav, L. Phani & Raju, D. Koteswara & Singh, Arvind R., 2021. "Intelligent demand side management for optimal energy scheduling of grid connected microgrids," Applied Energy, Elsevier, vol. 285(C).
    5. Omu, Akomeno & Choudhary, Ruchi & Boies, Adam, 2013. "Distributed energy resource system optimisation using mixed integer linear programming," Energy Policy, Elsevier, vol. 61(C), pages 249-266.
    6. Arroyo, Javier & Manna, Carlo & Spiessens, Fred & Helsen, Lieve, 2022. "Reinforced model predictive control (RL-MPC) for building energy management," Applied Energy, Elsevier, vol. 309(C).
    7. Babonneau, Frédéric & Caramanis, Michael & Haurie, Alain, 2016. "A linear programming model for power distribution with demand response and variable renewable energy," Applied Energy, Elsevier, vol. 181(C), pages 83-95.
    8. Fernandez, Edstan & Hossain, M.J. & Nizami, M.S.H., 2018. "Game-theoretic approach to demand-side energy management for a smart neighbourhood in Sydney incorporating renewable resources," Applied Energy, Elsevier, vol. 232(C), pages 245-257.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ajagekar, Akshay & Decardi-Nelson, Benjamin & You, Fengqi, 2024. "Energy management for demand response in networked greenhouses with multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 355(C).
    2. Alabi, Tobi Michael & Aghimien, Emmanuel I. & Agbajor, Favour D. & Yang, Zaiyue & Lu, Lin & Adeoye, Adebusola R. & Gopaluni, Bhushan, 2022. "A review on the integrated optimization techniques and machine learning approaches for modeling, prediction, and decision making on integrated energy systems," Renewable Energy, Elsevier, vol. 194(C), pages 822-849.
    3. Panagiotis Michailidis & Iakovos Michailidis & Socratis Gkelios & Elias Kosmatopoulos, 2024. "Artificial Neural Network Applications for Energy Management in Buildings: Current Trends and Future Directions," Energies, MDPI, vol. 17(3), pages 1-47, January.
    4. Panagiotis Michailidis & Iakovos Michailidis & Dimitrios Vamvakas & Elias Kosmatopoulos, 2023. "Model-Free HVAC Control in Buildings: A Review," Energies, MDPI, vol. 16(20), pages 1-45, October.
    5. Sajjad Ali & Imran Khan & Sadaqat Jan & Ghulam Hafeez, 2021. "An Optimization Based Power Usage Scheduling Strategy Using Photovoltaic-Battery System for Demand-Side Management in Smart Grid," Energies, MDPI, vol. 14(8), pages 1-29, April.
    6. Waibel, Christoph & Evins, Ralph & Carmeliet, Jan, 2019. "Co-simulation and optimization of building geometry and multi-energy systems: Interdependencies in energy supply, energy demand and solar potentials," Applied Energy, Elsevier, vol. 242(C), pages 1661-1682.
    7. Clarke, Fiona & Dorneanu, Bogdan & Mechleri, Evgenia & Arellano-Garcia, Harvey, 2021. "Optimal design of heating and cooling pipeline networks for residential distributed energy resource systems," Energy, Elsevier, vol. 235(C).
    8. Zhao, Jing & Yang, Zilan & Shi, Linyu & Liu, Dehan & Li, Haonan & Mi, Yumiao & Wang, Hongbin & Feng, Meili & Hutagaol, Timothy Joseph, 2024. "Photovoltaic capacity dynamic tracking model predictive control strategy of air-conditioning systems with consideration of flexible loads," Applied Energy, Elsevier, vol. 356(C).
    9. Angel L. Cedeño & Reinier López Ahuar & José Rojas & Gonzalo Carvajal & César Silva & Juan C. Agüero, 2022. "Model Predictive Control for Photovoltaic Plants with Non-Ideal Energy Storage Using Mixed Integer Linear Programming," Energies, MDPI, vol. 15(17), pages 1-21, September.
    10. Shahid Nawaz Khan & Syed Ali Abbas Kazmi & Abdullah Altamimi & Zafar A. Khan & Mohammed A. Alghassab, 2022. "Smart Distribution Mechanisms—Part I: From the Perspectives of Planning," Sustainability, MDPI, vol. 14(23), pages 1-109, December.
    11. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    12. Rocha, Paula & Kaut, Michal & Siddiqui, Afzal S., 2016. "Energy-efficient building retrofits: An assessment of regulatory proposals under uncertainty," Energy, Elsevier, vol. 101(C), pages 278-287.
    13. Prasanna, Ashreeta & Dorer, Viktor & Vetterli, Nadège, 2017. "Optimisation of a district energy system with a low temperature network," Energy, Elsevier, vol. 137(C), pages 632-648.
    14. Aliakbari Sani, Sajad & Bahn, Olivier & Delage, Erick, 2022. "Affine decision rule approximation to address demand response uncertainty in smart Grids’ capacity planning," European Journal of Operational Research, Elsevier, vol. 303(1), pages 438-455.
    15. Cindy Paola Guzman & Nataly Bañol Arias & John Fredy Franco & Marcos J. Rider & Rubén Romero, 2020. "Enhanced Coordination Strategy for an Aggregator of Distributed Energy Resources Participating in the Day-Ahead Reserve Market," Energies, MDPI, vol. 13(8), pages 1-22, April.
    16. Zheng, Yingying & Jenkins, Bryan M. & Kornbluth, Kurt & Træholt, Chresten, 2018. "Optimization under uncertainty of a biomass-integrated renewable energy microgrid with energy storage," Renewable Energy, Elsevier, vol. 123(C), pages 204-217.
    17. Woltmann, Stefan & Kittel, Julia, 2022. "Development and implementation of multi-agent systems for demand response aggregators in an industrial context," Applied Energy, Elsevier, vol. 314(C).
    18. Charalampos Rafail Lazaridis & Iakovos Michailidis & Georgios Karatzinis & Panagiotis Michailidis & Elias Kosmatopoulos, 2024. "Evaluating Reinforcement Learning Algorithms in Residential Energy Saving and Comfort Management," Energies, MDPI, vol. 17(3), pages 1-33, January.
    19. Nweye, Kingsley & Sankaranarayanan, Siva & Nagy, Zoltan, 2023. "MERLIN: Multi-agent offline and transfer learning for occupant-centric operation of grid-interactive communities," Applied Energy, Elsevier, vol. 346(C).
    20. Nizami, M.S.H. & Hossain, M.J. & Amin, B.M. Ruhul & Fernandez, Edstan, 2020. "A residential energy management system with bi-level optimization-based bidding strategy for day-ahead bi-directional electricity trading," Applied Energy, Elsevier, vol. 261(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:20:p:5211-:d:1502573. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.