IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i19p7047-d924888.html
   My bibliography  Save this article

Secondary Voltage Collaborative Control of Distributed Energy System via Multi-Agent Reinforcement Learning

Author

Listed:
  • Tianhao Wang

    (Electric Power Research Institute, State Grid Tianjin Electric Power Company, No. 8, Haitai Huake 4th Road, Huayuan Industrial Zone, Binhai High Tech Zone, Tianjin 300384, China)

  • Shiqian Ma

    (Electric Power Research Institute, State Grid Tianjin Electric Power Company, No. 8, Haitai Huake 4th Road, Huayuan Industrial Zone, Binhai High Tech Zone, Tianjin 300384, China)

  • Na Xu

    (Tianjin University, No. 92, Weijin Road, Nankai District, Tianjin 300072, China)

  • Tianchun Xiang

    (State Grid Tianjin Electric Power Company, No. 39 Wujing, Guangfu Street, Hebei District, Tianjin 300010, China)

  • Xiaoyun Han

    (Tianjin University, No. 92, Weijin Road, Nankai District, Tianjin 300072, China)

  • Chaoxu Mu

    (Tianjin University, No. 92, Weijin Road, Nankai District, Tianjin 300072, China)

  • Yao Jin

    (State Grid Tianjin Electric Power Company, No. 39 Wujing, Guangfu Street, Hebei District, Tianjin 300010, China)

Abstract

In this paper, a new voltage cooperative control strategy for a distributed power generation system is proposed based on the multi-agent advantage actor-critic (MA2C) algorithm, which realizes flexible management and effective control of distributed energy. The attentional actor-critic message processor (AACMP) is extended into the MA2C method to select the important messages from all communication messages adaptively and process important messages efficiently. The cooperative control strategy trained by centralized training and decentralized execution frame will take over the responsibility of the secondary control level for voltage restoration in a distributed manner. The introduction of the attention mechanism reduces the amount of information exchanged and the requirements of the communication network. Finally, a distributed system with six energy nodes is used to verify the effectiveness of the proposed control strategy.

Suggested Citation

  • Tianhao Wang & Shiqian Ma & Na Xu & Tianchun Xiang & Xiaoyun Han & Chaoxu Mu & Yao Jin, 2022. "Secondary Voltage Collaborative Control of Distributed Energy System via Multi-Agent Reinforcement Learning," Energies, MDPI, vol. 15(19), pages 1-12, September.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:19:p:7047-:d:924888
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/19/7047/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/19/7047/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ardi Tampuu & Tambet Matiisen & Dorian Kodelja & Ilya Kuzovkin & Kristjan Korjus & Juhan Aru & Jaan Aru & Raul Vicente, 2017. "Multiagent cooperation and competition with deep reinforcement learning," PLOS ONE, Public Library of Science, vol. 12(4), pages 1-15, April.
    2. Ju, Liwei & Zhang, Qi & Tan, Zhongfu & Wang, Wei & Xin, He & Zhang, Zehao, 2018. "Multi-agent-system-based coupling control optimization model for micro-grid group intelligent scheduling considering autonomy-cooperative operation strategy," Energy, Elsevier, vol. 157(C), pages 1035-1052.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jihed Hmad & Azeddine Houari & Allal El Moubarek Bouzid & Abdelhakim Saim & Hafedh Trabelsi, 2023. "A Review on Mode Transition Strategies between Grid-Connected and Standalone Operation of Voltage Source Inverters-Based Microgrids," Energies, MDPI, vol. 16(13), pages 1-41, June.
    2. Li, Qiang & Gao, Mengkai & Lin, Houfei & Chen, Ziyu & Chen, Minyou, 2019. "MAS-based distributed control method for multi-microgrids with high-penetration renewable energy," Energy, Elsevier, vol. 171(C), pages 284-295.
    3. Jicheng Liu & Fangqiu Xu & Shuaishuai Lin & Hua Cai & Suli Yan, 2018. "A Multi-Agent-Based Optimization Model for Microgrid Operation Using Dynamic Guiding Chaotic Search Particle Swarm Optimization," Energies, MDPI, vol. 11(12), pages 1-22, November.
    4. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2019. "Algorithmic Pricing What Implications for Competition Policy?," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 55(1), pages 155-171, August.
    5. Harrold, Daniel J.B. & Cao, Jun & Fan, Zhong, 2022. "Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 318(C).
    6. Wang, Xuekai & D’Ariano, Andrea & Su, Shuai & Tang, Tao, 2023. "Cooperative train control during the power supply shortage in metro system: A multi-agent reinforcement learning approach," Transportation Research Part B: Methodological, Elsevier, vol. 170(C), pages 244-278.
    7. Wagner, Lukas Peter & Reinpold, Lasse Matthias & Kilthau, Maximilian & Fay, Alexander, 2023. "A systematic review of modeling approaches for flexible energy resources," Renewable and Sustainable Energy Reviews, Elsevier, vol. 184(C).
    8. Se-Heon Lim & Sung-Guk Yoon, 2022. "Dynamic DNR and Solar PV Smart Inverter Control Scheme Using Heterogeneous Multi-Agent Deep Reinforcement Learning," Energies, MDPI, vol. 15(23), pages 1-18, December.
    9. Young Joon Park & Yoon Sang Cho & Seoung Bum Kim, 2019. "Multi-agent reinforcement learning with approximate model learning for competitive games," PLOS ONE, Public Library of Science, vol. 14(9), pages 1-20, September.
    10. Romain Mannini & Julien Eynard & Stéphane Grieu, 2022. "A Survey of Recent Advances in the Smart Management of Microgrids and Networked Microgrids," Energies, MDPI, vol. 15(19), pages 1-37, September.
    11. Li, Xingyu & Epureanu, Bogdan I., 2020. "AI-based competition of autonomous vehicle fleets with application to fleet modularity," European Journal of Operational Research, Elsevier, vol. 287(3), pages 856-874.
    12. Lee, Hyun-Rok & Lee, Taesik, 2021. "Multi-agent reinforcement learning algorithm to solve a partially-observable multi-agent problem in disaster response," European Journal of Operational Research, Elsevier, vol. 291(1), pages 296-308.
    13. Christoph Aymanns & Jakob Foerster & Co-Pierre Georg & Matthias Weber, 2022. "Fake News in Social Networks," Swiss Finance Institute Research Paper Series 22-58, Swiss Finance Institute.
    14. Ju, Liwei & Zhao, Rui & Tan, Qinliang & Lu, Yan & Tan, Qingkun & Wang, Wei, 2019. "A multi-objective robust scheduling model and solution algorithm for a novel virtual power plant connected with power-to-gas and gas storage tank considering uncertainty and demand response," Applied Energy, Elsevier, vol. 250(C), pages 1336-1355.
    15. Qingyan Li & Tao Lin & Qianyi Yu & Hui Du & Jun Li & Xiyue Fu, 2023. "Review of Deep Reinforcement Learning and Its Application in Modern Renewable Power System Control," Energies, MDPI, vol. 16(10), pages 1-23, May.
    16. Hu, Maomao & Xiao, Fu & Wang, Shengwei, 2021. "Neighborhood-level coordination and negotiation techniques for managing demand-side flexibility in residential microgrids," Renewable and Sustainable Energy Reviews, Elsevier, vol. 135(C).
    17. Marilleau, Nicolas & Lang, Christophe & Giraudoux, Patrick, 2018. "Coupling agent-based with equation-based models to study spatially explicit megapopulation dynamics," Ecological Modelling, Elsevier, vol. 384(C), pages 34-42.
    18. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    19. Pan, Yushu & Ju, Liwei & Yang, Shenbo & Guo, Xinyu & Tan, Zhongfu, 2024. "A multi-objective robust optimal dispatch and cost allocation model for microgrids-shared hybrid energy storage system considering flexible ramping capacity," Applied Energy, Elsevier, vol. 369(C).
    20. Yanbin Li & Yanting Sun & Junjie Zhang & Feng Zhang, 2022. "Optimal Microgrid System Operating Strategy Considering Variable Wind Power Outputs and the Cooperative Game among Subsystem Operators," Energies, MDPI, vol. 15(18), pages 1-20, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:19:p:7047-:d:924888. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.