IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v332y2023ics0306261922017676.html
   My bibliography  Save this article

Deep reinforcement learning based topology-aware voltage regulation of distribution networks with distributed energy storage

Author

Listed:
  • Xiang, Yue
  • Lu, Yu
  • Liu, Junyong

Abstract

Both the high penetration of clean energy with strong fluctuation and the complicated variable operation condition bring great challenges to the voltage regulation of the distribution network. To deal with the problem, a topology-aware voltage regulation multi-agent deep reinforcement learning (MADRL) algorithm is proposed. The distributed energy storages (DESs) are modeled as agents to regulate voltage autonomously in real-time, which could fast adapt to dynamic topological scenarios. Firstly, taking the minimization of voltage fluctuation and maximization of reserve capacity as the target, the optimal voltage regulation model is established. Secondly, a topology extraction method considering voltage sensitivity is proposed for dynamic topology clustering, and the obtained typical topology is added to the observation set of agents. Then, the optimal voltage regulation model is formulated to the decentralized partially observable Markov decision process (Dec-POMDP) framework, in which only local information is required for the agent during the test process to decision-making to realize the hierarchical and partitioned control of voltage. Finally, the multi-agent deep deterministic policy gradient (MADDPG) algorithm is used to solve the Dec-POMDP model. The feasibility and superiority of the proposed algorithm are verified and analyzed in the simulation under different scenarios.

Suggested Citation

  • Xiang, Yue & Lu, Yu & Liu, Junyong, 2023. "Deep reinforcement learning based topology-aware voltage regulation of distribution networks with distributed energy storage," Applied Energy, Elsevier, vol. 332(C).
    • Handle: RePEc:eee:appene:v:332:y:2023:i:c:s0306261922017676
    DOI: 10.1016/j.apenergy.2022.120510
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261922017676
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2022.120510?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ma, Wei & Wang, Wei & Chen, Zhe & Wu, Xuezhi & Hu, Ruonan & Tang, Fen & Zhang, Weige, 2021. "Voltage regulation methods for active distribution networks considering the reactive power optimization of substations," Applied Energy, Elsevier, vol. 284(C).
    2. Mehrjerdi, Hasan, 2019. "Simultaneous load leveling and voltage profile improvement in distribution networks by optimal battery storage planning," Energy, Elsevier, vol. 181(C), pages 916-926.
    3. Zhenming Li & Yunfeng Yan & Donglian Qi & Shuo Yan & Minghao Wang, 2022. "Distributed Voltage Optimization Control of BESS in AC Distribution Networks with High PV Penetration," Energies, MDPI, vol. 15(11), pages 1-14, June.
    4. Lai, Qiupin & Liu, Chengxi & Sun, Kai, 2021. "Vulnerability assessment for voltage stability based on solvability regions of decoupled power flow equations," Applied Energy, Elsevier, vol. 304(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhu, Xingxu & Hou, Xiangchen & Li, Junhui & Yan, Gangui & Li, Cuiping & Wang, Dongbo, 2023. "Distributed online prediction optimization algorithm for distributed energy resources considering the multi-periods optimal operation," Applied Energy, Elsevier, vol. 348(C).
    2. Zhao, Yincheng & Zhang, Guozhou & Hu, Weihao & Huang, Qi & Chen, Zhe & Blaabjerg, Frede, 2023. "Meta-learning based voltage control strategy for emergency faults of active distribution networks," Applied Energy, Elsevier, vol. 349(C).
    3. Jianxun Luo & Wei Zhang & Hui Wang & Wenmiao Wei & Jinpeng He, 2023. "Research on Data-Driven Optimal Scheduling of Power System," Energies, MDPI, vol. 16(6), pages 1-15, March.
    4. Chen, Yongdong & Liu, Youbo & Zhao, Junbo & Qiu, Gao & Yin, Hang & Li, Zhengbo, 2023. "Physical-assisted multi-agent graph reinforcement learning enabled fast voltage regulation for PV-rich active distribution network," Applied Energy, Elsevier, vol. 351(C).
    5. Rabea Jamil Mahfoud & Nizar Faisal Alkayem & Emmanuel Fernandez-Rodriguez & Yuan Zheng & Yonghui Sun & Shida Zhang & Yuquan Zhang, 2024. "Evolutionary Approach for DISCO Profit Maximization by Optimal Planning of Distributed Generators and Energy Storage Systems in Active Distribution Networks," Mathematics, MDPI, vol. 12(2), pages 1-33, January.
    6. Escobar, Eros D. & Betancur, Daniel & Manrique, Tatiana & Isaac, Idi A., 2023. "Model predictive real-time architecture for secondary voltage control of microgrids," Applied Energy, Elsevier, vol. 345(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. A.S. Jameel Hassan & Umar Marikkar & G.W. Kasun Prabhath & Aranee Balachandran & W.G. Chaminda Bandara & Parakrama B. Ekanayake & Roshan I. Godaliyadda & Janaka B. Ekanayake, 2021. "A Sensitivity Matrix Approach Using Two-Stage Optimization for Voltage Regulation of LV Networks with High PV Penetration," Energies, MDPI, vol. 14(20), pages 1-24, October.
    2. Lu, Yu & Xiang, Yue & Huang, Yuan & Yu, Bin & Weng, Liguo & Liu, Junyong, 2023. "Deep reinforcement learning based optimal scheduling of active distribution system considering distributed generation, energy storage and flexible load," Energy, Elsevier, vol. 271(C).
    3. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).
    4. Zhu, Xingxu & Hou, Xiangchen & Li, Junhui & Yan, Gangui & Li, Cuiping & Wang, Dongbo, 2023. "Distributed online prediction optimization algorithm for distributed energy resources considering the multi-periods optimal operation," Applied Energy, Elsevier, vol. 348(C).
    5. Yin, Linfei & He, Xiaoyu, 2023. "Artificial emotional deep Q learning for real-time smart voltage control of cyber-physical social power systems," Energy, Elsevier, vol. 273(C).
    6. Zhou, Yuekuan & Cao, Sunliang & Hensen, Jan L.M., 2021. "An energy paradigm transition framework from negative towards positive district energy sharing networks—Battery cycling aging, advanced battery management strategies, flexible vehicles-to-buildings in," Applied Energy, Elsevier, vol. 288(C).
    7. Gonçalves Rigueira Pinheiro Castro, Pedro Henrique & Filho, Delly Oliveira & Rosa, André Pereira & Navas Gracia, Luis Manuel & Almeida Silva, Thais Cristina, 2024. "Comparison of externalities of biogas and photovoltaic solar energy for energy planning," Energy Policy, Elsevier, vol. 188(C).
    8. Jerzy Andruszkiewicz & Józef Lorenc & Agnieszka Weychan, 2023. "Determination of the Optimal Level of Reactive Power Compensation That Minimizes the Costs of Losses in Distribution Networks," Energies, MDPI, vol. 17(1), pages 1-24, December.
    9. Mehrjerdi, Hasan & Hemmati, Reza, 2020. "Coordination of vehicle-to-home and renewable capacity resources for energy management in resilience and self-healing building," Renewable Energy, Elsevier, vol. 146(C), pages 568-579.
    10. Wiktor Wróblewski & Ryszard Kowalik & Marcin Januszewski & Karol Kurek, 2024. "A Fuzzy OLTC Controller: Applicability in the Transition Stage of the Energy System Transformation," Energies, MDPI, vol. 17(11), pages 1-16, June.
    11. Hu, Yusha & Man, Yi, 2022. "Two-stage energy scheduling optimization model for complex industrial process and its industrial verification," Renewable Energy, Elsevier, vol. 193(C), pages 879-894.
    12. Paweł Kelm & Rozmysław Mieński & Irena Wasiak, 2024. "Modular PV System for Applications in Prosumer Installations with Uncontrolled, Unbalanced and Non-Linear Loads," Energies, MDPI, vol. 17(7), pages 1-13, March.
    13. Rabea Jamil Mahfoud & Nizar Faisal Alkayem & Emmanuel Fernandez-Rodriguez & Yuan Zheng & Yonghui Sun & Shida Zhang & Yuquan Zhang, 2024. "Evolutionary Approach for DISCO Profit Maximization by Optimal Planning of Distributed Generators and Energy Storage Systems in Active Distribution Networks," Mathematics, MDPI, vol. 12(2), pages 1-33, January.
    14. Adetunji, Kayode E. & Hofsajer, Ivan W. & Abu-Mahfouz, Adnan M. & Cheng, Ling, 2022. "An optimization planning framework for allocating multiple distributed energy resources and electric vehicle charging stations in distribution networks," Applied Energy, Elsevier, vol. 322(C).
    15. Mandhir Kumar Verma & Vivekananda Mukherjee & Vinod Kumar Yadav & Santosh Ghosh, 2020. "Constraints for effective distribution network expansion planning: an ample review," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 11(3), pages 531-546, June.
    16. He, Shuaijia & Gao, Hongjun & Liu, Junyong & Zhang, Xi & Chen, Zhe, 2022. "Distribution system planning considering peak shaving of energy station," Applied Energy, Elsevier, vol. 312(C).
    17. Liu, Jiantao & Wang, Lei & Jiao, Ticao & Yang, Wenjing & Xu, Ruilong & Wu, Ke & Ha, Hengxu, 2024. "Multi-period voltage stability-constrained optimal power flow with uncertainties," Applied Energy, Elsevier, vol. 369(C).
    18. Jiang, Qian & Mu, Yunfei & Jia, Hongjie & Cao, Yan & Wang, Zibo & Wei, Wei & Hou, Kai & Yu, Xiaodan, 2022. "A Stackelberg Game-based planning approach for integrated community energy system considering multiple participants," Energy, Elsevier, vol. 258(C).
    19. Shen, Lu & Dou, Xiaobo & Long, Huan & Li, Chen & Chen, Kang & Zhou, Ji, 2021. "A collaborative voltage optimization utilizing flexibility of community heating systems for high PV penetration," Energy, Elsevier, vol. 232(C).
    20. Masood, Nahid-Al- & Mahmud, Sajjad Uddin & Ansary, Md Nazmuddoha & Deeba, Shohana Rahman, 2022. "Improvement of system strength under high wind penetration: A techno-economic assessment using synchronous condenser and SVC," Energy, Elsevier, vol. 246(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:332:y:2023:i:c:s0306261922017676. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.