IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v332y2023ics0306261922017676.html
   My bibliography  Save this article

Deep reinforcement learning based topology-aware voltage regulation of distribution networks with distributed energy storage

Author

Listed:
  • Xiang, Yue
  • Lu, Yu
  • Liu, Junyong

Abstract

Both the high penetration of clean energy with strong fluctuation and the complicated variable operation condition bring great challenges to the voltage regulation of the distribution network. To deal with the problem, a topology-aware voltage regulation multi-agent deep reinforcement learning (MADRL) algorithm is proposed. The distributed energy storages (DESs) are modeled as agents to regulate voltage autonomously in real-time, which could fast adapt to dynamic topological scenarios. Firstly, taking the minimization of voltage fluctuation and maximization of reserve capacity as the target, the optimal voltage regulation model is established. Secondly, a topology extraction method considering voltage sensitivity is proposed for dynamic topology clustering, and the obtained typical topology is added to the observation set of agents. Then, the optimal voltage regulation model is formulated to the decentralized partially observable Markov decision process (Dec-POMDP) framework, in which only local information is required for the agent during the test process to decision-making to realize the hierarchical and partitioned control of voltage. Finally, the multi-agent deep deterministic policy gradient (MADDPG) algorithm is used to solve the Dec-POMDP model. The feasibility and superiority of the proposed algorithm are verified and analyzed in the simulation under different scenarios.

Suggested Citation

  • Xiang, Yue & Lu, Yu & Liu, Junyong, 2023. "Deep reinforcement learning based topology-aware voltage regulation of distribution networks with distributed energy storage," Applied Energy, Elsevier, vol. 332(C).
    • Handle: RePEc:eee:appene:v:332:y:2023:i:c:s0306261922017676
    DOI: 10.1016/j.apenergy.2022.120510
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261922017676
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2022.120510?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ma, Wei & Wang, Wei & Chen, Zhe & Wu, Xuezhi & Hu, Ruonan & Tang, Fen & Zhang, Weige, 2021. "Voltage regulation methods for active distribution networks considering the reactive power optimization of substations," Applied Energy, Elsevier, vol. 284(C).
    2. Mehrjerdi, Hasan, 2019. "Simultaneous load leveling and voltage profile improvement in distribution networks by optimal battery storage planning," Energy, Elsevier, vol. 181(C), pages 916-926.
    3. Lai, Qiupin & Liu, Chengxi & Sun, Kai, 2021. "Vulnerability assessment for voltage stability based on solvability regions of decoupled power flow equations," Applied Energy, Elsevier, vol. 304(C).
    4. Zhenming Li & Yunfeng Yan & Donglian Qi & Shuo Yan & Minghao Wang, 2022. "Distributed Voltage Optimization Control of BESS in AC Distribution Networks with High PV Penetration," Energies, MDPI, vol. 15(11), pages 1-14, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhu, Xingxu & Hou, Xiangchen & Li, Junhui & Yan, Gangui & Li, Cuiping & Wang, Dongbo, 2023. "Distributed online prediction optimization algorithm for distributed energy resources considering the multi-periods optimal operation," Applied Energy, Elsevier, vol. 348(C).
    2. Jianxun Luo & Wei Zhang & Hui Wang & Wenmiao Wei & Jinpeng He, 2023. "Research on Data-Driven Optimal Scheduling of Power System," Energies, MDPI, vol. 16(6), pages 1-15, March.
    3. Chen, Yongdong & Liu, Youbo & Zhao, Junbo & Qiu, Gao & Yin, Hang & Li, Zhengbo, 2023. "Physical-assisted multi-agent graph reinforcement learning enabled fast voltage regulation for PV-rich active distribution network," Applied Energy, Elsevier, vol. 351(C).
    4. Zhao, Yincheng & Zhang, Guozhou & Hu, Weihao & Huang, Qi & Chen, Zhe & Blaabjerg, Frede, 2023. "Meta-learning based voltage control strategy for emergency faults of active distribution networks," Applied Energy, Elsevier, vol. 349(C).
    5. Rabea Jamil Mahfoud & Nizar Faisal Alkayem & Emmanuel Fernandez-Rodriguez & Yuan Zheng & Yonghui Sun & Shida Zhang & Yuquan Zhang, 2024. "Evolutionary Approach for DISCO Profit Maximization by Optimal Planning of Distributed Generators and Energy Storage Systems in Active Distribution Networks," Mathematics, MDPI, vol. 12(2), pages 1-33, January.
    6. Escobar, Eros D. & Betancur, Daniel & Manrique, Tatiana & Isaac, Idi A., 2023. "Model predictive real-time architecture for secondary voltage control of microgrids," Applied Energy, Elsevier, vol. 345(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. A.S. Jameel Hassan & Umar Marikkar & G.W. Kasun Prabhath & Aranee Balachandran & W.G. Chaminda Bandara & Parakrama B. Ekanayake & Roshan I. Godaliyadda & Janaka B. Ekanayake, 2021. "A Sensitivity Matrix Approach Using Two-Stage Optimization for Voltage Regulation of LV Networks with High PV Penetration," Energies, MDPI, vol. 14(20), pages 1-24, October.
    2. Lu, Yu & Xiang, Yue & Huang, Yuan & Yu, Bin & Weng, Liguo & Liu, Junyong, 2023. "Deep reinforcement learning based optimal scheduling of active distribution system considering distributed generation, energy storage and flexible load," Energy, Elsevier, vol. 271(C).
    3. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).
    4. Zhu, Xingxu & Hou, Xiangchen & Li, Junhui & Yan, Gangui & Li, Cuiping & Wang, Dongbo, 2023. "Distributed online prediction optimization algorithm for distributed energy resources considering the multi-periods optimal operation," Applied Energy, Elsevier, vol. 348(C).
    5. Yin, Linfei & He, Xiaoyu, 2023. "Artificial emotional deep Q learning for real-time smart voltage control of cyber-physical social power systems," Energy, Elsevier, vol. 273(C).
    6. Gonçalves Rigueira Pinheiro Castro, Pedro Henrique & Filho, Delly Oliveira & Rosa, André Pereira & Navas Gracia, Luis Manuel & Almeida Silva, Thais Cristina, 2024. "Comparison of externalities of biogas and photovoltaic solar energy for energy planning," Energy Policy, Elsevier, vol. 188(C).
    7. Jerzy Andruszkiewicz & Józef Lorenc & Agnieszka Weychan, 2023. "Determination of the Optimal Level of Reactive Power Compensation That Minimizes the Costs of Losses in Distribution Networks," Energies, MDPI, vol. 17(1), pages 1-24, December.
    8. Mehrjerdi, Hasan & Hemmati, Reza, 2020. "Coordination of vehicle-to-home and renewable capacity resources for energy management in resilience and self-healing building," Renewable Energy, Elsevier, vol. 146(C), pages 568-579.
    9. Hu, Yusha & Man, Yi, 2022. "Two-stage energy scheduling optimization model for complex industrial process and its industrial verification," Renewable Energy, Elsevier, vol. 193(C), pages 879-894.
    10. Rabea Jamil Mahfoud & Nizar Faisal Alkayem & Emmanuel Fernandez-Rodriguez & Yuan Zheng & Yonghui Sun & Shida Zhang & Yuquan Zhang, 2024. "Evolutionary Approach for DISCO Profit Maximization by Optimal Planning of Distributed Generators and Energy Storage Systems in Active Distribution Networks," Mathematics, MDPI, vol. 12(2), pages 1-33, January.
    11. Shen, Lu & Dou, Xiaobo & Long, Huan & Li, Chen & Chen, Kang & Zhou, Ji, 2021. "A collaborative voltage optimization utilizing flexibility of community heating systems for high PV penetration," Energy, Elsevier, vol. 232(C).
    12. Masood, Nahid-Al- & Mahmud, Sajjad Uddin & Ansary, Md Nazmuddoha & Deeba, Shohana Rahman, 2022. "Improvement of system strength under high wind penetration: A techno-economic assessment using synchronous condenser and SVC," Energy, Elsevier, vol. 246(C).
    13. Tianhao Song & Xiaoqing Han & Baifu Zhang, 2021. "Multi-Time-Scale Optimal Scheduling in Active Distribution Network with Voltage Stability Constraints," Energies, MDPI, vol. 14(21), pages 1-20, November.
    14. Kang, Wenfa & Chen, Minyou & Guan, Yajuan & Wei, Baoze & Vasquez Q., Juan C. & Guerrero, Josep M., 2022. "Event-triggered distributed voltage regulation by heterogeneous BESS in low-voltage distribution networks," Applied Energy, Elsevier, vol. 312(C).
    15. Solanke, Tirupati U. & Khatua, Pradeep K. & Ramachandaramurthy, Vigna K. & Yong, Jia Ying & Tan, Kang Miao, 2021. "Control and management of a multilevel electric vehicles infrastructure integrated with distributed resources: A comprehensive review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 144(C).
    16. Yang, Jun & Su, Changqi, 2021. "Robust optimization of microgrid based on renewable distributed power generation and load demand uncertainty," Energy, Elsevier, vol. 223(C).
    17. Anis ur Rehman & Muhammad Ali & Sheeraz Iqbal & Aqib Shafiq & Nasim Ullah & Sattam Al Otaibi, 2022. "Artificial Intelligence-Based Control and Coordination of Multiple PV Inverters for Reactive Power/Voltage Control of Power Distribution Networks," Energies, MDPI, vol. 15(17), pages 1-13, August.
    18. Zhou, Yuekuan & Cao, Sunliang & Hensen, Jan L.M., 2021. "An energy paradigm transition framework from negative towards positive district energy sharing networks—Battery cycling aging, advanced battery management strategies, flexible vehicles-to-buildings in," Applied Energy, Elsevier, vol. 288(C).
    19. Wiktor Wróblewski & Ryszard Kowalik & Marcin Januszewski & Karol Kurek, 2024. "A Fuzzy OLTC Controller: Applicability in the Transition Stage of the Energy System Transformation," Energies, MDPI, vol. 17(11), pages 1-16, June.
    20. Paweł Kelm & Rozmysław Mieński & Irena Wasiak, 2024. "Modular PV System for Applications in Prosumer Installations with Uncontrolled, Unbalanced and Non-Linear Loads," Energies, MDPI, vol. 17(7), pages 1-13, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:332:y:2023:i:c:s0306261922017676. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.