IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i20p5187-d1501500.html
   My bibliography  Save this article

A Generalized Deep Reinforcement Learning Model for Distribution Network Reconfiguration with Power Flow-Based Action-Space Sampling

Author

Listed:
  • Nastaran Gholizadeh

    (Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada)

  • Petr Musilek

    (Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada)

Abstract

Distribution network reconfiguration (DNR) is used by utilities to enhance power system performance in various ways, such as reducing line losses. Conventional DNR algorithms rely on accurate values of network parameters and lack scalability and optimality. To tackle these issues, a new data-driven algorithm based on reinforcement learning is developed for DNR in this paper. The proposed algorithm comprises two main parts. The first part, named action-space sampling, aims at analyzing the network structure, finding all feasible reconfiguration actions, and reducing the size of the action space to only the most optimal actions. In the second part, deep Q-learning (DQN) and dueling DQN methods are used to train an agent to take the best switching actions according to the switch states and loads of the system. The results show that both DQN and dueling DQN are effective in reducing system losses through grid reconfiguration. The proposed methods have faster execution time compared to the conventional methods and are more scalable.

Suggested Citation

  • Nastaran Gholizadeh & Petr Musilek, 2024. "A Generalized Deep Reinforcement Learning Model for Distribution Network Reconfiguration with Power Flow-Based Action-Space Sampling," Energies, MDPI, vol. 17(20), pages 1-18, October.
    • Handle: RePEc:gam:jeners:v:17:y:2024:i:20:p:5187-:d:1501500
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/20/5187/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/20/5187/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Oh, Seok Hwa & Yoon, Yong Tae & Kim, Seung Wan, 2020. "Online reconfiguration scheme of self-sufficient distribution network based on a reinforcement learning approach," Applied Energy, Elsevier, vol. 280(C).
    2. Elham Mahdavi & Seifollah Asadpour & Leonardo H. Macedo & Rubén Romero, 2023. "Reconfiguration of Distribution Networks with Simultaneous Allocation of Distributed Generation Using the Whale Optimization Algorithm," Energies, MDPI, vol. 16(12), pages 1-19, June.
    3. Wu, Tao & Wang, Jianhui & Lu, Xiaonan & Du, Yuhua, 2022. "AC/DC hybrid distribution network reconfiguration with microgrid formation using multi-agent soft actor-critic," Applied Energy, Elsevier, vol. 307(C).
    4. Ezequiel C. Pereira & Carlos H. N. R. Barbosa & João A. Vasconcelos, 2023. "Distribution Network Reconfiguration Using Iterative Branch Exchange and Clustering Technique," Energies, MDPI, vol. 16(5), pages 1-20, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mohammad Javad Bordbari & Fuzhan Nasiri, 2024. "Networked Microgrids: A Review on Configuration, Operation, and Control Strategies," Energies, MDPI, vol. 17(3), pages 1-28, February.
    2. Zhao, Yincheng & Zhang, Guozhou & Hu, Weihao & Huang, Qi & Chen, Zhe & Blaabjerg, Frede, 2023. "Meta-learning based voltage control strategy for emergency faults of active distribution networks," Applied Energy, Elsevier, vol. 349(C).
    3. Samson Oladayo Ayanlade & Funso Kehinde Ariyo & Abdulrasaq Jimoh & Kayode Timothy Akindeji & Adeleye Oluwaseye Adetunji & Emmanuel Idowu Ogunwole & Dolapo Eniola Owolabi, 2023. "Optimal Allocation of Photovoltaic Distributed Generations in Radial Distribution Networks," Sustainability, MDPI, vol. 15(18), pages 1-26, September.
    4. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    5. Boza, Pal & Evgeniou, Theodoros, 2021. "Artificial intelligence to support the integration of variable renewable energy sources to the power system," Applied Energy, Elsevier, vol. 290(C).
    6. Ning Xin & Laijun Chen & Linrui Ma & Yang Si, 2022. "A Rolling Horizon Optimization Framework for Resilient Restoration of Active Distribution Systems," Energies, MDPI, vol. 15(9), pages 1-14, April.
    7. Xie, Haipeng & Tang, Lingfeng & Zhu, Hao & Cheng, Xiaofeng & Bie, Zhaohong, 2023. "Robustness assessment and enhancement of deep reinforcement learning-enabled load restoration for distribution systems," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    8. Cao, Di & Zhao, Junbo & Hu, Weihao & Ding, Fei & Yu, Nanpeng & Huang, Qi & Chen, Zhe, 2022. "Model-free voltage control of active distribution system with PVs using surrogate model-based deep reinforcement learning," Applied Energy, Elsevier, vol. 306(PA).
    9. Ibrahim Salem Jahan & Vojtech Blazek & Stanislav Misak & Vaclav Snasel & Lukas Prokop, 2022. "Forecasting of Power Quality Parameters Based on Meteorological Data in Small-Scale Household Off-Grid Systems," Energies, MDPI, vol. 15(14), pages 1-20, July.
    10. Zhang, Lu & Yu, Shunjiang & Zhang, Bo & Li, Gen & Cai, Yongxiang & Tang, Wei, 2023. "Outage management of hybrid AC/DC distribution systems: Co-optimize service restoration with repair crew and mobile energy storage system dispatch," Applied Energy, Elsevier, vol. 335(C).
    11. Elham Mahdavi & Seifollah Asadpour & Leonardo H. Macedo & Rubén Romero, 2023. "Reconfiguration of Distribution Networks with Simultaneous Allocation of Distributed Generation Using the Whale Optimization Algorithm," Energies, MDPI, vol. 16(12), pages 1-19, June.
    12. Qiu, Dawei & Wang, Yi & Zhang, Tingqi & Sun, Mingyang & Strbac, Goran, 2023. "Hierarchical multi-agent reinforcement learning for repair crews dispatch control towards multi-energy microgrid resilience," Applied Energy, Elsevier, vol. 336(C).
    13. Mudhafar Al-Saadi & Maher Al-Greer & Michael Short, 2023. "Reinforcement Learning-Based Intelligent Control Strategies for Optimal Power Management in Advanced Power Distribution Systems: A Survey," Energies, MDPI, vol. 16(4), pages 1-38, February.
    14. Min Zhu & Saber Arabi Nowdeh & Aspassia Daskalopulu, 2023. "An Improved Human-Inspired Algorithm for Distribution Network Stochastic Reconfiguration Using a Multi-Objective Intelligent Framework and Unscented Transformation," Mathematics, MDPI, vol. 11(17), pages 1-23, August.
    15. Aras Ghafoor & Jamal Aldahmashi & Judith Apsley & Siniša Djurović & Xiandong Ma & Mohamed Benbouzid, 2024. "Intelligent Integration of Renewable Energy Resources Review: Generation and Grid Level Opportunities and Challenges," Energies, MDPI, vol. 17(17), pages 1-29, September.
    16. Zhang, Guozhou & Hu, Weihao & Cao, Di & Zhou, Dao & Huang, Qi & Chen, Zhe & Blaabjerg, Frede, 2023. "Coordinated active and reactive power dynamic dispatch strategy for wind farms to minimize levelized production cost considering system uncertainty: A soft actor-critic approach," Renewable Energy, Elsevier, vol. 218(C).
    17. Matheus Diniz Gonçalves-Leite & Edgar Manuel Carreño-Franco & Jesús M. López-Lezama, 2023. "Impact of Distributed Generation on the Effectiveness of Electric Distribution System Reconfiguration," Energies, MDPI, vol. 16(17), pages 1-20, August.
    18. Hua, Weiqi & Stephen, Bruce & Wallom, David C.H., 2023. "Digital twin based reinforcement learning for extracting network structures and load patterns in planning and operation of distribution systems," Applied Energy, Elsevier, vol. 342(C).
    19. Tie Chen & Pingping Yang & Hongxin Li & Jiaqi Gao & Yimin Yuan, 2024. "Two-Stage Optimization Model Based on Neo4j-Dueling Deep Q Network," Energies, MDPI, vol. 17(19), pages 1-18, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:20:p:5187-:d:1501500. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.