IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v280y2020ics0306261920313672.html
   My bibliography  Save this article

Online reconfiguration scheme of self-sufficient distribution network based on a reinforcement learning approach

Author

Listed:
  • Oh, Seok Hwa
  • Yoon, Yong Tae
  • Kim, Seung Wan

Abstract

With increasing number of distributed renewable energy sources integrated in power distribution networks, network security issues such as line overloading or bus voltage violations are becoming increasingly common. Traditional capital-intensive system reinforcements could lead to overinvestment. Moreover, active network management solutions, which have emerged as important alternatives, may become a financial burden for distribution system operators or reduce profits for owners of distributed renewable energy sources, or both. To address these limitations, this paper proposes an online network reconfiguration scheme based on a deep reinforcement learning approach. In this scheme, the distribution network operator modifies the network topology to change the power flow when the reliability of network is threatened. Because the variability of distributed renewable energy is large in self-sufficient distribution networks, the reconfiguration process needs to be performed online within short time intervals, which involves the use of conventional algorithms. To solve this problem efficiently, a deep q-learning model is utilized to determine the optimal network topology. Performances of proposed and other algorithms were compared in modified CIGRE 14-bus and IEEE 123-bus test network, as well as varying penalties for frequent switching operation in consideration of physical characteristic of the network. Simulation results demonstrated that the proposed algorithm showed almost identical performances with brute-force search algorithm in both test networks, satisfying network constraints over almost all timespans. Further, the proposed method required very small computation times - under a second per each state and its scalability was verified by comparing the computation time between two test networks.

Suggested Citation

  • Oh, Seok Hwa & Yoon, Yong Tae & Kim, Seung Wan, 2020. "Online reconfiguration scheme of self-sufficient distribution network based on a reinforcement learning approach," Applied Energy, Elsevier, vol. 280(C).
  • Handle: RePEc:eee:appene:v:280:y:2020:i:c:s0306261920313672
    DOI: 10.1016/j.apenergy.2020.115900
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261920313672
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2020.115900?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kou, Peng & Liang, Deliang & Wang, Chen & Wu, Zihao & Gao, Lin, 2020. "Safe deep reinforcement learning-based constrained optimal control scheme for active distribution networks," Applied Energy, Elsevier, vol. 264(C).
    2. Bai, Linquan & Jiang, Tao & Li, Fangxing & Chen, Houhe & Li, Xue, 2018. "Distributed energy storage planning in soft open point based active distribution networks incorporating network reconfiguration and DG reactive power capability," Applied Energy, Elsevier, vol. 210(C), pages 1082-1091.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    2. Cao, Di & Zhao, Junbo & Hu, Weihao & Ding, Fei & Yu, Nanpeng & Huang, Qi & Chen, Zhe, 2022. "Model-free voltage control of active distribution system with PVs using surrogate model-based deep reinforcement learning," Applied Energy, Elsevier, vol. 306(PA).
    3. Aras Ghafoor & Jamal Aldahmashi & Judith Apsley & Siniša Djurović & Xiandong Ma & Mohamed Benbouzid, 2024. "Intelligent Integration of Renewable Energy Resources Review: Generation and Grid Level Opportunities and Challenges," Energies, MDPI, vol. 17(17), pages 1-29, September.
    4. Zhao, Yincheng & Zhang, Guozhou & Hu, Weihao & Huang, Qi & Chen, Zhe & Blaabjerg, Frede, 2023. "Meta-learning based voltage control strategy for emergency faults of active distribution networks," Applied Energy, Elsevier, vol. 349(C).
    5. Nastaran Gholizadeh & Petr Musilek, 2024. "A Generalized Deep Reinforcement Learning Model for Distribution Network Reconfiguration with Power Flow-Based Action-Space Sampling," Energies, MDPI, vol. 17(20), pages 1-18, October.
    6. Mohammad Javad Bordbari & Fuzhan Nasiri, 2024. "Networked Microgrids: A Review on Configuration, Operation, and Control Strategies," Energies, MDPI, vol. 17(3), pages 1-28, February.
    7. Hua, Weiqi & Stephen, Bruce & Wallom, David C.H., 2023. "Digital twin based reinforcement learning for extracting network structures and load patterns in planning and operation of distribution systems," Applied Energy, Elsevier, vol. 342(C).
    8. Boza, Pal & Evgeniou, Theodoros, 2021. "Artificial intelligence to support the integration of variable renewable energy sources to the power system," Applied Energy, Elsevier, vol. 290(C).
    9. Ibrahim Salem Jahan & Vojtech Blazek & Stanislav Misak & Vaclav Snasel & Lukas Prokop, 2022. "Forecasting of Power Quality Parameters Based on Meteorological Data in Small-Scale Household Off-Grid Systems," Energies, MDPI, vol. 15(14), pages 1-20, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jiao, P.H. & Chen, J.J. & Cai, X. & Wang, L.L. & Zhao, Y.L. & Zhang, X.H. & Chen, W.G., 2021. "Joint active and reactive for allocation of renewable energy and energy storage under uncertain coupling," Applied Energy, Elsevier, vol. 302(C).
    2. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).
    3. Jannesar, Mohammad Rasol & Sedighi, Alireza & Savaghebi, Mehdi & Guerrero, Josep M., 2018. "Optimal placement, sizing, and daily charge/discharge of battery energy storage in low voltage distribution network with high photovoltaic penetration," Applied Energy, Elsevier, vol. 226(C), pages 957-966.
    4. Li, Yang & Vilathgamuwa, Mahinda & Choi, San Shing & Farrell, Troy W. & Tran, Ngoc Tham & Teague, Joseph, 2019. "Development of a degradation-conscious physics-based lithium-ion battery model for use in power system planning studies," Applied Energy, Elsevier, vol. 248(C), pages 512-525.
    5. Jude Suchithra & Amin Rajabi & Duane A. Robinson, 2024. "Enhancing PV Hosting Capacity of Electricity Distribution Networks Using Deep Reinforcement Learning-Based Coordinated Voltage Control," Energies, MDPI, vol. 17(20), pages 1-27, October.
    6. Yin, Linfei & Lu, Yuejiang, 2021. "Expandable deep width learning for voltage control of three-state energy model based smart grids containing flexible energy sources," Energy, Elsevier, vol. 226(C).
    7. Li, Yang & Vilathgamuwa, Mahinda & Choi, San Shing & Xiong, Binyu & Tang, Jinrui & Su, Yixin & Wang, Yu, 2020. "Design of minimum cost degradation-conscious lithium-ion battery energy storage system to achieve renewable power dispatchability," Applied Energy, Elsevier, vol. 260(C).
    8. Qin Wang & Jincan Zeng & Beibei Cheng & Minwei Liu & Guori Huang & Xi Liu & Gengsheng He & Shangheng Yao & Peng Wang & Longxi Li, 2024. "A Cooperative Game Approach for Optimal Design of Shared Energy Storage System," Sustainability, MDPI, vol. 16(17), pages 1-19, August.
    9. Dranka, Géremi Gilson & Ferreira, Paula & Vaz, A. Ismael F., 2021. "A review of co-optimization approaches for operational and planning problems in the energy sector," Applied Energy, Elsevier, vol. 304(C).
    10. Guocheng Liu & Weiqing Sun & Haoyun Hong & Gang Shi, 2024. "Coordinated Configuration of SOPs and DESSs in an Active Distribution Network Considering Social Welfare Maximization," Sustainability, MDPI, vol. 16(6), pages 1-20, March.
    11. Zhu, Dafeng & Yang, Bo & Liu, Yuxiang & Wang, Zhaojian & Ma, Kai & Guan, Xinping, 2022. "Energy management based on multi-agent deep reinforcement learning for a multi-energy industrial park," Applied Energy, Elsevier, vol. 311(C).
    12. Homod, Raad Z. & Togun, Hussein & Kadhim Hussein, Ahmed & Noraldeen Al-Mousawi, Fadhel & Yaseen, Zaher Mundher & Al-Kouz, Wael & Abd, Haider J. & Alawi, Omer A. & Goodarzi, Marjan & Hussein, Omar A., 2022. "Dynamics analysis of a novel hybrid deep clustering for unsupervised learning by reinforcement of multi-agent to energy saving in intelligent buildings," Applied Energy, Elsevier, vol. 313(C).
    13. Se-Heon Lim & Sung-Guk Yoon, 2022. "Dynamic DNR and Solar PV Smart Inverter Control Scheme Using Heterogeneous Multi-Agent Deep Reinforcement Learning," Energies, MDPI, vol. 15(23), pages 1-18, December.
    14. Meng, He & Jia, Hongjie & Xu, Tao & Wei, Wei & Wu, Yuhan & Liang, Lemeng & Cai, Shuqi & Liu, Zuozheng & Wang, Rujing & Li, Mengchao, 2022. "Optimal configuration of cooperative stationary and mobile energy storage considering ambient temperature: A case for Winter Olympic Game," Applied Energy, Elsevier, vol. 325(C).
    15. Jude Suchithra & Duane Robinson & Amin Rajabi, 2023. "Hosting Capacity Assessment Strategies and Reinforcement Learning Methods for Coordinated Voltage Control in Electricity Distribution Networks: A Review," Energies, MDPI, vol. 16(5), pages 1-28, March.
    16. Mandhir Kumar Verma & Vivekananda Mukherjee & Vinod Kumar Yadav & Santosh Ghosh, 2020. "Constraints for effective distribution network expansion planning: an ample review," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 11(3), pages 531-546, June.
    17. Saman Nikkhah & Arman Alahyari & Adib Allahham & Khaled Alawasa, 2023. "Optimal Integration of Hybrid Energy Systems: A Security-Constrained Network Topology Reconfiguration," Energies, MDPI, vol. 16(6), pages 1-19, March.
    18. Zhang, Zhengfa & da Silva, Filipe Faria & Guo, Yifei & Bak, Claus Leth & Chen, Zhe, 2021. "Double-layer stochastic model predictive voltage control in active distribution networks with high penetration of renewables," Applied Energy, Elsevier, vol. 302(C).
    19. Kewei Wang & Yonghong Huang & Junjun Xu & Yanbo Liu, 2024. "A Flexible Envelope Method for the Operation Domain of Distribution Networks Based on “Degree of Squareness” Adjustable Superellipsoid," Energies, MDPI, vol. 17(16), pages 1-19, August.
    20. Su, Hongzhi & Wang, Chengshan & Li, Peng & Liu, Zhelin & Yu, Li & Wu, Jianzhong, 2019. "Optimal placement of phasor measurement unit in distribution networks considering the changes in topology," Applied Energy, Elsevier, vol. 250(C), pages 313-322.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:280:y:2020:i:c:s0306261920313672. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.