IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v335y2023ics0306261922018864.html
   My bibliography  Save this article

Deep reinforcement learning-based two-timescale Volt-VAR control with degradation-aware smart inverters in power distribution systems

Author

Listed:
  • Kabir, Farzana
  • Yu, Nanpeng
  • Gao, Yuanqi
  • Wang, Wenyu

Abstract

Higher penetration of intermittent solar photovoltaic (PV) systems in the distribution grid results in frequent voltage fluctuations. The conventional voltage regulating devices operating on a slow-timescale need to be supplemented with the fast-operating smart inverters with adjustable reactive power setpoints. Complete and accurate information about distribution network topology and line parameters is necessary for conventional model-based Volt-VAR control (VVC) methods. However, such information is often unavailable. To tackle these challenges, a reinforcement learning-based two-timescale VVC algorithm is proposed in this paper that jointly controls the conventional voltage regulating devices at the slow-timescale and the smart inverters at the fast-timescale. Our proposed VVC algorithm simultaneously minimizes voltage violation costs and system operation costs in a model-free manner utilizing historical operational data. Two hierarchically organized agents are set up for the slow-timescale and fast-timescale problems, which are coupled through a communication scheme. The two sets of control policies are learned concurrently by a deep deterministic policy gradient and multi-agent soft actor-critic algorithm respectively. Comprehensive numerical studies performed with the IEEE 123-bus distribution test feeder show that the proposed framework can identify near optimal control actions of voltage regulating devices and smart inverters in real-time operations.

Suggested Citation

  • Kabir, Farzana & Yu, Nanpeng & Gao, Yuanqi & Wang, Wenyu, 2023. "Deep reinforcement learning-based two-timescale Volt-VAR control with degradation-aware smart inverters in power distribution systems," Applied Energy, Elsevier, vol. 335(C).
  • Handle: RePEc:eee:appene:v:335:y:2023:i:c:s0306261922018864
    DOI: 10.1016/j.apenergy.2022.120629
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261922018864
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2022.120629?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lee, Xian Yeow & Sarkar, Soumik & Wang, Yubo, 2022. "A graph policy network approach for Volt-Var Control in power distribution systems," Applied Energy, Elsevier, vol. 323(C).
    2. Mak, Davye & Choi, Dae-Hyun, 2020. "Optimization framework for coordinated operation of home energy management system and Volt-VAR optimization in unbalanced active distribution networks considering uncertainties," Applied Energy, Elsevier, vol. 276(C).
    3. Jeon, Soi & Choi, Dae-Hyun, 2022. "Joint optimization of Volt/VAR control and mobile energy storage system scheduling in active power distribution networks under PV prediction uncertainty," Applied Energy, Elsevier, vol. 310(C).
    4. Zhang, Wenjie & Gandhi, Oktoviano & Quan, Hao & Rodríguez-Gallegos, Carlos D. & Srinivasan, Dipti, 2018. "A multi-agent based integrated volt-var optimization engine for fast vehicle-to-grid reactive power dispatch and electric vehicle coordination," Applied Energy, Elsevier, vol. 229(C), pages 96-110.
    5. Gao, Yuanqi & Yu, Nanpeng, 2022. "Model-augmented safe reinforcement learning for Volt-VAR control in power distribution networks," Applied Energy, Elsevier, vol. 313(C).
    6. Haider, Rabab & Annaswamy, Anuradha M., 2022. "A hybrid architecture for volt-var control in active distribution grids," Applied Energy, Elsevier, vol. 312(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Boscaino, Valeria & Ditta, Vito & Marsala, Giuseppe & Panzavecchia, Nicola & Tinè, Giovanni & Cosentino, Valentina & Cataliotti, Antonio & Di Cara, Dario, 2024. "Grid-connected photovoltaic inverters: Grid codes, topologies and control techniques," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PA).
    2. Zhao, Yincheng & Zhang, Guozhou & Hu, Weihao & Huang, Qi & Chen, Zhe & Blaabjerg, Frede, 2023. "Meta-learning based voltage control strategy for emergency faults of active distribution networks," Applied Energy, Elsevier, vol. 349(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guo, Guodong & Zhang, Mengfan & Gong, Yanfeng & Xu, Qianwen, 2023. "Safe multi-agent deep reinforcement learning for real-time decentralized control of inverter based renewable energy resources considering communication delay," Applied Energy, Elsevier, vol. 349(C).
    2. Jeon, Soi & Choi, Dae-Hyun, 2022. "Joint optimization of Volt/VAR control and mobile energy storage system scheduling in active power distribution networks under PV prediction uncertainty," Applied Energy, Elsevier, vol. 310(C).
    3. Edward J. Smith & Duane A. Robinson & Sean Elphick, 2024. "DER Control and Management Strategies for Distribution Networks: A Review of Current Practices and Future Directions," Energies, MDPI, vol. 17(11), pages 1-40, May.
    4. Mak, Davye & Choeum, Daranith & Choi, Dae-Hyun, 2020. "Sensitivity analysis of volt-VAR optimization to data changes in distribution networks with distributed energy resources," Applied Energy, Elsevier, vol. 261(C).
    5. Jeddi, Babak & Mishra, Yateendra & Ledwich, Gerard, 2021. "Distributed load scheduling in residential neighborhoods for coordinated operation of multiple home energy management systems," Applied Energy, Elsevier, vol. 300(C).
    6. Zhou, Yu & Li, Zhengshuo & Wang, Guangrui, 2021. "Study on leveraging wind farms' robust reactive power range for uncertain power system reactive power optimization," Applied Energy, Elsevier, vol. 298(C).
    7. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).
    8. Liu, Shan & Yan, Jie & Yan, Yamin & Zhang, Haoran & Zhang, Jing & Liu, Yongqian & Han, Shuang, 2024. "Joint operation of mobile battery, power system, and transportation system for improving the renewable energy penetration rate," Applied Energy, Elsevier, vol. 357(C).
    9. Zhao, Yincheng & Zhang, Guozhou & Hu, Weihao & Huang, Qi & Chen, Zhe & Blaabjerg, Frede, 2023. "Meta-learning based voltage control strategy for emergency faults of active distribution networks," Applied Energy, Elsevier, vol. 349(C).
    10. Zhuoxin Lu & Xiaoyuan Xu & Zheng Yan & Dong Han & Shiwei Xia, 2024. "Mobile Energy-Storage Technology in Power Grid: A Review of Models and Applications," Sustainability, MDPI, vol. 16(16), pages 1-19, August.
    11. Das, H.S. & Rahman, M.M. & Li, S. & Tan, C.W., 2020. "Electric vehicles standards, charging infrastructure, and impact on grid integration: A technological review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 120(C).
    12. Jude Suchithra & Amin Rajabi & Duane A. Robinson, 2024. "Enhancing PV Hosting Capacity of Electricity Distribution Networks Using Deep Reinforcement Learning-Based Coordinated Voltage Control," Energies, MDPI, vol. 17(20), pages 1-27, October.
    13. Tostado-Véliz, Marcos & Kamel, Salah & Aymen, Flah & Jurado, Francisco, 2022. "A novel hybrid lexicographic-IGDT methodology for robust multi-objective solution of home energy management systems," Energy, Elsevier, vol. 253(C).
    14. Binghui Han & Younes Zahraoui & Marizan Mubin & Saad Mekhilef & Mehdi Seyedmahmoudian & Alex Stojcevski, 2023. "Optimal Strategy for Comfort-Based Home Energy Management System Considering Impact of Battery Degradation Cost Model," Mathematics, MDPI, vol. 11(6), pages 1-26, March.
    15. Ruan, Hebin & Gao, Hongjun & Qiu, Haifeng & Gooi, Hoay Beng & Liu, Junyong, 2023. "Distributed operation optimization of active distribution network with P2P electricity trading in blockchain environment," Applied Energy, Elsevier, vol. 331(C).
    16. Jude Suchithra & Duane Robinson & Amin Rajabi, 2023. "Hosting Capacity Assessment Strategies and Reinforcement Learning Methods for Coordinated Voltage Control in Electricity Distribution Networks: A Review," Energies, MDPI, vol. 16(5), pages 1-28, March.
    17. Vijayan, Vineeth & Mohapatra, Abheejeet & Singh, S.N., 2021. "Demand Response with Volt/Var Optimization for unbalanced active distribution systems," Applied Energy, Elsevier, vol. 300(C).
    18. Buonomano, Annamaria, 2020. "Building to Vehicle to Building concept: A comprehensive parametric and sensitivity analysis for decision making aims," Applied Energy, Elsevier, vol. 261(C).
    19. Chen, Yongdong & Liu, Youbo & Zhao, Junbo & Qiu, Gao & Yin, Hang & Li, Zhengbo, 2023. "Physical-assisted multi-agent graph reinforcement learning enabled fast voltage regulation for PV-rich active distribution network," Applied Energy, Elsevier, vol. 351(C).
    20. Buonomano, A. & Calise, F. & Cappiello, F.L. & Palombo, A. & Vicidomini, M., 2019. "Dynamic analysis of the integration of electric vehicles in efficient buildings fed by renewables," Applied Energy, Elsevier, vol. 245(C), pages 31-50.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:335:y:2023:i:c:s0306261922018864. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.