IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v14y2021i12p3540-d574791.html
   My bibliography  Save this article

Deep-Reinforcement-Learning-Based Two-Timescale Voltage Control for Distribution Systems

Author

Listed:
  • Jing Zhang

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China)

  • Yiqi Li

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China)

  • Zhi Wu

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China)

  • Chunyan Rong

    (Institute of State Grid Hebei Electric Power Company Economic and Technological Research, Shijiazhuang 050000, China)

  • Tao Wang

    (Institute of State Grid Hebei Electric Power Company Economic and Technological Research, Shijiazhuang 050000, China)

  • Zhang Zhang

    (Institute of State Grid Hebei Electric Power Company Economic and Technological Research, Shijiazhuang 050000, China)

  • Suyang Zhou

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China)

Abstract

Because of the high penetration of renewable energies and the installation of new control devices, modern distribution networks are faced with voltage regulation challenges. Recently, the rapid development of artificial intelligence technology has introduced new solutions for optimal control problems with high dimensions and dynamics. In this paper, a deep reinforcement learning method is proposed to solve the two-timescale optimal voltage control problem. All control variables are assigned to different agents, and discrete variables are solved by a deep Q network (DQN) agent while the continuous variables are solved by a deep deterministic policy gradient (DDPG) agent. All agents are trained simultaneously with specially designed reward aiming at minimizing long-term average voltage deviation. Case study is executed on a modified IEEE-123 bus system, and the results demonstrate that the proposed algorithm has similar or even better performance than the model-based optimal control scheme and has high computational efficiency and competitive potential for online application.

Suggested Citation

  • Jing Zhang & Yiqi Li & Zhi Wu & Chunyan Rong & Tao Wang & Zhang Zhang & Suyang Zhou, 2021. "Deep-Reinforcement-Learning-Based Two-Timescale Voltage Control for Distribution Systems," Energies, MDPI, vol. 14(12), pages 1-15, June.
    • Handle: RePEc:gam:jeners:v:14:y:2021:i:12:p:3540-:d:574791
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/14/12/3540/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/14/12/3540/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kirstin Beyer & Robert Beckmann & Stefan Geißendörfer & Karsten von Maydell & Carsten Agert, 2021. "Adaptive Online-Learning Volt-Var Control for Smart Inverters Using Deep Reinforcement Learning," Energies, MDPI, vol. 14(7), pages 1-11, April.
    2. Oleh Lukianykhin & Tetiana Bogodorova, 2021. "Voltage Control-Based Ancillary Service Using Deep Reinforcement Learning," Energies, MDPI, vol. 14(8), pages 1-22, April.
    3. Mojgan Hojabri & Ulrich Dersch & Antonios Papaemmanouil & Peter Bosshart, 2019. "A Comprehensive Survey on Phasor Measurement Unit Applications in Distribution Systems," Energies, MDPI, vol. 12(23), pages 1-23, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Qingyan Li & Tao Lin & Qianyi Yu & Hui Du & Jun Li & Xiyue Fu, 2023. "Review of Deep Reinforcement Learning and Its Application in Modern Renewable Power System Control," Energies, MDPI, vol. 16(10), pages 1-23, May.
    2. Di Liu & Junwei Cao & Mingshuang Liu, 2022. "Joint Optimization of Energy Storage Sharing and Demand Response in Microgrid Considering Multiple Uncertainties," Energies, MDPI, vol. 15(9), pages 1-20, April.
    3. Zhang, Xiao & Wu, Zhi & Sun, Qirun & Gu, Wei & Zheng, Shu & Zhao, Jingtao, 2024. "Application and progress of artificial intelligence technology in the field of distribution network voltage Control：A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 192(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chinmayee Biswal & Binod Kumar Sahu & Manohar Mishra & Pravat Kumar Rout, 2023. "Real-Time Grid Monitoring and Protection: A Comprehensive Survey on the Advantages of Phasor Measurement Units," Energies, MDPI, vol. 16(10), pages 1-34, May.
    2. David Granados-Lieberman, 2020. "Global Harmonic Parameters for Estimation of Power Quality Indices: An Approach for PMUs," Energies, MDPI, vol. 13(9), pages 1-17, May.
    3. Do-In Kim, 2021. "Complementary Feature Extractions for Event Identification in Power Systems Using Multi-Channel Convolutional Neural Network," Energies, MDPI, vol. 14(15), pages 1-15, July.
    4. Franz Harke & Philipp Otto, 2023. "Solar Self-Sufficient Households as a Driving Factor for Sustainability Transformation," Sustainability, MDPI, vol. 15(3), pages 1-20, February.
    5. Karthikeyan Subramanian & Ashok Kumar Loganathan, 2020. "Islanding Detection Using a Micro-Synchrophasor for Distribution Systems with Distributed Generation," Energies, MDPI, vol. 13(19), pages 1-31, October.
    6. Alessandro Mingotti & Federica Costa & Lorenzo Peretto & Roberto Tinarelli, 2021. "Closed-Form Expressions to Estimate the Mean and Variance of the Total Vector Error," Energies, MDPI, vol. 14(15), pages 1-15, July.
    7. Ode Bokker & Henning Schlachter & Vanessa Beutel & Stefan Geißendörfer & Karsten von Maydell, 2022. "Reactive Power Control of a Converter in a Hardware-Based Environment Using Deep Reinforcement Learning," Energies, MDPI, vol. 16(1), pages 1-12, December.
    8. Yu Fujimoto & Akihisa Kaneko & Yutaka Iino & Hideo Ishii & Yasuhiro Hayashi, 2023. "Challenges in Smartizing Operational Management of Functionally-Smart Inverters for Distributed Energy Resources: A Review on Machine Learning Aspects," Energies, MDPI, vol. 16(3), pages 1-26, January.
    9. Nikolaos-Antonios I. Livanos & Sami Hammal & Nikolaos Giamarelos & Vagelis Alifragkis & Constantinos S. Psomopoulos & Elias N. Zois, 2023. "OpenEdgePMU: An Open PMU Architecture with Edge Processing for Future Resilient Smart Grids," Energies, MDPI, vol. 16(6), pages 1-29, March.
    10. Muhammad Musadiq Ahmed & Muhammad Amjad & Muhammad Ali Qureshi & Kashif Imran & Zunaib Maqsood Haider & Muhammad Omer Khan, 2022. "A Critical Review of State-of-the-Art Optimal PMU Placement Techniques," Energies, MDPI, vol. 15(6), pages 1-25, March.
    11. Giovanni Artale & Giuseppe Caravello & Antonio Cataliotti & Valentina Cosentino & Dario Di Cara & Salvatore Guaiana & Ninh Nguyen Quang & Marco Palmeri & Nicola Panzavecchia & Giovanni Tinè, 2020. "A Virtual Tool for Load Flow Analysis in a Micro-Grid," Energies, MDPI, vol. 13(12), pages 1-26, June.
    12. Khaoula Hassini & Ahmed Fakhfakh & Faouzi Derbel, 2023. "Optimal Placement of μ PMUs in Distribution Networks with Adaptive Topology Changes," Energies, MDPI, vol. 16(20), pages 1-27, October.
    13. Mussawir Ul Mehmood & Abasin Ulasyar & Abraiz Khattak & Kashif Imran & Haris Sheh Zad & Shibli Nisar, 2020. "Cloud Based IoT Solution for Fault Detection and Localization in Power Distribution Systems," Energies, MDPI, vol. 13(11), pages 1-19, May.
    14. Phylicia Cicilio & David Glennon & Adam Mate & Arthur Barnes & Vishvas Chalishazar & Eduardo Cotilla-Sanchez & Bjorn Vaagensmith & Jake Gentle & Craig Rieger & Richard Wies & Mohammad Heidari Kapourch, 2021. "Resilience in an Evolving Electrical Grid," Energies, MDPI, vol. 14(3), pages 1-25, January.
    15. Jarosław Korpikiewicz & Mostefa Mohamed-Seghir, 2022. "Static Analysis and Optimization of Voltage and Reactive Power Regulation Systems in the HV/MV Substation with Electronic Transformer Tap-Changers," Energies, MDPI, vol. 15(13), pages 1-26, June.
    16. Zunaib Ali & Komal Saleem & Robert Brown & Nicholas Christofides & Sandra Dudley, 2022. "Performance Analysis and Benchmarking of PLL-Driven Phasor Measurement Units for Renewable Energy Systems," Energies, MDPI, vol. 15(5), pages 1-22, March.
    17. Ivana Damjanović & Ivica Pavić & Mate Puljiz & Mario Brcic, 2022. "Deep Reinforcement Learning-Based Approach for Autonomous Power Flow Control Using Only Topology Changes," Energies, MDPI, vol. 15(19), pages 1-16, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:14:y:2021:i:12:p:3540-:d:574791. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.