IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i2p614-d725819.html
   My bibliography  Save this article

Active Exploration by Chance-Constrained Optimization for Voltage Regulation with Reinforcement Learning

Author

Listed:
  • Zhenhuan Ding

    (Department of Electrical and Computer Engineering, State University of New York at Binghamton, New York, NY 13902, USA)

  • Xiaoge Huang

    (Department of Electrical and Computer Engineering, State University of New York at Binghamton, New York, NY 13902, USA)

  • Zhao Liu

    (School of Electrical Engineering, Beijing Jiaotong University, Beijing 100044, China)

Abstract

Voltage regulation in distribution networks encounters a challenge of handling uncertainties caused by the high penetration of photovoltaics (PV). This research proposes an active exploration (AE) method based on reinforcement learning (RL) to respond to the uncertainties by regulating the voltage of a distribution network with battery energy storage systems (BESS). The proposed method integrates engineering knowledge to accelerate the training process of RL. The engineering knowledge is the chance-constrained optimization. We formulate the problem in a chance-constrained optimization with a linear load flow approximation. The optimization results are used to guide the action selection of the exploration for improving training efficiency and reducing the conserveness characteristic. The comparison of methods focuses on how BESSs are used, training efficiency, and robustness under varying uncertainties and BESS sizes. We implement the proposed algorithm, a chance-constrained optimization, and a traditional Q-learning in the IEEE 13 Node Test Feeder. Our evaluation shows that the proposed AE method has a better response to the training efficiency compared to traditional Q-learning. Meanwhile, the proposed method has advantages in BESS usage in conserveness compared to the chance-constrained optimization.

Suggested Citation

  • Zhenhuan Ding & Xiaoge Huang & Zhao Liu, 2022. "Active Exploration by Chance-Constrained Optimization for Voltage Regulation with Reinforcement Learning," Energies, MDPI, vol. 15(2), pages 1-17, January.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:2:p:614-:d:725819
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/2/614/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/2/614/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ying-Yi Hong & Gerard Francesco DG. Apolinario, 2021. "Uncertainty in Unit Commitment in Power Systems: A Review of Models, Methods, and Applications," Energies, MDPI, vol. 14(20), pages 1-47, October.
    2. A.S. Jameel Hassan & Umar Marikkar & G.W. Kasun Prabhath & Aranee Balachandran & W.G. Chaminda Bandara & Parakrama B. Ekanayake & Roshan I. Godaliyadda & Janaka B. Ekanayake, 2021. "A Sensitivity Matrix Approach Using Two-Stage Optimization for Voltage Regulation of LV Networks with High PV Penetration," Energies, MDPI, vol. 14(20), pages 1-24, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. James Amankwah Adu & Alberto Berizzi & Francesco Conte & Fabio D’Agostino & Valentin Ilea & Fabio Napolitano & Tadeo Pontecorvo & Andrea Vicario, 2022. "Power System Stability Analysis of the Sicilian Network in the 2050 OSMOSE Project Scenario," Energies, MDPI, vol. 15(10), pages 1-33, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luis Montero & Antonio Bello & Javier Reneses, 2022. "A Review on the Unit Commitment Problem: Approaches, Techniques, and Resolution Methods," Energies, MDPI, vol. 15(4), pages 1-40, February.
    2. Donovin D. Lewis & Aron Patrick & Evan S. Jones & Rosemary E. Alden & Abdullah Al Hadi & Malcolm D. McCulloch & Dan M. Ionel, 2023. "Decarbonization Analysis for Thermal Generation and Regionally Integrated Large-Scale Renewables Based on Minutely Optimal Dispatch with a Kentucky Case Study," Energies, MDPI, vol. 16(4), pages 1-23, February.
    3. Stover, Oliver & Karve, Pranav & Mahadevan, Sankaran, 2023. "Reliability and risk metrics to assess operational adequacy and flexibility of power grids," Reliability Engineering and System Safety, Elsevier, vol. 231(C).
    4. Juseung Choi & Hoyong Eom & Seung-Mook Baek, 2022. "A Wind Power Probabilistic Model Using the Reflection Method and Multi-Kernel Function Kernel Density Estimation," Energies, MDPI, vol. 15(24), pages 1-17, December.
    5. Zhang, Qian & Qi, Jingwen & Zhen, Lu, 2023. "Optimization of integrated energy system considering multi-energy collaboration in carbon-free hydrogen port," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 180(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:2:p:614-:d:725819. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.