IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v16y2023i14p5369-d1194014.html
   My bibliography  Save this article

Model-Free Approach to DC Microgrid Optimal Operation under System Uncertainty Based on Reinforcement Learning

Author

Listed:
  • Roni Irnawan

    (Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia
    Center for Energy Studies, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia)

  • Ahmad Ataka Awwalur Rizqi

    (Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia)

  • Muhammad Yasirroni

    (Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia)

  • Lesnanto Multa Putranto

    (Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia
    Center for Energy Studies, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia)

  • Husni Rois Ali

    (Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia)

  • Eka Firmansyah

    (Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia)

  • Sarjiya

    (Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia
    Center for Energy Studies, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia)

Abstract

There has been tremendous interest in the development of DC microgrid systems which consist of interconnected DC renewable energy sources. However, operating a DC microgrid system optimally by minimizing operational cost and ensures stability remains a problem when the system’s model is not available. In this paper, a novel model-free approach to perform operation control of DC microgrids based on reinforcement learning algorithms, specifically Q-learning and Q-network, has been proposed. This approach circumvents the need to know the accurate model of a DC grid by exploiting an interaction with the DC microgrids to learn the best policy, which leads to more optimal operation. The proposed approach has been compared with with mixed-integer quadratic programming (MIQP) as the baseline deterministic model that requires an accurate system model. The result shows that, in a system of three nodes, both Q-learning (74.2707) and Q-network (74.4254) are able to learn to make a control decision that is close to the MIQP (75.0489) solution. With the introduction of both model uncertainty and noisy sensor measurements, the Q-network performs better (72.3714) compared to MIQP (72.1596), whereas Q-learn fails to learn.

Suggested Citation

  • Roni Irnawan & Ahmad Ataka Awwalur Rizqi & Muhammad Yasirroni & Lesnanto Multa Putranto & Husni Rois Ali & Eka Firmansyah & Sarjiya, 2023. "Model-Free Approach to DC Microgrid Optimal Operation under System Uncertainty Based on Reinforcement Learning," Energies, MDPI, vol. 16(14), pages 1-20, July.
    • Handle: RePEc:gam:jeners:v:16:y:2023:i:14:p:5369-:d:1194014
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/16/14/5369/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/16/14/5369/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Gholam Ali Alizadeh & Tohid Rahimi & Mohsen Hasan Babayi Nozadian & Sanjeevikumar Padmanaban & Zbigniew Leonowicz, 2019. "Improving Microgrid Frequency Regulation Based on the Virtual Inertia Concept while Considering Communication System Delay," Energies, MDPI, vol. 12(10), pages 1-15, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Anshuman Satapathy & Niranjan Nayak & Tanmoy Parida, 2022. "Real-Time Power Quality Enhancement in a Hybrid Micro-Grid Using Nonlinear Autoregressive Neural Network," Energies, MDPI, vol. 15(23), pages 1-35, November.
    2. Pierluigi Siano & Miadreza Shafie-khah, 2020. "Special Issue on Advanced Approaches, Business Models, and Novel Techniques for Management and Control of Smart Grids," Energies, MDPI, vol. 13(11), pages 1-3, May.
    3. Giulio Ferro & Michela Robba & Roberto Sacile, 2020. "A Model Predictive Control Strategy for Distribution Grids: Voltage and Frequency Regulation for Islanded Mode Operation," Energies, MDPI, vol. 13(10), pages 1-27, May.
    4. Ziba Rostami & Sajad Najafi Ravadanegh & Navid Taghizadegan Kalantari & Josep M. Guerrero & Juan C. Vasquez, 2020. "Dynamic Modeling of Multiple Microgrid Clusters Using Regional Demand Response Programs," Energies, MDPI, vol. 13(16), pages 1-19, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:16:y:2023:i:14:p:5369-:d:1194014. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.