IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i21p5425-d1510400.html
   My bibliography  Save this article

Reinforcement Learning-Enhanced Adaptive Scheduling of Battery Energy Storage Systems in Energy Markets

Author

Listed:
  • Yang Liu

    (Electric Power Dispatching and Control Center of Guangdong Power Grid Co., Ltd., Guangzhou 510220, China)

  • Qiuyu Lu

    (Electric Power Dispatching and Control Center of Guangdong Power Grid Co., Ltd., Guangzhou 510220, China)

  • Zhenfan Yu

    (Electric Power Dispatching and Control Center of Guangdong Power Grid Co., Ltd., Guangzhou 510220, China)

  • Yue Chen

    (Electric Power Dispatching and Control Center of Guangdong Power Grid Co., Ltd., Guangzhou 510220, China)

  • Yinguo Yang

    (Electric Power Dispatching and Control Center of Guangdong Power Grid Co., Ltd., Guangzhou 510220, China)

Abstract

Battery Energy Storage Systems (BESSs) play a vital role in modern power grids by optimally dispatching energy according to the price signal. This paper proposes a reinforcement learning-based model that optimizes BESS scheduling with the proposed Q-learning algorithm combined with an epsilon-greedy strategy. The proposed epsilon-greedy strategy-based Q-learning algorithm can efficiently manage energy dispatching under uncertain price signals and multi-day operations without retraining. Simulations are conducted under different scenarios, considering electricity price fluctuations and battery aging conditions. Results show that the proposed algorithm demonstrates enhanced economic returns and adaptability compared to traditional methods, providing a practical solution for intelligent BESS scheduling that supports grid stability and the efficient use of renewable energy.

Suggested Citation

  • Yang Liu & Qiuyu Lu & Zhenfan Yu & Yue Chen & Yinguo Yang, 2024. "Reinforcement Learning-Enhanced Adaptive Scheduling of Battery Energy Storage Systems in Energy Markets," Energies, MDPI, vol. 17(21), pages 1-17, October.
    • Handle: RePEc:gam:jeners:v:17:y:2024:i:21:p:5425-:d:1510400
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/21/5425/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/21/5425/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Wu, Peng & Partridge, Julius & Bucknall, Richard, 2020. "Cost-effective reinforcement learning energy management for plug-in hybrid fuel cell and battery ships," Applied Energy, Elsevier, vol. 275(C).
    2. Chen, Xi & Liu, Zhongbing & Wang, Pengcheng & Li, Benjia & Liu, Ruimiao & Zhang, Ling & Zhao, Chengliang & Luo, Songqin, 2023. "Multi-objective optimization of battery capacity of grid-connected PV-BESS system in hybrid building energy sharing community considering time-of-use tariff," Applied Energy, Elsevier, vol. 350(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ziad Ragab & Ehsan Pashajavid & Sumedha Rajakaruna, 2024. "Optimal Sizing and Economic Analysis of Community Battery Systems Considering Sensitivity and Uncertainty Factors," Energies, MDPI, vol. 17(18), pages 1-20, September.
    2. Liu, Hanyou & Fan, Ailong & Li, Yongping & Bucknall, Richard & Chen, Li, 2024. "Hierarchical distributed MPC method for hybrid energy management: A case study of ship with variable operating conditions," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PA).
    3. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    4. Maja Perčić & Nikola Vladimir & Marija Koričan, 2021. "Electrification of Inland Waterway Ships Considering Power System Lifetime Emissions and Costs," Energies, MDPI, vol. 14(21), pages 1-25, October.
    5. Elkholy, M.H. & Metwally, Hamid & Farahat, M.A. & Senjyu, Tomonobu & Elsayed Lotfy, Mohammed, 2022. "Smart centralized energy management system for autonomous microgrid using FPGA," Applied Energy, Elsevier, vol. 317(C).
    6. Yang, Zirong & Jiao, Kui & Wu, Kangcheng & Shi, Weilong & Jiang, Shangfeng & Zhang, Longhai & Du, Qing, 2021. "Numerical investigations of assisted heating cold start strategies for proton exchange membrane fuel cell systems," Energy, Elsevier, vol. 222(C).
    7. Touzani, Samir & Prakash, Anand Krishnan & Wang, Zhe & Agarwal, Shreya & Pritoni, Marco & Kiran, Mariam & Brown, Richard & Granderson, Jessica, 2021. "Controlling distributed energy resources via deep reinforcement learning for load flexibility and energy efficiency," Applied Energy, Elsevier, vol. 304(C).
    8. Xie, Peilin & Tan, Sen & Bazmohammadi, Najmeh & Guerrero, Josep. M. & Vasquez, Juan. C. & Alcala, Jose Matas & Carreño, Jorge El Mariachet, 2022. "A distributed real-time power management scheme for shipboard zonal multi-microgrid system," Applied Energy, Elsevier, vol. 317(C).
    9. Chen, Chunyu & Cui, Mingjian & Fang, Xin & Ren, Bixing & Chen, Yang, 2020. "Load altering attack-tolerant defense strategy for load frequency control system," Applied Energy, Elsevier, vol. 280(C).
    10. Macias, A. & Kandidayeni, M. & Boulon, L. & Trovão, J.P., 2021. "Fuel cell-supercapacitor topologies benchmark for a three-wheel electric vehicle powertrain," Energy, Elsevier, vol. 224(C).
    11. Tsoumpris, Charalampos & Theotokatos, Gerasimos, 2023. "A decision-making approach for the health-aware energy management of ship hybrid power plants," Reliability Engineering and System Safety, Elsevier, vol. 235(C).
    12. Song, Tiewei & Fu, Lijun & Zhong, Linlin & Fan, Yaxiang & Shang, Qianyi, 2024. "HP3O algorithm-based all electric ship energy management strategy integrating demand-side adjustment," Energy, Elsevier, vol. 295(C).
    13. Jonas Quernheim & Eberhard Waffenschmidt, 2024. "A Comparative Evaluation of Community-Used District and Individual Battery Storage Systems for Photovoltaic Energy Systems," Energies, MDPI, vol. 17(17), pages 1-12, August.
    14. Li, Yapeng & Wang, Feng & Tang, Xiaolin & Hu, Xiaosong & Lin, Xianke, 2022. "Convex optimization-based predictive and bi-level energy management for plug-in hybrid electric vehicles," Energy, Elsevier, vol. 257(C).
    15. Chen, Qi & Kuang, Zhonghong & Liu, Xiaohua & Zhang, Tao, 2024. "Optimal sizing and techno-economic analysis of the hybrid PV-battery-cooling storage system for commercial buildings in China," Applied Energy, Elsevier, vol. 355(C).
    16. Perčić, Maja & Frković, Lovro & Pukšec, Tomislav & Ćosić, Boris & Li, Oi Lun & Vladimir, Nikola, 2022. "Life-cycle assessment and life-cycle cost assessment of power batteries for all-electric vessels for short-sea navigation," Energy, Elsevier, vol. 251(C).
    17. Chiara Dall’Armi & Davide Pivetta & Rodolfo Taccani, 2023. "Hybrid PEM Fuel Cell Power Plants Fuelled by Hydrogen for Improving Sustainability in Shipping: State of the Art and Review on Active Projects," Energies, MDPI, vol. 16(4), pages 1-34, February.
    18. Lin, Xinyou & Xu, Xinhao & Wang, Zhaorui, 2022. "Deep Q-learning network based trip pattern adaptive battery longevity-conscious strategy of plug-in fuel cell hybrid electric vehicle," Applied Energy, Elsevier, vol. 321(C).
    19. Feng, Ran & Wang, Kai & Xu, Xu & Yu, Zi-Tao & Lin, Qingyang, 2024. "Triple-layer optimization of distributed photovoltaic energy storage capacity for manufacturing enterprises considering carbon emissions and load management," Applied Energy, Elsevier, vol. 364(C).
    20. Chiara Dall’Armi & Davide Pivetta & Rodolfo Taccani, 2021. "Health-Conscious Optimization of Long-Term Operation for Hybrid PEMFC Ship Propulsion Systems," Energies, MDPI, vol. 14(13), pages 1-20, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:21:p:5425-:d:1510400. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.