IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i12p2883-d1413400.html
   My bibliography  Save this article

Optimizing EV Battery Management: Advanced Hybrid Reinforcement Learning Models for Efficient Charging and Discharging

Author

Listed:
  • Sercan Yalçın

    (Department of Computer Engineering, Adiyaman University, Adiyaman 02040, Turkey)

  • Münür Sacit Herdem

    (Department of Mechanical Engineering, Adiyaman University, Adiyaman 02040, Turkey)

Abstract

This paper investigates the application of hybrid reinforcement learning (RL) models to optimize lithium-ion batteries’ charging and discharging processes in electric vehicles (EVs). By integrating two advanced RL algorithms—deep Q-learning (DQL) and active-critic learning—within the framework of battery management systems (BMSs), this study aims to harness the combined strengths of these techniques to improve battery efficiency, performance, and lifespan. The hybrid models are put through their paces via simulation and experimental validation, demonstrating their capability to devise optimal battery management strategies. These strategies effectively adapt to variations in battery state of health (SOH) and state of charge (SOC) relative error, combat battery voltage aging, and adhere to complex operational constraints, including charging/discharging schedules. The results underscore the potential of RL-based hybrid models to enhance BMSs in EVs, offering tangible contributions towards more sustainable and reliable electric transportation systems.

Suggested Citation

  • Sercan Yalçın & Münür Sacit Herdem, 2024. "Optimizing EV Battery Management: Advanced Hybrid Reinforcement Learning Models for Efficient Charging and Discharging," Energies, MDPI, vol. 17(12), pages 1-21, June.
    • Handle: RePEc:gam:jeners:v:17:y:2024:i:12:p:2883-:d:1413400
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/12/2883/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/12/2883/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Edoardo Lelli & Alessia Musa & Emilio Batista & Daniela Anna Misul & Giovanni Belingardi, 2023. "On-Road Experimental Campaign for Machine Learning Based State of Health Estimation of High-Voltage Batteries in Electric Vehicles," Energies, MDPI, vol. 16(12), pages 1-21, June.
    2. Michael Schneider & Andreas Stenger & Dominik Goeke, 2014. "The Electric Vehicle-Routing Problem with Time Windows and Recharging Stations," Transportation Science, INFORMS, vol. 48(4), pages 500-520, November.
    3. Hairui Wang & Xin Ye & Yuanbo Li & Guifu Zhu, 2023. "Remaining Useful Life Prediction for Lithium-Ion Batteries Based on Improved Mode Decomposition and Time Series," Sustainability, MDPI, vol. 15(12), pages 1-23, June.
    4. Schneider, M. & Stenger, A. & Goeke, D., 2014. "The Electric Vehicle Routing Problem with Time Windows and Recharging Stations," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 62382, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    5. Xuemei Li & Hao Chang & Ruichao Wei & Shenshi Huang & Shaozhang Chen & Zhiwei He & Dongxu Ouyang, 2023. "Online Prediction of Electric Vehicle Battery Failure Using LSTM Network," Energies, MDPI, vol. 16(12), pages 1-14, June.
    6. Meng, Huixing & Geng, Mengyao & Han, Te, 2023. "Long short-term memory network with Bayesian optimization for health prognostics of lithium-ion batteries based on partial incremental capacity analysis," Reliability Engineering and System Safety, Elsevier, vol. 236(C).
    7. Xu, Bin & Rathod, Dhruvang & Zhang, Darui & Yebi, Adamu & Zhang, Xueyu & Li, Xiaoya & Filipi, Zoran, 2020. "Parametric study on reinforcement learning optimized energy management strategy for a hybrid electric vehicle," Applied Energy, Elsevier, vol. 259(C).
    8. Zhongda Lu & Qilong Wang & Fengxia Xu & Mingqing Fan & Chuanshui Peng & Shiwei Yan, 2023. "Double-Layer SOC and SOH Equalization Scheme for LiFePO 4 Battery Energy Storage System Using MAS Blackboard System," Energies, MDPI, vol. 16(14), pages 1-14, July.
    9. Wei Ji & Xiaoqing Liu & Huijun Qi & Xunnan Liu & Chaoning Lin & Tongchun Li, 2020. "Mechanical Parameter Identification of Hydraulic Engineering with the Improved Deep Q-Network Algorithm," Mathematical Problems in Engineering, Hindawi, vol. 2020, pages 1-20, December.
    10. Du, Guodong & Zou, Yuan & Zhang, Xudong & Kong, Zehui & Wu, Jinlong & He, Dingbo, 2019. "Intelligent energy management for hybrid electric tracked vehicles using online reinforcement learning," Applied Energy, Elsevier, vol. 251(C), pages 1-1.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhou, Yu & Meng, Qiang & Ong, Ghim Ping, 2022. "Electric Bus Charging Scheduling for a Single Public Transport Route Considering Nonlinear Charging Profile and Battery Degradation Effect," Transportation Research Part B: Methodological, Elsevier, vol. 159(C), pages 49-75.
    2. Roel M. Post & Paul Buijs & Michiel A. J. uit het Broek & Jose A. Lopez Alvarez & Nick B. Szirbik & Iris F. A. Vis, 2018. "A solution approach for deriving alternative fuel station infrastructure requirements," Flexible Services and Manufacturing Journal, Springer, vol. 30(3), pages 592-607, September.
    3. Dessouky, Maged M & Shao, Yihuan E, 2017. "Routing Strategies for Efficient Deployment of Alternative Fuel Vehicles for Freight Delivery," Institute of Transportation Studies, Working Paper Series qt0nj024qn, Institute of Transportation Studies, UC Davis.
    4. Yang, Ningkang & Han, Lijin & Xiang, Changle & Liu, Hui & Li, Xunmin, 2021. "An indirect reinforcement learning based real-time energy management strategy via high-order Markov Chain model for a hybrid electric vehicle," Energy, Elsevier, vol. 236(C).
    5. Bogyrbayeva, Aigerim & Kwon, Changhyun, 2021. "Pessimistic evasive flow capturing problems," European Journal of Operational Research, Elsevier, vol. 293(1), pages 133-148.
    6. Yang, Jun & Guo, Fang & Zhang, Min, 2017. "Optimal planning of swapping/charging station network with customer satisfaction," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 103(C), pages 174-197.
    7. Ana Bricia Galindo-Muro & Riccardo Cespi & Stephany Isabel Vallarta-Serrano, 2023. "Applications of Electric Vehicles in Instant Deliveries," Energies, MDPI, vol. 16(4), pages 1-18, February.
    8. Gitae Kim, 2024. "Electric Vehicle Routing Problem with States of Charging Stations," Sustainability, MDPI, vol. 16(8), pages 1-17, April.
    9. Roberti, R. & Wen, M., 2016. "The Electric Traveling Salesman Problem with Time Windows," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 89(C), pages 32-52.
    10. Tengkuo Zhu & Stephen D. Boyles & Avinash Unnikrishnan, 2024. "Battery Electric Vehicle Traveling Salesman Problem with Drone," Networks and Spatial Economics, Springer, vol. 24(1), pages 49-97, March.
    11. Baals, Julian & Emde, Simon & Turkensteen, Marcel, 2023. "Minimizing earliness-tardiness costs in supplier networks—A just-in-time truck routing problem," European Journal of Operational Research, Elsevier, vol. 306(2), pages 707-741.
    12. Timothy M. Sweda & Irina S. Dolinskaya & Diego Klabjan, 2017. "Adaptive Routing and Recharging Policies for Electric Vehicles," Transportation Science, INFORMS, vol. 51(4), pages 1326-1348, November.
    13. Raeesi, Ramin & Zografos, Konstantinos G., 2020. "The electric vehicle routing problem with time windows and synchronised mobile battery swapping," Transportation Research Part B: Methodological, Elsevier, vol. 140(C), pages 101-129.
    14. Cortés-Murcia, David L. & Prodhon, Caroline & Murat Afsar, H., 2019. "The electric vehicle routing problem with time windows, partial recharges and satellite customers," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 130(C), pages 184-206.
    15. Snežana Tadić & Mladen Krstić & Ljubica Radovanović, 2024. "Assessing Strategies to Overcome Barriers for Drone Usage in Last-Mile Logistics: A Novel Hybrid Fuzzy MCDM Model," Mathematics, MDPI, vol. 12(3), pages 1-25, January.
    16. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    17. Yang, Ningkang & Han, Lijin & Bo, Lin & Liu, Baoshuai & Chen, Xiuqi & Liu, Hui & Xiang, Changle, 2023. "Real-time adaptive energy management for off-road hybrid electric vehicles based on decision-time planning," Energy, Elsevier, vol. 282(C).
    18. Alberto Ceselli & Ángel Felipe & M. Teresa Ortuño & Giovanni Righini & Gregorio Tirado, 2021. "A Branch-and-Cut-and-Price Algorithm for the Electric Vehicle Routing Problem with Multiple Technologies," SN Operations Research Forum, Springer, vol. 2(1), pages 1-33, March.
    19. Maximilian Schiffer & Michael Schneider & Grit Walther & Gilbert Laporte, 2019. "Vehicle Routing and Location Routing with Intermediate Stops: A Review," Transportation Science, INFORMS, vol. 53(2), pages 319-343, March.
    20. Yıldız, Barış & Arslan, Okan & Karaşan, Oya Ekin, 2016. "A branch and price approach for routing and refueling station location model," European Journal of Operational Research, Elsevier, vol. 248(3), pages 815-826.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:12:p:2883-:d:1413400. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.