IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i12p2883-d1413400.html
   My bibliography  Save this article

Optimizing EV Battery Management: Advanced Hybrid Reinforcement Learning Models for Efficient Charging and Discharging

Author

Listed:
  • Sercan Yalçın

    (Department of Computer Engineering, Adiyaman University, Adiyaman 02040, Turkey)

  • Münür Sacit Herdem

    (Department of Mechanical Engineering, Adiyaman University, Adiyaman 02040, Turkey)

Abstract

This paper investigates the application of hybrid reinforcement learning (RL) models to optimize lithium-ion batteries’ charging and discharging processes in electric vehicles (EVs). By integrating two advanced RL algorithms—deep Q-learning (DQL) and active-critic learning—within the framework of battery management systems (BMSs), this study aims to harness the combined strengths of these techniques to improve battery efficiency, performance, and lifespan. The hybrid models are put through their paces via simulation and experimental validation, demonstrating their capability to devise optimal battery management strategies. These strategies effectively adapt to variations in battery state of health (SOH) and state of charge (SOC) relative error, combat battery voltage aging, and adhere to complex operational constraints, including charging/discharging schedules. The results underscore the potential of RL-based hybrid models to enhance BMSs in EVs, offering tangible contributions towards more sustainable and reliable electric transportation systems.

Suggested Citation

  • Sercan Yalçın & Münür Sacit Herdem, 2024. "Optimizing EV Battery Management: Advanced Hybrid Reinforcement Learning Models for Efficient Charging and Discharging," Energies, MDPI, vol. 17(12), pages 1-21, June.
    • Handle: RePEc:gam:jeners:v:17:y:2024:i:12:p:2883-:d:1413400
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/12/2883/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/12/2883/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Michael Schneider & Andreas Stenger & Dominik Goeke, 2014. "The Electric Vehicle-Routing Problem with Time Windows and Recharging Stations," Transportation Science, INFORMS, vol. 48(4), pages 500-520, November.
    2. Hairui Wang & Xin Ye & Yuanbo Li & Guifu Zhu, 2023. "Remaining Useful Life Prediction for Lithium-Ion Batteries Based on Improved Mode Decomposition and Time Series," Sustainability, MDPI, vol. 15(12), pages 1-23, June.
    3. Xuemei Li & Hao Chang & Ruichao Wei & Shenshi Huang & Shaozhang Chen & Zhiwei He & Dongxu Ouyang, 2023. "Online Prediction of Electric Vehicle Battery Failure Using LSTM Network," Energies, MDPI, vol. 16(12), pages 1-14, June.
    4. Xu, Bin & Rathod, Dhruvang & Zhang, Darui & Yebi, Adamu & Zhang, Xueyu & Li, Xiaoya & Filipi, Zoran, 2020. "Parametric study on reinforcement learning optimized energy management strategy for a hybrid electric vehicle," Applied Energy, Elsevier, vol. 259(C).
    5. Du, Guodong & Zou, Yuan & Zhang, Xudong & Kong, Zehui & Wu, Jinlong & He, Dingbo, 2019. "Intelligent energy management for hybrid electric tracked vehicles using online reinforcement learning," Applied Energy, Elsevier, vol. 251(C), pages 1-1.
    6. Edoardo Lelli & Alessia Musa & Emilio Batista & Daniela Anna Misul & Giovanni Belingardi, 2023. "On-Road Experimental Campaign for Machine Learning Based State of Health Estimation of High-Voltage Batteries in Electric Vehicles," Energies, MDPI, vol. 16(12), pages 1-21, June.
    7. Schneider, M. & Stenger, A. & Goeke, D., 2014. "The Electric Vehicle Routing Problem with Time Windows and Recharging Stations," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 62382, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    8. Meng, Huixing & Geng, Mengyao & Han, Te, 2023. "Long short-term memory network with Bayesian optimization for health prognostics of lithium-ion batteries based on partial incremental capacity analysis," Reliability Engineering and System Safety, Elsevier, vol. 236(C).
    9. Zhongda Lu & Qilong Wang & Fengxia Xu & Mingqing Fan & Chuanshui Peng & Shiwei Yan, 2023. "Double-Layer SOC and SOH Equalization Scheme for LiFePO 4 Battery Energy Storage System Using MAS Blackboard System," Energies, MDPI, vol. 16(14), pages 1-14, July.
    10. Wei Ji & Xiaoqing Liu & Huijun Qi & Xunnan Liu & Chaoning Lin & Tongchun Li, 2020. "Mechanical Parameter Identification of Hydraulic Engineering with the Improved Deep Q-Network Algorithm," Mathematical Problems in Engineering, Hindawi, vol. 2020, pages 1-20, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roel M. Post & Paul Buijs & Michiel A. J. uit het Broek & Jose A. Lopez Alvarez & Nick B. Szirbik & Iris F. A. Vis, 2018. "A solution approach for deriving alternative fuel station infrastructure requirements," Flexible Services and Manufacturing Journal, Springer, vol. 30(3), pages 592-607, September.
    2. Bogyrbayeva, Aigerim & Kwon, Changhyun, 2021. "Pessimistic evasive flow capturing problems," European Journal of Operational Research, Elsevier, vol. 293(1), pages 133-148.
    3. Yang, Jun & Guo, Fang & Zhang, Min, 2017. "Optimal planning of swapping/charging station network with customer satisfaction," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 103(C), pages 174-197.
    4. Ana Bricia Galindo-Muro & Riccardo Cespi & Stephany Isabel Vallarta-Serrano, 2023. "Applications of Electric Vehicles in Instant Deliveries," Energies, MDPI, vol. 16(4), pages 1-18, February.
    5. Gitae Kim, 2024. "Electric Vehicle Routing Problem with States of Charging Stations," Sustainability, MDPI, vol. 16(8), pages 1-17, April.
    6. Su, Yue & Dupin, Nicolas & Parragh, Sophie N. & Puchinger, Jakob, 2024. "A Branch-and-Price algorithm for the electric autonomous Dial-A-Ride Problem," Transportation Research Part B: Methodological, Elsevier, vol. 186(C).
    7. Baals, Julian & Emde, Simon & Turkensteen, Marcel, 2023. "Minimizing earliness-tardiness costs in supplier networks—A just-in-time truck routing problem," European Journal of Operational Research, Elsevier, vol. 306(2), pages 707-741.
    8. Timothy M. Sweda & Irina S. Dolinskaya & Diego Klabjan, 2017. "Adaptive Routing and Recharging Policies for Electric Vehicles," Transportation Science, INFORMS, vol. 51(4), pages 1326-1348, November.
    9. Raeesi, Ramin & Zografos, Konstantinos G., 2020. "The electric vehicle routing problem with time windows and synchronised mobile battery swapping," Transportation Research Part B: Methodological, Elsevier, vol. 140(C), pages 101-129.
    10. Cortés-Murcia, David L. & Prodhon, Caroline & Murat Afsar, H., 2019. "The electric vehicle routing problem with time windows, partial recharges and satellite customers," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 130(C), pages 184-206.
    11. Snežana Tadić & Mladen Krstić & Ljubica Radovanović, 2024. "Assessing Strategies to Overcome Barriers for Drone Usage in Last-Mile Logistics: A Novel Hybrid Fuzzy MCDM Model," Mathematics, MDPI, vol. 12(3), pages 1-25, January.
    12. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    13. Han, Lijin & You, Congwen & Yang, Ningkang & Liu, Hui & Chen, Ke & Xiang, Changle, 2024. "Adaptive real-time energy management strategy using heuristic search for off-road hybrid electric vehicles," Energy, Elsevier, vol. 304(C).
    14. Ramirez Ibarra, Monica & Saphores, Jean-Daniel M., 2023. "1,000 HP electric drayage trucks as a substitute for new freeway lanes construction," Transportation Research Part A: Policy and Practice, Elsevier, vol. 171(C).
    15. Raeesi, Ramin & Zografos, Konstantinos G., 2022. "Coordinated routing of electric commercial vehicles with intra-route recharging and en-route battery swapping," European Journal of Operational Research, Elsevier, vol. 301(1), pages 82-109.
    16. Azra Ghobadi & Mohammad Fallah & Reza Tavakkoli-Moghaddam & Hamed Kazemipoor, 2022. "A Fuzzy Two-Echelon Model to Optimize Energy Consumption in an Urban Logistics Network with Electric Vehicles," Sustainability, MDPI, vol. 14(21), pages 1-31, October.
    17. Cárdenas-Barrón, Leopoldo Eduardo & González-Velarde, José Luis & Treviño-Garza, Gerardo & Garza-Nuñez, Dagoberto, 2019. "Heuristic algorithm based on reduce and optimize approach for a selective and periodic inventory routing problem in a waste vegetable oil collection environment," International Journal of Production Economics, Elsevier, vol. 211(C), pages 44-59.
    18. Koyuncu, Işıl & Yavuz, Mesut, 2019. "Duplicating nodes or arcs in green vehicle routing: A computational comparison of two formulations," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 122(C), pages 605-623.
    19. Imen Ben Mohamed & Walid Klibi & Olivier Labarthe & Jean-Christophe Deschamps & Mohamed Zied Babai, 2017. "Modelling and solution approaches for the interconnected city logistics," International Journal of Production Research, Taylor & Francis Journals, vol. 55(9), pages 2664-2684, May.
    20. Amirmahdi Tafreshian & Neda Masoud & Yafeng Yin, 2020. "Frontiers in Service Science: Ride Matching for Peer-to-Peer Ride Sharing: A Review and Future Directions," Service Science, INFORMS, vol. 12(2-3), pages 44-60, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:12:p:2883-:d:1413400. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.