IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v229y2021ics0360544221009531.html
   My bibliography  Save this article

Energy consumption and battery aging minimization using a Q-learning strategy for a battery/ultracapacitor electric vehicle

Author

Listed:
  • Xu, Bin
  • Shi, Junzhe
  • Li, Sixu
  • Li, Huayi
  • Wang, Zhe

Abstract

Propulsion system electrification revolution has been undergoing in the automotive industry. The electrified propulsion system improves energy efficiency and reduces the dependence on fossil fuel. However, the batteries of electric vehicles experience degradation process during vehicle operation. Research considering both battery degradation and energy consumption in battery/ultracapacitor electric vehicles is still lacking. This study proposes a Q-learning-based strategy to minimize battery degradation and energy consumption. Besides Q-learning, two rule-based energy management methods are also proposed and optimized using Particle Swarm Optimization algorithm. A vehicle propulsion system model is first presented, where the severity factor battery degradation model is considered and experimentally validated with the help of Genetic Algorithm. In the results analysis, Q-learning is first explained with the optimal policy map after learning. Then, the result from a vehicle without ultracapacitor is used as the baseline, which is compared with the results from the vehicle with ultracapacitor using Q-learning, and two rule-based methods as the energy management strategies. At the learning and validation driving cycles, the results indicate that the Q-learning strategy slows down the battery degradation by 13–20% and increases the vehicle range by 1.5–2% compared with the baseline vehicle without ultracapacitor.

Suggested Citation

  • Xu, Bin & Shi, Junzhe & Li, Sixu & Li, Huayi & Wang, Zhe, 2021. "Energy consumption and battery aging minimization using a Q-learning strategy for a battery/ultracapacitor electric vehicle," Energy, Elsevier, vol. 229(C).
    • Handle: RePEc:eee:energy:v:229:y:2021:i:c:s0360544221009531
    DOI: 10.1016/j.energy.2021.120705
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544221009531
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2021.120705?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Xu, Bin & Rathod, Dhruvang & Zhang, Darui & Yebi, Adamu & Zhang, Xueyu & Li, Xiaoya & Filipi, Zoran, 2020. "Parametric study on reinforcement learning optimized energy management strategy for a hybrid electric vehicle," Applied Energy, Elsevier, vol. 259(C).
    2. Xiong, Rui & Duan, Yanzhou & Cao, Jiayi & Yu, Quanqing, 2018. "Battery and ultracapacitor in-the-loop approach to validate a real-time power management method for an all-climate electric vehicle," Applied Energy, Elsevier, vol. 217(C), pages 153-165.
    3. Arumugam Manthiram, 2020. "A reflection on lithium-ion battery cathode chemistry," Nature Communications, Nature, vol. 11(1), pages 1-9, December.
    4. Suri, Girish & Onori, Simona, 2016. "A control-oriented cycle-life model for hybrid electric vehicle lithium-ion batteries," Energy, Elsevier, vol. 96(C), pages 644-653.
    5. Kim, Youngki & Figueroa-Santos, Miriam & Prakash, Niket & Baek, Stanley & Siegel, Jason B. & Rizzo, Denise M., 2020. "Co-optimization of speed trajectory and power management for a fuel-cell/battery electric vehicle," Applied Energy, Elsevier, vol. 260(C).
    6. Pelletier, Samuel & Jabali, Ola & Laporte, Gilbert & Veneroni, Marco, 2017. "Battery degradation and behaviour for electric vehicles: Review and numerical analyses of several models," Transportation Research Part B: Methodological, Elsevier, vol. 103(C), pages 158-187.
    7. Cong Zhang & Dai Wang & Bin Wang & Fan Tong, 2020. "Battery Degradation Minimization-Oriented Hybrid Energy Storage System for Electric Vehicles," Energies, MDPI, vol. 13(1), pages 1-21, January.
    8. Fan Yang & Yuanyuan Xie & Yelin Deng & Chris Yuan, 2018. "Predictive modeling of battery degradation and greenhouse gas emissions from U.S. state-level electric vehicle operation," Nature Communications, Nature, vol. 9(1), pages 1-10, December.
    9. Xie, Yunkun & Li, Yangyang & Zhao, Zhichao & Dong, Hao & Wang, Shuqian & Liu, Jingping & Guan, Jinhuan & Duan, Xiongbo, 2020. "Microsimulation of electric vehicle energy consumption and driving range," Applied Energy, Elsevier, vol. 267(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Awab Baqar & Mamadou Baïlo Camara & Brayima Dakyo, 2022. "Energy Management in the Multi-Source Systems," Energies, MDPI, vol. 15(8), pages 1-4, April.
    2. Mokesioluwa Fanoro & Mladen Božanić & Saurabh Sinha, 2022. "A Review of the Impact of Battery Degradation on Energy Management Systems with a Special Emphasis on Electric Vehicles," Energies, MDPI, vol. 15(16), pages 1-29, August.
    3. Ye, Yiming & Wang, Hanchen & Xu, Bin & Zhang, Jiangfeng, 2023. "An imitation learning-based energy management strategy for electric vehicles considering battery aging," Energy, Elsevier, vol. 283(C).
    4. Liu, Qin & Zhang, Wencan & Zhang, Zhongbo & Qin, Qichao, 2022. "A drive system global control strategy for electric vehicle based on optimized acceleration curve," Energy, Elsevier, vol. 248(C).
    5. Caulfield, Brian & Furszyfer, Dylan & Stefaniec, Agnieszka & Foley, Aoife, 2022. "Measuring the equity impacts of government subsidies for electric vehicles," Energy, Elsevier, vol. 248(C).
    6. Harri Aaltonen & Seppo Sierla & Rakshith Subramanya & Valeriy Vyatkin, 2021. "A Simulation Environment for Training a Reinforcement Learning Agent Trading a Battery Storage," Energies, MDPI, vol. 14(17), pages 1-20, September.
    7. Du, Guodong & Zou, Yuan & Zhang, Xudong & Guo, Lingxiong & Guo, Ningyuan, 2022. "Energy management for a hybrid electric vehicle based on prioritized deep reinforcement learning framework," Energy, Elsevier, vol. 241(C).
    8. Andrea Di Martino & Seyed Mahdi Miraftabzadeh & Michela Longo, 2022. "Strategies for the Modelisation of Electric Vehicle Energy Consumption: A Review," Energies, MDPI, vol. 15(21), pages 1-20, October.
    9. Miranda, Matheus H.R. & Silva, Fabrício L. & Lourenço, Maria A.M. & Eckert, Jony J. & Silva, Ludmila C.A., 2023. "Particle swarm optimization of Elman neural network applied to battery state of charge and state of health estimation," Energy, Elsevier, vol. 285(C).
    10. Bo, Lin & Han, Lijin & Xiang, Changle & Liu, Hui & Ma, Tian, 2022. "A Q-learning fuzzy inference system based online energy management strategy for off-road hybrid electric vehicles," Energy, Elsevier, vol. 252(C).
    11. Guo, Xiaokai & Yan, Xianguo & Chen, Zhi & Meng, Zhiyu, 2022. "Research on energy management strategy of heavy-duty fuel cell hybrid vehicles based on dueling-double-deep Q-network," Energy, Elsevier, vol. 260(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhao, Yang & Wang, Zhenpo & Shen, Zuo-Jun Max & Zhang, Lei & Dorrell, David G. & Sun, Fengchun, 2022. "Big data-driven decoupling framework enabling quantitative assessments of electric vehicle performance degradation," Applied Energy, Elsevier, vol. 327(C).
    2. Ye, Yiming & Wang, Hanchen & Xu, Bin & Zhang, Jiangfeng, 2023. "An imitation learning-based energy management strategy for electric vehicles considering battery aging," Energy, Elsevier, vol. 283(C).
    3. Sun, Xilei & Fu, Jianqin, 2024. "Many-objective optimization of BEV design parameters based on gradient boosting decision tree models and the NSGA-III algorithm considering the ambient temperature," Energy, Elsevier, vol. 288(C).
    4. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    5. Zhang, Cetengfei & Zhou, Quan & Hua, Min & Xu, Hongming & Bassett, Mike & Zhang, Fanggang, 2023. "Cuboid equivalent consumption minimization strategy for energy management of multi-mode plug-in hybrid vehicles considering diverse time scale objectives," Applied Energy, Elsevier, vol. 351(C).
    6. Wang, Hua & Zhao, De & Cai, Yutong & Meng, Qiang & Ong, Ghim Ping, 2021. "Taxi trajectory data based fast-charging facility planning for urban electric taxi systems," Applied Energy, Elsevier, vol. 286(C).
    7. Sarvaiya, Shradhdha & Ganesh, Sachin & Xu, Bin, 2021. "Comparative analysis of hybrid vehicle energy management strategies with optimization of fuel economy and battery life," Energy, Elsevier, vol. 228(C).
    8. Rauf, Huzaifa & Khalid, Muhammad & Arshad, Naveed, 2022. "Machine learning in state of health and remaining useful life estimation: Theoretical and technological development in battery degradation modelling," Renewable and Sustainable Energy Reviews, Elsevier, vol. 156(C).
    9. Wang, Hua & Zhao, De & Meng, Qiang & Ong, Ghim Ping & Lee, Der-Horng, 2020. "Network-level energy consumption estimation for electric vehicles considering vehicle and user heterogeneity," Transportation Research Part A: Policy and Practice, Elsevier, vol. 132(C), pages 30-46.
    10. Shi, Dehua & Liu, Sheng & Cai, Yingfeng & Wang, Shaohua & Li, Haoran & Chen, Long, 2021. "Pontryagin’s minimum principle based fuzzy adaptive energy management for hybrid electric vehicle using real-time traffic information," Applied Energy, Elsevier, vol. 286(C).
    11. Farmann, Alexander & Waag, Wladislaw & Sauer, Dirk Uwe, 2016. "Application-specific electrical characterization of high power batteries with lithium titanate anodes for electric vehicles," Energy, Elsevier, vol. 112(C), pages 294-306.
    12. He, Qiang & Yang, Yang & Luo, Chang & Zhai, Jun & Luo, Ronghua & Fu, Chunyun, 2022. "Energy recovery strategy optimization of dual-motor drive electric vehicle based on braking safety and efficient recovery," Energy, Elsevier, vol. 248(C).
    13. Barelli, L. & Bidini, G. & Bonucci, F. & Castellini, L. & Fratini, A. & Gallorini, F. & Zuccari, A., 2019. "Flywheel hybridization to improve battery life in energy storage systems coupled to RES plants," Energy, Elsevier, vol. 173(C), pages 937-950.
    14. Eckert, Jony Javorski & Silva, Fabrício L. & da Silva, Samuel Filgueira & Bueno, André Valente & de Oliveira, Mona Lisa Moura & Silva, Ludmila C.A., 2022. "Optimal design and power management control of hybrid biofuel–electric powertrain," Applied Energy, Elsevier, vol. 325(C).
    15. Matteo Acquarone & Claudio Maino & Daniela Misul & Ezio Spessa & Antonio Mastropietro & Luca Sorrentino & Enrico Busto, 2023. "Influence of the Reward Function on the Selection of Reinforcement Learning Agents for Hybrid Electric Vehicles Real-Time Control," Energies, MDPI, vol. 16(6), pages 1-22, March.
    16. Nan, Sirui & Tu, Ran & Li, Tiezhu & Sun, Jian & Chen, Haibo, 2022. "From driving behavior to energy consumption: A novel method to predict the energy consumption of electric bus," Energy, Elsevier, vol. 261(PA).
    17. Duggal, Angel Swastik & Singh, Rajesh & Gehlot, Anita & Gupta, Lovi Raj & Akram, Sheik Vaseem & Prakash, Chander & Singh, Sunpreet & Kumar, Raman, 2021. "Infrastructure, mobility and safety 4.0: Modernization in road transportation," Technology in Society, Elsevier, vol. 67(C).
    18. Yang Xia & Wenjia Zeng & Xinjie Xing & Yuanzhu Zhan & Kim Hua Tan & Ajay Kumar, 2023. "Joint optimisation of drone routing and battery wear for sustainable supply chain development: a mixed-integer programming model based on blockchain-enabled fleet sharing," Annals of Operations Research, Springer, vol. 327(1), pages 89-127, August.
    19. Zhu, Tao & Wills, Richard G.A. & Lot, Roberto & Ruan, Haijun & Jiang, Zhihao, 2021. "Adaptive energy management of a battery-supercapacitor energy storage system for electric vehicles based on flexible perception and neural network fitting," Applied Energy, Elsevier, vol. 292(C).
    20. Zhang, Wei & Wang, Jixin & Liu, Yong & Gao, Guangzong & Liang, Siwen & Ma, Hongfeng, 2020. "Reinforcement learning-based intelligent energy management architecture for hybrid construction machinery," Applied Energy, Elsevier, vol. 275(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:229:y:2021:i:c:s0360544221009531. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.