IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v8y2015i7p7243-7260d52695.html
   My bibliography  Save this article

Reinforcement Learning–Based Energy Management Strategy for a Hybrid Electric Tracked Vehicle

Author

Listed:
  • Teng Liu

    (Collaborative Innovation Center of Electric Vehicles in Beijing, School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China)

  • Yuan Zou

    (Collaborative Innovation Center of Electric Vehicles in Beijing, School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China)

  • Dexing Liu

    (Collaborative Innovation Center of Electric Vehicles in Beijing, School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China)

  • Fengchun Sun

    (Collaborative Innovation Center of Electric Vehicles in Beijing, School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China)

Abstract

This paper presents a reinforcement learning (RL)–based energy management strategy for a hybrid electric tracked vehicle. A control-oriented model of the powertrain and vehicle dynamics is first established. According to the sample information of the experimental driving schedule, statistical characteristics at various velocities are determined by extracting the transition probability matrix of the power request. Two RL-based algorithms, namely Q -learning and Dyna algorithms, are applied to generate optimal control solutions. The two algorithms are simulated on the same driving schedule, and the simulation results are compared to clarify the merits and demerits of these algorithms. Although the Q -learning algorithm is faster (3 h) than the Dyna algorithm (7 h), its fuel consumption is 1.7% higher than that of the Dyna algorithm. Furthermore, the Dyna algorithm registers approximately the same fuel consumption as the dynamic programming–based global optimal solution. The computational cost of the Dyna algorithm is substantially lower than that of the stochastic dynamic programming.

Suggested Citation

  • Teng Liu & Yuan Zou & Dexing Liu & Fengchun Sun, 2015. "Reinforcement Learning–Based Energy Management Strategy for a Hybrid Electric Tracked Vehicle," Energies, MDPI, vol. 8(7), pages 1-18, July.
    • Handle: RePEc:gam:jeners:v:8:y:2015:i:7:p:7243-7260:d:52695
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/8/7/7243/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/8/7/7243/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Felipe Jiménez & Wilmar Cabrera-Montiel, 2014. "System for Road Vehicle Energy Optimization Using Real Time Road and Traffic Information," Energies, MDPI, vol. 7(6), pages 1-23, June.
    2. Yuan Zou & Fengchun Sun & Xiaosong Hu & Lino Guzzella & Huei Peng, 2012. "Combined Optimal Sizing and Control for a Hybrid Tracked Vehicle," Energies, MDPI, vol. 5(11), pages 1-14, November.
    3. Abdelsalam Ahmed Abdelsalam & Shumei Cui, 2012. "A Fuzzy Logic Global Power Management Strategy for Hybrid Electric Vehicles Based on a Permanent Magnet Electric Variable Transmission," Energies, MDPI, vol. 5(4), pages 1-24, April.
    4. Zou Yuan & Liu Teng & Sun Fengchun & Huei Peng, 2013. "Comparative Study of Dynamic Programming and Pontryagin’s Minimum Principle on Energy Management for a Parallel Hybrid Electric Vehicle," Energies, MDPI, vol. 6(4), pages 1-14, April.
    5. Bingtuan Gao & Wenhu Zhang & Yi Tang & Mingjin Hu & Mingcheng Zhu & Huiyu Zhan, 2014. "Game-Theoretic Energy Management for Residential Users with Dischargeable Plug-in Electric Vehicles," Energies, MDPI, vol. 7(11), pages 1-20, November.
    6. Tobias Nüesch & Alberto Cerofolini & Giorgio Mancini & Nicolò Cavina & Christopher Onder & Lino Guzzella, 2014. "Equivalent Consumption Minimization Strategy for the Control of Real Driving NOx Emissions of a Diesel Hybrid Electric Vehicle," Energies, MDPI, vol. 7(5), pages 1-31, May.
    7. Zhongyue Zou & Jun Xu & Chris Mi & Binggang Cao & Zheng Chen, 2014. "Evaluation of Model Based State of Charge Estimation Methods for Lithium-Ion Batteries," Energies, MDPI, vol. 7(8), pages 1-18, August.
    8. Tobias Nüesch & Philipp Elbert & Michael Flankl & Christopher Onder & Lino Guzzella, 2014. "Convex Optimization for the Energy Management of Hybrid Electric Vehicles Considering Engine Start and Gearshift Costs," Energies, MDPI, vol. 7(2), pages 1-23, February.
    9. Jingang Guo & Xiaoping Jian & Guangyu Lin, 2014. "Performance Evaluation of an Anti-Lock Braking System for Electric Vehicles with a Fuzzy Sliding Mode Controller," Energies, MDPI, vol. 7(10), pages 1-18, October.
    10. Tri-Vien Vu & Chih-Keng Chen & Chih-Wei Hung, 2014. "A Model Predictive Control Approach for Fuel Economy Improvement of a Series Hydraulic Hybrid Vehicle," Energies, MDPI, vol. 7(11), pages 1-24, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liu, Teng & Tan, Wenhao & Tang, Xiaolin & Zhang, Jinwei & Xing, Yang & Cao, Dongpu, 2021. "Driving conditions-driven energy management strategies for hybrid electric vehicles: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 151(C).
    2. Chen, Zheng & Xia, Bing & You, Chenwen & Mi, Chunting Chris, 2015. "A novel energy management method for series plug-in hybrid electric vehicles," Applied Energy, Elsevier, vol. 145(C), pages 172-179.
    3. Du, Guodong & Zou, Yuan & Zhang, Xudong & Liu, Teng & Wu, Jinlong & He, Dingbo, 2020. "Deep reinforcement learning based energy management for a hybrid electric vehicle," Energy, Elsevier, vol. 201(C).
    4. Fengqi Zhang & Lihua Wang & Serdar Coskun & Hui Pang & Yahui Cui & Junqiang Xi, 2020. "Energy Management Strategies for Hybrid Electric Vehicles: Review, Classification, Comparison, and Outlook," Energies, MDPI, vol. 13(13), pages 1-35, June.
    5. Balerna, Camillo & Lanzetti, Nicolas & Salazar, Mauro & Cerofolini, Alberto & Onder, Christopher, 2020. "Optimal low-level control strategies for a high-performance hybrid electric power unit," Applied Energy, Elsevier, vol. 276(C).
    6. Tobias Nüesch & Philipp Elbert & Michael Flankl & Christopher Onder & Lino Guzzella, 2014. "Convex Optimization for the Energy Management of Hybrid Electric Vehicles Considering Engine Start and Gearshift Costs," Energies, MDPI, vol. 7(2), pages 1-23, February.
    7. Duhr, Pol & Christodoulou, Grigorios & Balerna, Camillo & Salazar, Mauro & Cerofolini, Alberto & Onder, Christopher H., 2021. "Time-optimal gearshift and energy management strategies for a hybrid electric race car," Applied Energy, Elsevier, vol. 282(PA).
    8. Zou, Yuan & Liu, Teng & Liu, Dexing & Sun, Fengchun, 2016. "Reinforcement learning-based real-time energy management for a hybrid tracked vehicle," Applied Energy, Elsevier, vol. 171(C), pages 372-382.
    9. Wei, Shouyang & Zou, Yuan & Sun, Fengchun & Christopher, Onder, 2017. "A pseudospectral method for solving optimal control problem of a hybrid tracked vehicle," Applied Energy, Elsevier, vol. 194(C), pages 588-595.
    10. Yuping Zeng & Yang Cai & Guiyue Kou & Wei Gao & Datong Qin, 2018. "Energy Management for Plug-In Hybrid Electric Vehicle Based on Adaptive Simplified-ECMS," Sustainability, MDPI, vol. 10(6), pages 1-24, June.
    11. Xiong, Rui & Cao, Jiayi & Yu, Quanqing, 2018. "Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle," Applied Energy, Elsevier, vol. 211(C), pages 538-548.
    12. Camillo Balerna & Marc-Philippe Neumann & Nicolò Robuschi & Pol Duhr & Alberto Cerofolini & Vittorio Ravaglioli & Christopher Onder, 2020. "Time-Optimal Low-Level Control and Gearshift Strategies for the Formula 1 Hybrid Electric Powertrain," Energies, MDPI, vol. 14(1), pages 1-30, December.
    13. Jixiang Fan & Jiangyan Zhang & Tielong Shen, 2015. "Map-Based Power-Split Strategy Design with Predictive Performance Optimization for Parallel Hybrid Electric Vehicles," Energies, MDPI, vol. 8(9), pages 1-23, September.
    14. Yuping Zeng & Yang Cai & Changbao Chu & Guiyue Kou & Wei Gao, 2018. "Integrated Energy and Catalyst Thermal Management for Plug-In Hybrid Electric Vehicles," Energies, MDPI, vol. 11(7), pages 1-29, July.
    15. Zhang, Pei & Yan, Fuwu & Du, Changqing, 2015. "A comprehensive analysis of energy management strategies for hybrid electric vehicles based on bibliometrics," Renewable and Sustainable Energy Reviews, Elsevier, vol. 48(C), pages 88-104.
    16. Di Guo & Changqing Du & Fuwu Yan, 2016. "Drivability-Related Discrete-Time Model Predictive Control of Mode Transition in Pre-Transmission Parallel Hybrid Powertrains," Energies, MDPI, vol. 9(9), pages 1-31, September.
    17. Caiyang Wei & Theo Hofman & Esin Ilhan Caarls & Rokus van Iperen, 2020. "A Review of the Integrated Design and Control of Electrified Vehicles," Energies, MDPI, vol. 13(20), pages 1-31, October.
    18. Johannes Schalk & Harald Aschemann, 2016. "A Causal and Real-Time Capable Power Management Algorithm for Off-Highway Hybrid Propulsion Systems," Energies, MDPI, vol. 10(1), pages 1-17, December.
    19. Felipe Jiménez & Wilmar Cabrera-Montiel, 2014. "System for Road Vehicle Energy Optimization Using Real Time Road and Traffic Information," Energies, MDPI, vol. 7(6), pages 1-23, June.
    20. Shaobo Xie & Xiaosong Hu & Kun Lang & Shanwei Qi & Tong Liu, 2018. "Powering Mode-Integrated Energy Management Strategy for a Plug-In Hybrid Electric Truck with an Automatic Mechanical Transmission Based on Pontryagin’s Minimum Principle," Sustainability, MDPI, vol. 10(10), pages 1-23, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:8:y:2015:i:7:p:7243-7260:d:52695. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.