IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0180491.html
   My bibliography  Save this article

Implementation of real-time energy management strategy based on reinforcement learning for hybrid electric vehicles and simulation validation

Author

Listed:
  • Zehui Kong
  • Yuan Zou
  • Teng Liu

Abstract

To further improve the fuel economy of series hybrid electric tracked vehicles, a reinforcement learning (RL)-based real-time energy management strategy is developed in this paper. In order to utilize the statistical characteristics of online driving schedule effectively, a recursive algorithm for the transition probability matrix (TPM) of power-request is derived. The reinforcement learning (RL) is applied to calculate and update the control policy at regular time, adapting to the varying driving conditions. A facing-forward powertrain model is built in detail, including the engine-generator model, battery model and vehicle dynamical model. The robustness and adaptability of real-time energy management strategy are validated through the comparison with the stationary control strategy based on initial transition probability matrix (TPM) generated from a long naturalistic driving cycle in the simulation. Results indicate that proposed method has better fuel economy than stationary one and is more effective in real-time control.

Suggested Citation

  • Zehui Kong & Yuan Zou & Teng Liu, 2017. "Implementation of real-time energy management strategy based on reinforcement learning for hybrid electric vehicles and simulation validation," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-16, July.
    • Handle: RePEc:plo:pone00:0180491
    DOI: 10.1371/journal.pone.0180491
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0180491
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0180491&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0180491?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Castaings, Ali & Lhomme, Walter & Trigui, Rochdi & Bouscayrol, Alain, 2016. "Comparison of energy management strategies of a battery/supercapacitors system for electric vehicle under real-time constraints," Applied Energy, Elsevier, vol. 163(C), pages 190-200.
    2. Hou, Cong & Ouyang, Minggao & Xu, Liangfei & Wang, Hewu, 2014. "Approximate Pontryagin’s minimum principle applied to the energy management of plug-in hybrid electric vehicles," Applied Energy, Elsevier, vol. 115(C), pages 174-189.
    3. Trovão, João P. & Pereirinha, Paulo G. & Jorge, Humberto M. & Antunes, Carlos Henggeler, 2013. "A multi-level energy management system for multi-source electric vehicles – An integrated rule-based meta-heuristic approach," Applied Energy, Elsevier, vol. 105(C), pages 304-318.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Liu, Teng & Tan, Wenhao & Tang, Xiaolin & Zhang, Jinwei & Xing, Yang & Cao, Dongpu, 2021. "Driving conditions-driven energy management strategies for hybrid electric vehicles: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 151(C).
    2. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    3. Du, Guodong & Zou, Yuan & Zhang, Xudong & Kong, Zehui & Wu, Jinlong & He, Dingbo, 2019. "Intelligent energy management for hybrid electric tracked vehicles using online reinforcement learning," Applied Energy, Elsevier, vol. 251(C), pages 1-1.
    4. Zhang, Yahui & Wang, Zimeng & Tian, Yang & Wang, Zhong & Kang, Mingxin & Xie, Fangxi & Wen, Guilin, 2024. "Pre-optimization-assisted deep reinforcement learning-based energy management strategy for a series–parallel hybrid electric truck," Energy, Elsevier, vol. 302(C).
    5. Mingliang Bai & Wenjiang Yang & Dongbin Song & Marek Kosuda & Stanislav Szabo & Pavol Lipovsky & Afshar Kasaei, 2020. "Research on Energy Management of Hybrid Unmanned Aerial Vehicles to Improve Energy-Saving and Emission Reduction Performance," IJERPH, MDPI, vol. 17(8), pages 1-24, April.
    6. Vamsi Krishna Reddy, Aala Kalananda & Venkata Lakshmi Narayana, Komanapalli, 2022. "Meta-heuristics optimization in electric vehicles -an extensive review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 160(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zou, Yuan & Liu, Teng & Liu, Dexing & Sun, Fengchun, 2016. "Reinforcement learning-based real-time energy management for a hybrid tracked vehicle," Applied Energy, Elsevier, vol. 171(C), pages 372-382.
    2. Bizon, Nicu, 2019. "Real-time optimization strategies of Fuel Cell Hybrid Power Systems based on Load-following control: A new strategy, and a comparative study of topologies and fuel economy obtained," Applied Energy, Elsevier, vol. 241(C), pages 444-460.
    3. Peng, Fei & Zhao, Yuanzhe & Li, Xiaopeng & Liu, Zhixiang & Chen, Weirong & Liu, Yang & Zhou, Donghua, 2017. "Development of master-slave energy management strategy based on fuzzy logic hysteresis state machine and differential power processing compensation for a PEMFC-LIB-SC hybrid tramway," Applied Energy, Elsevier, vol. 206(C), pages 346-363.
    4. Bảo-Huy Nguyễn & João Pedro F. Trovão & Ronan German & Alain Bouscayrol, 2020. "Real-Time Energy Management of Parallel Hybrid Electric Vehicles Using Linear Quadratic Regulation," Energies, MDPI, vol. 13(21), pages 1-19, October.
    5. Wang, Bin & Xu, Jun & Cao, Binggang & Ning, Bo, 2017. "Adaptive mode switch strategy based on simulated annealing optimization of a multi-mode hybrid energy storage system for electric vehicles," Applied Energy, Elsevier, vol. 194(C), pages 596-608.
    6. Zhuang, Weichao & Li (Eben), Shengbo & Zhang, Xiaowu & Kum, Dongsuk & Song, Ziyou & Yin, Guodong & Ju, Fei, 2020. "A survey of powertrain configuration studies on hybrid electric vehicles," Applied Energy, Elsevier, vol. 262(C).
    7. Bizon, Nicu & Radut, Marin & Oproescu, Mihai, 2015. "Energy control strategies for the Fuel Cell Hybrid Power Source under unknown load profile," Energy, Elsevier, vol. 86(C), pages 31-41.
    8. Xiang, Changle & Ding, Feng & Wang, Weida & He, Wei, 2017. "Energy management of a dual-mode power-split hybrid electric vehicle based on velocity prediction and nonlinear model predictive control," Applied Energy, Elsevier, vol. 189(C), pages 640-653.
    9. Feroldi, Diego & Carignano, Mauro, 2016. "Sizing for fuel cell/supercapacitor hybrid vehicles based on stochastic driving cycles," Applied Energy, Elsevier, vol. 183(C), pages 645-658.
    10. Bizon, Nicu, 2017. "Energy optimization of fuel cell system by using global extremum seeking algorithm," Applied Energy, Elsevier, vol. 206(C), pages 458-474.
    11. Xiong, Rui & Cao, Jiayi & Yu, Quanqing, 2018. "Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle," Applied Energy, Elsevier, vol. 211(C), pages 538-548.
    12. Jixiang Fan & Jiangyan Zhang & Tielong Shen, 2015. "Map-Based Power-Split Strategy Design with Predictive Performance Optimization for Parallel Hybrid Electric Vehicles," Energies, MDPI, vol. 8(9), pages 1-23, September.
    13. Hoai-Linh T. Nguyen & Bảo-Huy Nguyễn & Thanh Vo-Duy & João Pedro F. Trovão, 2021. "A Comparative Study of Adaptive Filtering Strategies for Hybrid Energy Storage Systems in Electric Vehicles," Energies, MDPI, vol. 14(12), pages 1-23, June.
    14. Zhuang, Weichao & Ye, Jianwei & Song, Ziyou & Yin, Guodong & Li, Guangmin, 2020. "Comparison of semi-active hybrid battery system configurations for electric taxis application," Applied Energy, Elsevier, vol. 259(C).
    15. Kong, Xiangdong & Zheng, Yuejiu & Ouyang, Minggao & Li, Xiangjun & Lu, Languang & Li, Jianqiu & Zhang, Zhendong, 2017. "Signal synchronization for massive data storage in modular battery management system with controller area network," Applied Energy, Elsevier, vol. 197(C), pages 52-62.
    16. Zhang, Yahui & Wang, Zimeng & Tian, Yang & Wang, Zhong & Kang, Mingxin & Xie, Fangxi & Wen, Guilin, 2024. "Pre-optimization-assisted deep reinforcement learning-based energy management strategy for a series–parallel hybrid electric truck," Energy, Elsevier, vol. 302(C).
    17. Ashleigh Townsend & Rupert Gouws, 2023. "A Comparative Review of Capacity Measurement in Energy Storage Devices," Energies, MDPI, vol. 16(10), pages 1-26, May.
    18. Trovão, João P. & Silva, Mário A. & Antunes, Carlos Henggeler & Dubois, Maxime R., 2017. "Stability enhancement of the motor drive DC input voltage of an electric vehicle using on-board hybrid energy storage systems," Applied Energy, Elsevier, vol. 205(C), pages 244-259.
    19. Desreveaux, A. & Bouscayrol, A. & Trigui, R. & Hittinger, E. & Castex, E. & Sirbu, G.M., 2023. "Accurate energy consumption for comparison of climate change impact of thermal and electric vehicles," Energy, Elsevier, vol. 268(C).
    20. Mpho J. Lencwe & Shyama P. Chowdhury & Thomas O. Olwal, 2018. "A Multi-Stage Approach to a Hybrid Lead Acid Battery and Supercapacitor System for Transport Vehicles," Energies, MDPI, vol. 11(11), pages 1-16, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0180491. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.