IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v288y2024ics0360544223032437.html
   My bibliography  Save this article

Soft actor-critic-based EMS design for dual motor battery electric bus

Author

Listed:
  • Liang, Zhaowen
  • Ruan, Jiageng
  • Wang, Zhenpo
  • Liu, Kai
  • Li, Bin

Abstract

Thanks to the outstanding adaptability and relatively simple design process, machine learning-based energy management strategies (EMSs) show their superiority in reducing the energy consumption for multi-power powertrains. Given the popularity of distributed-drive in battery electric vehicles (BEVs), this study selects a dual-motor four-speed electric bus to investigate the proposed soft actor-critic (SAC)-based EMS. Two improvements are made to the traditional SAC algorithm to meet the special requirements of EMS in a distributed-drive electric bus in this study. Firstly, the introduced combination of Gumbel-SoftMax and actor-network allows the SAC agents to explore the hybrid action space (discrete operating modes and continuous torque distribution coefficients). Secondly, a heuristic rule-interposing action controller (HRIAC) is involved to reduce illogical exploration of SAC agents in searching for optimal power distributing ratio. Simulation results demonstrate that Gumbel-SoftMax facilitates agents' exploration and narrows the energy consumption gap between the proposed EMS and the global optimal EMS, meanwhile, HRIAC accelerates the convergence of agent training. The comparative results of proposed EMS performance in unknown driving cycles show that the proposed EMS outperforms other deep reinforcement learning (DRL)-based EMSs in adaptability, which is taken as a solid foundation for energy efficiency improvement of dual-motor electric bus in practice.

Suggested Citation

  • Liang, Zhaowen & Ruan, Jiageng & Wang, Zhenpo & Liu, Kai & Li, Bin, 2024. "Soft actor-critic-based EMS design for dual motor battery electric bus," Energy, Elsevier, vol. 288(C).
    • Handle: RePEc:eee:energy:v:288:y:2024:i:c:s0360544223032437
    DOI: 10.1016/j.energy.2023.129849
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544223032437
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2023.129849?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Li, Weihan & Cui, Han & Nemeth, Thomas & Jansen, Jonathan & Ünlübayir, Cem & Wei, Zhongbao & Feng, Xuning & Han, Xuebing & Ouyang, Minggao & Dai, Haifeng & Wei, Xuezhe & Sauer, Dirk Uwe, 2021. "Cloud-based health-conscious energy management of hybrid battery systems in electric vehicles with deep reinforcement learning," Applied Energy, Elsevier, vol. 293(C).
    2. Wang, Hanchen & Ye, Yiming & Zhang, Jiangfeng & Xu, Bin, 2023. "A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle," Energy, Elsevier, vol. 266(C).
    3. Xiao, Boyi & Yang, Weiwei & Wu, Jiamin & Walker, Paul D. & Zhang, Nong, 2022. "Energy management strategy via maximum entropy reinforcement learning for an extended range logistics vehicle," Energy, Elsevier, vol. 253(C).
    4. Han, Xuefeng & He, Hongwen & Wu, Jingda & Peng, Jiankun & Li, Yuecheng, 2019. "Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle," Applied Energy, Elsevier, vol. 254(C).
    5. Ruan, Jiageng & Walker, Paul & Zhang, Nong, 2016. "A comparative study energy consumption and costs of battery electric vehicle transmissions," Applied Energy, Elsevier, vol. 165(C), pages 119-134.
    6. Zhang, Kaixuan & Ruan, Jiageng & Li, Tongyang & Cui, Hanghang & Wu, Changcheng, 2023. "The effects investigation of data-driven fitting cycle and deep deterministic policy gradient algorithm on energy management strategy of dual-motor electric bus," Energy, Elsevier, vol. 269(C).
    7. Ruan, Jiageng & Wu, Changcheng & Liang, Zhaowen & Liu, Kai & Li, Bin & Li, Weihan & Li, Tongyang, 2023. "The application of machine learning-based energy management strategy in a multi-mode plug-in hybrid electric vehicle, part II: Deep deterministic policy gradient algorithm design for electric mode," Energy, Elsevier, vol. 269(C).
    8. Li, Yuecheng & He, Hongwen & Khajepour, Amir & Wang, Hong & Peng, Jiankun, 2019. "Energy management for a power-split hybrid electric bus via deep reinforcement learning with terrain information," Applied Energy, Elsevier, vol. 255(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhou, Jie & Zhang, Tiezhu & Zhang, Hongxin & Zhang, Zhen & Hong, Jichao & Yang, Jian, 2024. "Energy management strategy for electro-hydraulic hybrid electric vehicles considering optimal mode switching: A soft actor-critic approach trained on a multi-modal driving cycle," Energy, Elsevier, vol. 305(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peng, Jiankun & Shen, Yang & Wu, ChangCheng & Wang, Chunhai & Yi, Fengyan & Ma, Chunye, 2023. "Research on energy-saving driving control of hydrogen fuel bus based on deep reinforcement learning in freeway ramp weaving area," Energy, Elsevier, vol. 285(C).
    2. Huang, Ruchen & He, Hongwen & Su, Qicong & Härtl, Martin & Jaensch, Malte, 2025. "Type- and task-crossing energy management for fuel cell vehicles with longevity consideration: A heterogeneous deep transfer reinforcement learning framework," Applied Energy, Elsevier, vol. 377(PC).
    3. Tan, Yingqi & Xu, Jingyi & Ma, Junyi & Li, Zirui & Chen, Huiyan & Xi, Junqiang & Liu, Haiou, 2024. "A transferable perception-guided EMS for series hybrid electric unmanned tracked vehicles," Energy, Elsevier, vol. 306(C).
    4. Liu, Zemin Eitan & Li, Yong & Zhou, Quan & Shuai, Bin & Hua, Min & Xu, Hongming & Xu, Lubing & Tan, Guikun & Li, Yanfei, 2025. "Real-time energy management for HEV combining naturalistic driving data and deep reinforcement learning with high generalization," Applied Energy, Elsevier, vol. 377(PA).
    5. Huang, Ruchen & He, Hongwen & Su, Qicong, 2024. "Smart energy management for hybrid electric bus via improved soft actor-critic algorithm in a heuristic learning framework," Energy, Elsevier, vol. 309(C).
    6. Wu, Changcheng & Peng, Jiankun & Chen, Jun & He, Hongwen & Pi, Dawei & Wang, Zhongwei & Ma, Chunye, 2024. "Battery health-considered energy management strategy for a dual-motor two-speed battery electric vehicle based on a hybrid soft actor-critic algorithm with memory function," Applied Energy, Elsevier, vol. 376(PB).
    7. He, Hongwen & Su, Qicong & Huang, Ruchen & Niu, Zegong, 2024. "Enabling intelligent transferable energy management of series hybrid electric tracked vehicle across motion dimensions via soft actor-critic algorithm," Energy, Elsevier, vol. 294(C).
    8. Zhou, Yujie & Huang, Yin & Mao, Xuping & Kang, Zehao & Huang, Xuejin & Xuan, Dongji, 2024. "Research on energy management strategy of fuel cell hybrid power via an improved TD3 deep reinforcement learning," Energy, Elsevier, vol. 293(C).
    9. Yang, Ningkang & Han, Lijin & Xiang, Changle & Liu, Hui & Li, Xunmin, 2021. "An indirect reinforcement learning based real-time energy management strategy via high-order Markov Chain model for a hybrid electric vehicle," Energy, Elsevier, vol. 236(C).
    10. Tang, Tianfeng & Peng, Qianlong & Shi, Qing & Peng, Qingguo & Zhao, Jin & Chen, Chaoyi & Wang, Guangwei, 2024. "Energy management of fuel cell hybrid electric bus in mountainous regions: A deep reinforcement learning approach considering terrain characteristics," Energy, Elsevier, vol. 311(C).
    11. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    12. Chen, Jiaxin & Shu, Hong & Tang, Xiaolin & Liu, Teng & Wang, Weida, 2022. "Deep reinforcement learning-based multi-objective control of hybrid power system combined with road recognition under time-varying environment," Energy, Elsevier, vol. 239(PC).
    13. Zhang, Hao & Lei, Nuo & Chen, Boli & Li, Bingbing & Li, Rulong & Wang, Zhi, 2024. "Modeling and control system optimization for electrified vehicles: A data-driven approach," Energy, Elsevier, vol. 310(C).
    14. Huang, Ruchen & He, Hongwen & Zhao, Xuyang & Wang, Yunlong & Li, Menglin, 2022. "Battery health-aware and naturalistic data-driven energy management for hybrid electric bus based on TD3 deep reinforcement learning algorithm," Applied Energy, Elsevier, vol. 321(C).
    15. Chi T. P. Nguyen & Bảo-Huy Nguyễn & Minh C. Ta & João Pedro F. Trovão, 2023. "Dual-Motor Dual-Source High Performance EV: A Comprehensive Review," Energies, MDPI, vol. 16(20), pages 1-28, October.
    16. Yang, Dongpo & Liu, Tong & Song, Dafeng & Zhang, Xuanming & Zeng, Xiaohua, 2023. "A real time multi-objective optimization Guided-MPC strategy for power-split hybrid electric bus based on velocity prediction," Energy, Elsevier, vol. 276(C).
    17. Wang, Yue & Li, Keqiang & Zeng, Xiaohua & Gao, Bolin & Hong, Jichao, 2023. "Investigation of novel intelligent energy management strategies for connected HEB considering global planning of fixed-route information," Energy, Elsevier, vol. 263(PB).
    18. Feng, Zhiyan & Zhang, Qingang & Zhang, Yiming & Fei, Liangyu & Jiang, Fei & Zhao, Shengdun, 2024. "Practicability analysis of online deep reinforcement learning towards energy management strategy of 4WD-BEVs driven by dual-motor in-wheel motors," Energy, Elsevier, vol. 290(C).
    19. Xiao, Boyi & Yang, Weiwei & Wu, Jiamin & Walker, Paul D. & Zhang, Nong, 2022. "Energy management strategy via maximum entropy reinforcement learning for an extended range logistics vehicle," Energy, Elsevier, vol. 253(C).
    20. Zhou, Jie & Zhang, Tiezhu & Zhang, Hongxin & Zhang, Zhen & Hong, Jichao & Yang, Jian, 2024. "Energy management strategy for electro-hydraulic hybrid electric vehicles considering optimal mode switching: A soft actor-critic approach trained on a multi-modal driving cycle," Energy, Elsevier, vol. 305(C).

    More about this item

    Keywords

    SAC; RL; DDPG; TD3; EMS;
    All these keywords.

    JEL classification:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:288:y:2024:i:c:s0360544223032437. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.