IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v375y2024ics0306261924014612.html
   My bibliography  Save this article

An intelligent full-knowledge transferable collaborative eco-driving framework based on improved soft actor-critic algorithm

Author

Listed:
  • Huang, Ruchen
  • He, Hongwen
  • Su, Qicong

Abstract

Eco-driving is a promising technology for fuel cell vehicles (FCVs) that simultaneously achieves safe driving and energy saving in the urban transport sector, particularly through the application of cutting-edge deep reinforcement learning (DRL). However, developing specific DRL-based eco-driving strategies for different FCVs is a laborious task, since repetitive training is required when encountering various FCV types. To tackle this challenge, this paper proposes an intelligent transferable collaborative eco-driving framework across FCV types. Firstly, the eco-driving problem in the vehicle-following scenario is formulated by collaboratively integrating adaptive cruise control (ACC) with energy management strategy (EMS), and then an improved soft actor-critic (I-SAC) algorithm is designed to solve this problem. After that, a source eco-driving strategy based on I-SAC is pre-trained for a light fuel cell hybrid electric vehicle (FCHEV). Finally, all learned knowledge in the source strategy is fully transferred and reused for a heavy-duty fuel cell hybrid electric bus (FCHEB) to get the target eco-driving strategy. Experimental simulations show that the proposed framework can expedite the development of the eco-driving strategy for FCHEB by 94.83% while reducing hydrogen consumption by 10.05%.

Suggested Citation

  • Huang, Ruchen & He, Hongwen & Su, Qicong, 2024. "An intelligent full-knowledge transferable collaborative eco-driving framework based on improved soft actor-critic algorithm," Applied Energy, Elsevier, vol. 375(C).
    • Handle: RePEc:eee:appene:v:375:y:2024:i:c:s0306261924014612
    DOI: 10.1016/j.apenergy.2024.124078
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261924014612
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2024.124078?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Chen, Zheng & Wu, Simin & Shen, Shiquan & Liu, Yonggang & Guo, Fengxiang & Zhang, Yuanjian, 2023. "Co-optimization of velocity planning and energy management for autonomous plug-in hybrid electric vehicles in urban driving scenarios," Energy, Elsevier, vol. 263(PF).
    2. Huang, Ruchen & He, Hongwen & Su, Qicong, 2024. "Towards a fossil-free urban transport system: An intelligent cross-type transferable energy management framework based on deep transfer reinforcement learning," Applied Energy, Elsevier, vol. 363(C).
    3. Wang, Yong & Wu, Yuankai & Tang, Yingjuan & Li, Qin & He, Hongwen, 2023. "Cooperative energy management and eco-driving of plug-in hybrid electric vehicle via multi-agent reinforcement learning," Applied Energy, Elsevier, vol. 332(C).
    4. Lin, Xinyou & Xu, Xinhao & Wang, Zhaorui, 2022. "Deep Q-learning network based trip pattern adaptive battery longevity-conscious strategy of plug-in fuel cell hybrid electric vehicle," Applied Energy, Elsevier, vol. 321(C).
    5. Xie, Shaobo & Hu, Xiaosong & Liu, Teng & Qi, Shanwei & Lang, Kun & Li, Huiling, 2019. "Predictive vehicle-following power management for plug-in hybrid electric vehicles," Energy, Elsevier, vol. 166(C), pages 701-714.
    6. Xue, Jiaqi & Jiao, Xiaohong & Yu, Danmei & Zhang, Yahui, 2023. "Predictive hierarchical eco-driving control involving speed planning and energy management for connected plug-in hybrid electric vehicles," Energy, Elsevier, vol. 283(C).
    7. Wang, Siyang & Lin, Xianke, 2020. "Eco-driving control of connected and automated hybrid vehicles in mixed driving scenarios," Applied Energy, Elsevier, vol. 271(C).
    8. Wang, Hanchen & Ye, Yiming & Zhang, Jiangfeng & Xu, Bin, 2023. "A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle," Energy, Elsevier, vol. 266(C).
    9. Tang, Wenbin & Wang, Yaqian & Jiao, Xiaohong & Ren, Lina, 2023. "Hierarchical energy management strategy based on adaptive dynamic programming for hybrid electric vehicles in car-following scenarios," Energy, Elsevier, vol. 265(C).
    10. Zhang, Hailong & Peng, Jiankun & Dong, Hanxuan & Tan, Huachun & Ding, Fan, 2023. "Hierarchical reinforcement learning based energy management strategy of plug-in hybrid electric vehicle for ecological car-following process," Applied Energy, Elsevier, vol. 333(C).
    11. Huang, Ruchen & He, Hongwen & Su, Qicong & Härtl, Martin & Jaensch, Malte, 2024. "Enabling cross-type full-knowledge transferable energy management for hybrid electric vehicles via deep transfer reinforcement learning," Energy, Elsevier, vol. 305(C).
    12. Huang, Ruchen & He, Hongwen & Gao, Miaojue, 2023. "Training-efficient and cost-optimal energy management for fuel cell hybrid electric bus based on a novel distributed deep reinforcement learning framework," Applied Energy, Elsevier, vol. 346(C).
    13. Xie, Shaobo & Qi, Shanwei & Lang, Kun & Tang, Xiaolin & Lin, Xianke, 2020. "Coordinated management of connected plug-in hybrid electric buses for energy saving, inter-vehicle safety, and battery health," Applied Energy, Elsevier, vol. 268(C).
    14. Sun, Chao & Zhang, Chuntao & Sun, Fengchun & Zhou, Xingyu, 2022. "Stochastic co-optimization of speed planning and powertrain control with dynamic probabilistic constraints for safe and ecological driving," Applied Energy, Elsevier, vol. 325(C).
    15. Xiong, Rui & Sun, Wanzhou & Yu, Quanqing & Sun, Fengchun, 2020. "Research progress, challenges and prospects of fault diagnosis on battery system of electric vehicles," Applied Energy, Elsevier, vol. 279(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhang, Hao & Chen, Boli & Lei, Nuo & Li, Bingbing & Chen, Chaoyi & Wang, Zhi, 2024. "Coupled velocity and energy management optimization of connected hybrid electric vehicles for maximum collective efficiency," Applied Energy, Elsevier, vol. 360(C).
    2. Zhang, Baodi & Chang, Liang & Teng, Teng & Chen, Qifang & Li, Qiangwei & Cao, Yaoguang & Yang, Shichun & Zhang, Xin, 2024. "Multi-objective optimization with Q-learning for cruise and power allocation control parameters of connected fuel cell hybrid vehicles," Applied Energy, Elsevier, vol. 373(C).
    3. Niu, Zegong & He, Hongwen, 2024. "A data-driven solution for intelligent power allocation of connected hybrid electric vehicles inspired by offline deep reinforcement learning in V2X scenario," Applied Energy, Elsevier, vol. 372(C).
    4. Li, Jie & Fotouhi, Abbas & Pan, Wenjun & Liu, Yonggang & Zhang, Yuanjian & Chen, Zheng, 2023. "Deep reinforcement learning-based eco-driving control for connected electric vehicles at signalized intersections considering traffic uncertainties," Energy, Elsevier, vol. 279(C).
    5. Shi, Dehua & Xu, Han & Wang, Shaohua & Hu, Jia & Chen, Long & Yin, Chunfang, 2024. "Deep reinforcement learning based adaptive energy management for plug-in hybrid electric vehicle with double deep Q-network," Energy, Elsevier, vol. 305(C).
    6. Liu, Rui & Liu, Hui & Han, Lijin & Nie, Shida & Ruan, Shumin & Yang, Ningkang, 2023. "Predictive eco-driving strategy for hybrid electric vehicles on off-road terrain considering vehicle stability constraint," Applied Energy, Elsevier, vol. 350(C).
    7. Liu, Huimin & Lin, Cheng & Yu, Xiao & Tao, Zhenyi & Xu, Jiaqi, 2024. "Variable horizon multivariate driving pattern recognition framework based on vehicle-road two-dimensional information for electric vehicle," Applied Energy, Elsevier, vol. 365(C).
    8. Li, Cheng & Xu, Xiangyang & Zhu, Helong & Gan, Jiongpeng & Chen, Zhige & Tang, Xiaolin, 2024. "Research on car-following control and energy management strategy of hybrid electric vehicles in connected scene," Energy, Elsevier, vol. 293(C).
    9. Cui, Wei & Cui, Naxin & Li, Tao & Cui, Zhongrui & Du, Yi & Zhang, Chenghui, 2022. "An efficient multi-objective hierarchical energy management strategy for plug-in hybrid electric vehicle in connected scenario," Energy, Elsevier, vol. 257(C).
    10. Li, Daofei & Jiang, Yangye & Shen, Yijie, 2024. "Intersection eco-driving for automated vehicles: SMPC-based strategies for handling leading vehicle starting-up uncertainties," Energy, Elsevier, vol. 302(C).
    11. Ren, Xiaoxia & Ye, Jinze & Xie, Liping & Lin, Xinyou, 2024. "Battery longevity-conscious energy management predictive control strategy optimized by using deep reinforcement learning algorithm for a fuel cell hybrid electric vehicle," Energy, Elsevier, vol. 286(C).
    12. Ma, Yan & Ma, Qian & Liu, Yongqin & Gao, Jinwu & Chen, Hong, 2024. "Two-level optimization strategy for vehicle speed and battery thermal management in connected and automated EVs," Applied Energy, Elsevier, vol. 361(C).
    13. He, Hongwen & Su, Qicong & Huang, Ruchen & Niu, Zegong, 2024. "Enabling intelligent transferable energy management of series hybrid electric tracked vehicle across motion dimensions via soft actor-critic algorithm," Energy, Elsevier, vol. 294(C).
    14. Huang, Ruchen & He, Hongwen & Su, Qicong & Härtl, Martin & Jaensch, Malte, 2024. "Enabling cross-type full-knowledge transferable energy management for hybrid electric vehicles via deep transfer reinforcement learning," Energy, Elsevier, vol. 305(C).
    15. Wu, Yue & Huang, Zhiwu & Hofmann, Heath & Liu, Yongjie & Huang, Jiahao & Hu, Xiaosong & Peng, Jun & Song, Ziyou, 2022. "Hierarchical predictive control for electric vehicles with hybrid energy storage system under vehicle-following scenarios," Energy, Elsevier, vol. 251(C).
    16. Wilberforce, Tabbi & Anser, Afaaq & Swamy, Jangam Aishwarya & Opoku, Richard, 2023. "An investigation into hybrid energy storage system control and power distribution for hybrid electric vehicles," Energy, Elsevier, vol. 279(C).
    17. Chen, Zheng & Wu, Simin & Shen, Shiquan & Liu, Yonggang & Guo, Fengxiang & Zhang, Yuanjian, 2023. "Co-optimization of velocity planning and energy management for autonomous plug-in hybrid electric vehicles in urban driving scenarios," Energy, Elsevier, vol. 263(PF).
    18. Peng, Jiankun & Shen, Yang & Wu, ChangCheng & Wang, Chunhai & Yi, Fengyan & Ma, Chunye, 2023. "Research on energy-saving driving control of hydrogen fuel bus based on deep reinforcement learning in freeway ramp weaving area," Energy, Elsevier, vol. 285(C).
    19. Li, Menglin & Yin, Long & Yan, Mei & Wu, Jingda & He, Hongwe & Jia, Chunchun, 2024. "Hierarchical intelligent energy-saving control strategy for fuel cell hybrid electric buses based on traffic flow predictions," Energy, Elsevier, vol. 304(C).
    20. Pietro Stabile & Federico Ballo & Giorgio Previati & Giampiero Mastinu & Massimiliano Gobbi, 2023. "Eco-Driving Strategy Implementation for Ultra-Efficient Lightweight Electric Vehicles in Realistic Driving Scenarios," Energies, MDPI, vol. 16(3), pages 1-19, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:375:y:2024:i:c:s0306261924014612. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.