IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v305y2024ics0360544224019467.html
   My bibliography  Save this article

Energy management strategy for electro-hydraulic hybrid electric vehicles considering optimal mode switching: A soft actor-critic approach trained on a multi-modal driving cycle

Author

Listed:
  • Zhou, Jie
  • Zhang, Tiezhu
  • Zhang, Hongxin
  • Zhang, Zhen
  • Hong, Jichao
  • Yang, Jian

Abstract

Hybrid electric vehicles (HEVs) feature multiple working modes. Thoughtful selection of these modes can optimally balance driving performance, power demands, and energy consumption, thereby enhancing the overall efficiency of the vehicle. This paper presents a soft actor-critic (SAC) approach trained on a multi-modal driving cycle (MDC) for selecting operational modes of electro-hydraulic hybrid electric vehicle (EHHEV). Firstly, characteristic parameters are extracted and clustered for five typical driving cycles through principal component analysis and K-means clustering, creating a multi-modal driving cycle. Secondly, based on the operational characteristics of EHHEV, state variables, action variables, reward functions, learning rates, and other parameters are set for the SAC algorithm, and the EMS framework is built based on the electro-hydraulic hybrid electric power system. Subsequently, the SAC algorithm is trained using the MDC to construct the SAC-MDC EMS. Results demonstrate that compared to EV, RB EMS, and SAC EMS, IREC achieves maximum improvements of 22.38 %, 5.55 % and 0.80 %, respectively. The dynamic performance and the motor load optimization capability are also enhanced. To further validate the practicality and reliability of the SAC-MDC EMS, this paper validates it using actual driving data, revealing that it still exhibits outstanding performance.

Suggested Citation

  • Zhou, Jie & Zhang, Tiezhu & Zhang, Hongxin & Zhang, Zhen & Hong, Jichao & Yang, Jian, 2024. "Energy management strategy for electro-hydraulic hybrid electric vehicles considering optimal mode switching: A soft actor-critic approach trained on a multi-modal driving cycle," Energy, Elsevier, vol. 305(C).
    • Handle: RePEc:eee:energy:v:305:y:2024:i:c:s0360544224019467
    DOI: 10.1016/j.energy.2024.132172
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544224019467
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2024.132172?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. He, Jiaxin & Li, Jingyi & Zhao, Daiqing & Chen, Xing, 2022. "Does oil price affect corporate innovation? Evidence from new energy vehicle enterprises in China," Renewable and Sustainable Energy Reviews, Elsevier, vol. 156(C).
    2. Zongjun Yin & Xuegang Ma & Chunying Zhang & Rong Su & Qingqing Wang, 2023. "A Logic Threshold Control Strategy to Improve the Regenerative Braking Energy Recovery of Electric Vehicles," Sustainability, MDPI, vol. 15(24), pages 1-33, December.
    3. Qiu, Dawei & Wang, Yi & Hua, Weiqi & Strbac, Goran, 2023. "Reinforcement learning for electric vehicle applications in power systems:A critical review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 173(C).
    4. Maroto Estrada, Pedro & de Lima, Daniela & Bauer, Peter H. & Mammetti, Marco & Bruno, Joan Carles, 2023. "Deep learning in the development of energy Management strategies of hybrid electric Vehicles: A hybrid modeling approach," Applied Energy, Elsevier, vol. 329(C).
    5. Yang, Jian & Liu, Bo & Zhang, Tiezhu & Hong, Jichao & Zhang, Hongxin, 2023. "Multi-parameter controlled mechatronics-electro-hydraulic power coupling electric vehicle based on active energy regulation," Energy, Elsevier, vol. 263(PC).
    6. Li, Lin & Zhang, Tiezhu & Lu, Liqun & Zhang, Hongxin & Yang, Jian & Zhang, Zhen, 2023. "An energy active regulation management strategy based on driving mode recognition for electro-hydraulic hybrid vehicles," Energy, Elsevier, vol. 285(C).
    7. Liang, Zhaowen & Ruan, Jiageng & Wang, Zhenpo & Liu, Kai & Li, Bin, 2024. "Soft actor-critic-based EMS design for dual motor battery electric bus," Energy, Elsevier, vol. 288(C).
    8. Tianxiao Wang & Zhecheng Jing & Shupei Zhang & Chengqun Qiu, 2023. "Utilizing Principal Component Analysis and Hierarchical Clustering to Develop Driving Cycles: A Case Study in Zhenjiang," Sustainability, MDPI, vol. 15(6), pages 1-13, March.
    9. Zhang, Zhen & Zhang, Tiezhu & Hong, Jichao & Zhang, Hongxin & Yang, Jian & Jia, Qingxiao, 2023. "Double deep Q-network guided energy management strategy of a novel electric-hydraulic hybrid electric vehicle," Energy, Elsevier, vol. 269(C).
    10. Wang, Hanchen & Ye, Yiming & Zhang, Jiangfeng & Xu, Bin, 2023. "A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle," Energy, Elsevier, vol. 266(C).
    11. Yang, Jian & Zhang, Tiezhu & Hong, Jichao & Zhang, Hongxin & Zhao, Qinghai & Meng, Zewen, 2021. "Research on driving control strategy and Fuzzy logic optimization of a novel mechatronics-electro-hydraulic power coupling electric vehicle," Energy, Elsevier, vol. 233(C).
    12. Zhou, Quan & Du, Changqing & Wu, Dongmei & Huang, Cheng & Yan, Fuwu, 2023. "A tolerant sequential correction predictive energy management strategy of hybrid electric vehicles with adaptive mesh discretization," Energy, Elsevier, vol. 274(C).
    13. Zou, Weitao & Li, Jianwei & Yang, Qingqing & Wan, Xinming & He, Yuntang & Lan, Hao, 2023. "A real-time energy management approach with fuel cell and battery competition-synergy control for the fuel cell vehicle," Applied Energy, Elsevier, vol. 334(C).
    14. Ruan, Jiageng & Wu, Changcheng & Liang, Zhaowen & Liu, Kai & Li, Bin & Li, Weihan & Li, Tongyang, 2023. "The application of machine learning-based energy management strategy in a multi-mode plug-in hybrid electric vehicle, part II: Deep deterministic policy gradient algorithm design for electric mode," Energy, Elsevier, vol. 269(C).
    15. Sun, Wenjing & Zou, Yuan & Zhang, Xudong & Guo, Ningyuan & Zhang, Bin & Du, Guodong, 2022. "High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning," Energy, Elsevier, vol. 258(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Angel Recalde & Ricardo Cajo & Washington Velasquez & Manuel S. Alvarez-Alvarado, 2024. "Machine Learning and Optimization in Energy Management Systems for Plug-In Hybrid Electric Vehicles: A Comprehensive Review," Energies, MDPI, vol. 17(13), pages 1-39, June.
    2. Jin, Rui & Li, Lei & Liang, Xiaoling & Zou, Xiang & Yang, Zeyuan & Ge, Shuzhi Sam & Huang, Haihong, 2024. "Energy-efficient design of the powertrain for mechanical-electro-hydraulic equipment via configuring multidimensional controllable variables," Renewable and Sustainable Energy Reviews, Elsevier, vol. 201(C).
    3. Shi, Dehua & Xu, Han & Wang, Shaohua & Hu, Jia & Chen, Long & Yin, Chunfang, 2024. "Deep reinforcement learning based adaptive energy management for plug-in hybrid electric vehicle with double deep Q-network," Energy, Elsevier, vol. 305(C).
    4. Hong, Jichao & Zhang, Tiezhu & Zhang, Zhen & Zhang, Hongxin, 2023. "Investigation of energy management strategy for a novel electric-hydraulic hybrid vehicle: Self-adaptive electric-hydraulic ratio," Energy, Elsevier, vol. 278(C).
    5. Zhang, Hao & Lei, Nuo & Liu, Shang & Fan, Qinhao & Wang, Zhi, 2023. "Data-driven predictive energy consumption minimization strategy for connected plug-in hybrid electric vehicles," Energy, Elsevier, vol. 283(C).
    6. Alessia Musa & Pier Giuseppe Anselma & Giovanni Belingardi & Daniela Anna Misul, 2023. "Energy Management in Hybrid Electric Vehicles: A Q-Learning Solution for Enhanced Drivability and Energy Efficiency," Energies, MDPI, vol. 17(1), pages 1-20, December.
    7. Li, Lin & Zhang, Tiezhu & Lu, Liqun & Zhang, Hongxin & Yang, Jian & Zhang, Zhen, 2023. "An energy active regulation management strategy based on driving mode recognition for electro-hydraulic hybrid vehicles," Energy, Elsevier, vol. 285(C).
    8. Niu, Zegong & He, Hongwen, 2024. "A data-driven solution for intelligent power allocation of connected hybrid electric vehicles inspired by offline deep reinforcement learning in V2X scenario," Applied Energy, Elsevier, vol. 372(C).
    9. He, Hongwen & Su, Qicong & Huang, Ruchen & Niu, Zegong, 2024. "Enabling intelligent transferable energy management of series hybrid electric tracked vehicle across motion dimensions via soft actor-critic algorithm," Energy, Elsevier, vol. 294(C).
    10. Zhou, Yujie & Huang, Yin & Mao, Xuping & Kang, Zehao & Huang, Xuejin & Xuan, Dongji, 2024. "Research on energy management strategy of fuel cell hybrid power via an improved TD3 deep reinforcement learning," Energy, Elsevier, vol. 293(C).
    11. Peng, Jiankun & Shen, Yang & Wu, ChangCheng & Wang, Chunhai & Yi, Fengyan & Ma, Chunye, 2023. "Research on energy-saving driving control of hydrogen fuel bus based on deep reinforcement learning in freeway ramp weaving area," Energy, Elsevier, vol. 285(C).
    12. Liang, Zhaowen & Ruan, Jiageng & Wang, Zhenpo & Liu, Kai & Li, Bin, 2024. "Soft actor-critic-based EMS design for dual motor battery electric bus," Energy, Elsevier, vol. 288(C).
    13. Shi, Ting & Wang, Huaiyu & Yang, Wenming & Peng, Xueyuan, 2024. "Mathematical modeling and optimization of gas foil bearings-rotor system in hydrogen fuel cell vehicles," Energy, Elsevier, vol. 290(C).
    14. Zhang, Hao & Chen, Boli & Lei, Nuo & Li, Bingbing & Chen, Chaoyi & Wang, Zhi, 2024. "Coupled velocity and energy management optimization of connected hybrid electric vehicles for maximum collective efficiency," Applied Energy, Elsevier, vol. 360(C).
    15. Kang, Hyuna & Jung, Seunghoon & Kim, Hakpyeong & Jeoung, Jaewon & Hong, Taehoon, 2024. "Reinforcement learning-based optimal scheduling model of battery energy storage system at the building level," Renewable and Sustainable Energy Reviews, Elsevier, vol. 190(PA).
    16. Stefan Tabacu & Dragos Popa, 2023. "Backward-Facing Analysis for the Preliminary Estimation of the Vehicle Fuel Consumption," Sustainability, MDPI, vol. 15(6), pages 1-19, March.
    17. Kakkar, Riya & Agrawal, Smita & Tanwar, Sudeep, 2024. "A systematic survey on demand response management schemes for electric vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 203(C).
    18. Guo, Kaiyuan & Huang, Chendan & Zhang, Zhenjun & Diaz Paiz, Ana Yamileth & Chen, Weiming, 2024. "The impact of new energy industry on environmental and economic benefits: Evidence from China," Energy, Elsevier, vol. 304(C).
    19. Cui, Feifei & An, Dou & Xi, Huan, 2024. "Integrated energy hub dispatch with a multi-mode CAES–BESS hybrid system: An option-based hierarchical reinforcement learning approach," Applied Energy, Elsevier, vol. 374(C).
    20. Nafiseh Mazaheri & Daniel Santamargarita & Emilio Bueno & Daniel Pizarro & Santiago Cobreces, 2024. "A Deep Reinforcement Learning Approach to DC-DC Power Electronic Converter Control with Practical Considerations," Energies, MDPI, vol. 17(14), pages 1-22, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:305:y:2024:i:c:s0360544224019467. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.