IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v301y2024ics0360544224013355.html
   My bibliography  Save this article

Online energy management strategy for ammonia-hydrogen hybrid electric vehicles harnessing deep reinforcement learning

Author

Listed:
  • Chen, Fujun
  • Wang, Bowen
  • Ni, Meng
  • Gong, Zhichao
  • Jiao, Kui

Abstract

Powertrain electrification and fuel decarbonization play pivotal roles in the pursuit of carbon peak and carbon neutrality within the realm of transportation. Ammonia and hydrogen are essential carbon-neutral fuels suited for deployment in heavy-duty, long-haul transportation applications. The technological complementarity of ammonia and hydrogen can be realized in the current context by combining various energy conversion devices. Energy management strategy (EMS) is one of the critical technologies in hybrid systems. In this study, a novel EMS based on deep reinforcement learning (DRL) is proposed for a heavy-duty automotive hybrid system containing an ammonia-hydrogen internal combustion engine (ICE), a fuel cell system (FCS), an ammonia electrolysis cell (AEC), and Li-ion batteries (LB). To address the protracted training times and convergence challenges intrinsic to DRL, a fuzzy logic control (FLC) is introduced to provide expert demonstrations during the DRL agent's learning process. The incorporation of FLC not only facilitates the convergence speed of the DRL algorithm, but also strikes a good balance between the operation efficiency improvement and battery SOC maintenance. The simulation results show that the proposed EMS improves average efficiency by 2 % with a modest 0.75 % SOC reduction across four operational scenarios compared to the FLC-based EMS. After thorough validation of the proposed EMS, a parametric study is conducted to examine three key parameters: charge mode threshold, FCS power change rate limit, and hybrid mode threshold. The study aims to offer valuable insights for formulating the EMS of the ammonia-hydrogen hybrid system.

Suggested Citation

  • Chen, Fujun & Wang, Bowen & Ni, Meng & Gong, Zhichao & Jiao, Kui, 2024. "Online energy management strategy for ammonia-hydrogen hybrid electric vehicles harnessing deep reinforcement learning," Energy, Elsevier, vol. 301(C).
    • Handle: RePEc:eee:energy:v:301:y:2024:i:c:s0360544224013355
    DOI: 10.1016/j.energy.2024.131562
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544224013355
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2024.131562?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Frans J. R. Verbruggen & Emilia Silvas & Theo Hofman, 2020. "Electric Powertrain Topology Analysis and Design for Heavy-Duty Trucks," Energies, MDPI, vol. 13(10), pages 1-30, May.
    2. Liu, Yonggang & Wu, Yitao & Wang, Xiangyu & Li, Liang & Zhang, Yuanjian & Chen, Zheng, 2023. "Energy management for hybrid electric vehicles based on imitation reinforcement learning," Energy, Elsevier, vol. 263(PC).
    3. Kui Jiao & Jin Xuan & Qing Du & Zhiming Bao & Biao Xie & Bowen Wang & Yan Zhao & Linhao Fan & Huizhi Wang & Zhongjun Hou & Sen Huo & Nigel P. Brandon & Yan Yin & Michael D. Guiver, 2021. "Designing the next generation of proton-exchange membrane fuel cells," Nature, Nature, vol. 595(7867), pages 361-369, July.
    4. Kanaan, Riham & Affonso Nóbrega, Pedro Henrique & Achard, Patrick & Beauger, Christian, 2023. "Economical assessment comparison for hydrogen reconversion from ammonia using thermal decomposition and electrolysis," Renewable and Sustainable Energy Reviews, Elsevier, vol. 188(C).
    5. Zhang, Kaixuan & Ruan, Jiageng & Li, Tongyang & Cui, Hanghang & Wu, Changcheng, 2023. "The effects investigation of data-driven fitting cycle and deep deterministic policy gradient algorithm on energy management strategy of dual-motor electric bus," Energy, Elsevier, vol. 269(C).
    6. Sun, Wenjing & Zou, Yuan & Zhang, Xudong & Guo, Ningyuan & Zhang, Bin & Du, Guodong, 2022. "High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning," Energy, Elsevier, vol. 258(C).
    7. Ezzat, M.F & Dincer, I., 2018. "Development and assessment of a new hybrid vehicle with ammonia and hydrogen," Applied Energy, Elsevier, vol. 219(C), pages 226-239.
    8. Guo, Xiaokai & Yan, Xianguo & Chen, Zhi & Meng, Zhiyu, 2022. "Research on energy management strategy of heavy-duty fuel cell hybrid vehicles based on dueling-double-deep Q-network," Energy, Elsevier, vol. 260(C).
    9. Tran, Dai-Duong & Vafaeipour, Majid & El Baghdadi, Mohamed & Barrero, Ricardo & Van Mierlo, Joeri & Hegazy, Omar, 2020. "Thorough state-of-the-art analysis of electric and hybrid vehicle powertrains: Topologies and integrated energy management strategies," Renewable and Sustainable Energy Reviews, Elsevier, vol. 119(C).
    10. Anselma, Pier Giuseppe & Belingardi, Giovanni, 2022. "Fuel cell electrified propulsion systems for long-haul heavy-duty trucks: present and future cost-oriented sizing," Applied Energy, Elsevier, vol. 321(C).
    11. Wang, Hao & He, Hongwen & Bai, Yunfei & Yue, Hongwei, 2022. "Parameterized deep Q-network based energy management with balanced energy economy and battery life for hybrid electric vehicles," Applied Energy, Elsevier, vol. 320(C).
    12. Riham Kanaan & Pedro Henrique Affonso Nóbrega & Patrick Achard & Christian Beauger, 2023. "Economical assessment comparison for hydrogen reconversion from ammonia using thermal decomposition and electrolysis," Post-Print hal-04337525, HAL.
    13. Zhou, Jianhao & Xue, Siwu & Xue, Yuan & Liao, Yuhui & Liu, Jun & Zhao, Wanzhong, 2021. "A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning," Energy, Elsevier, vol. 224(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhang, Hao & Lei, Nuo & Wang, Zhi, 2024. "Ammonia-hydrogen propulsion system for carbon-free heavy-duty vehicles," Applied Energy, Elsevier, vol. 369(C).
    2. Jinglin Li & Bowen Sheng & Yiqing Chen & Jiajia Yang & Ping Wang & Yixin Li & Tianqi Yu & Hu Pan & Liang Qiu & Ying Li & Jun Song & Lei Zhu & Xinqiang Wang & Zhen Huang & Baowen Zhou, 2024. "Utilizing full-spectrum sunlight for ammonia decomposition to hydrogen over GaN nanowires-supported Ru nanoparticles on silicon," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    3. Hu, Dong & Huang, Chao & Yin, Guodong & Li, Yangmin & Huang, Yue & Huang, Hailong & Wu, Jingda & Li, Wenfei & Xie, Hui, 2024. "A transfer-based reinforcement learning collaborative energy management strategy for extended-range electric buses with cabin temperature comfort consideration," Energy, Elsevier, vol. 290(C).
    4. Shantanu Pardhi & Sajib Chakraborty & Dai-Duong Tran & Mohamed El Baghdadi & Steven Wilkins & Omar Hegazy, 2022. "A Review of Fuel Cell Powertrains for Long-Haul Heavy-Duty Vehicles: Technology, Hydrogen, Energy and Thermal Management Solutions," Energies, MDPI, vol. 15(24), pages 1-55, December.
    5. Chi T. P. Nguyen & Bảo-Huy Nguyễn & Minh C. Ta & João Pedro F. Trovão, 2023. "Dual-Motor Dual-Source High Performance EV: A Comprehensive Review," Energies, MDPI, vol. 16(20), pages 1-28, October.
    6. Dong, Peng & Zhao, Junwei & Liu, Xuewu & Wu, Jian & Xu, Xiangyang & Liu, Yanfang & Wang, Shuhan & Guo, Wei, 2022. "Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies: Development stages, challenges, and future trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    7. Zuo, Jian & Steiner, Nadia Yousfi & Li, Zhongliang & Hissel, Daniel, 2024. "Health management review for fuel cells: Focus on action phase," Renewable and Sustainable Energy Reviews, Elsevier, vol. 201(C).
    8. Hu, Dong & Xie, Hui & Song, Kang & Zhang, Yuanyuan & Yan, Long, 2023. "An apprenticeship-reinforcement learning scheme based on expert demonstrations for energy management strategy of hybrid electric vehicles," Applied Energy, Elsevier, vol. 342(C).
    9. He, Hongwen & Meng, Xiangfei & Wang, Yong & Khajepour, Amir & An, Xiaowen & Wang, Renguang & Sun, Fengchun, 2024. "Deep reinforcement learning based energy management strategies for electrified vehicles: Recent advances and perspectives," Renewable and Sustainable Energy Reviews, Elsevier, vol. 192(C).
    10. Wang, Hanchen & Arjmandzadeh, Ziba & Ye, Yiming & Zhang, Jiangfeng & Xu, Bin, 2024. "FlexNet: A warm start method for deep reinforcement learning in hybrid electric vehicle energy management applications," Energy, Elsevier, vol. 288(C).
    11. Liu, Yonggang & Wu, Yitao & Wang, Xiangyu & Li, Liang & Zhang, Yuanjian & Chen, Zheng, 2023. "Energy management for hybrid electric vehicles based on imitation reinforcement learning," Energy, Elsevier, vol. 263(PC).
    12. Zhang, Yong & He, Shirong & Jiang, Xiaohui & Xiong, Mu & Ye, Yuntao & Yang, Xi, 2023. "Three-dimensional multi-phase simulation of proton exchange membrane fuel cell performance considering constriction straight channel," Energy, Elsevier, vol. 267(C).
    13. Shi, Dehua & Liu, Sheng & Cai, Yingfeng & Wang, Shaohua & Li, Haoran & Chen, Long, 2021. "Pontryagin’s minimum principle based fuzzy adaptive energy management for hybrid electric vehicle using real-time traffic information," Applied Energy, Elsevier, vol. 286(C).
    14. Pierpaolo Polverino & Ivan Arsie & Cesare Pianese, 2021. "Optimal Energy Management for Hybrid Electric Vehicles Based on Dynamic Programming and Receding Horizon," Energies, MDPI, vol. 14(12), pages 1-11, June.
    15. Ma, Zhikai & Huo, Qian & Wang, Wei & Zhang, Tao, 2023. "Voltage-temperature aware thermal runaway alarming framework for electric vehicles via deep learning with attention mechanism in time-frequency domain," Energy, Elsevier, vol. 278(C).
    16. Zhang, Xiaoqing & Yang, Jiapei & Ma, Xiao & Zhuge, Weilin & Shuai, Shijin, 2022. "Modelling and analysis on effects of penetration of microporous layer into gas diffusion layer in PEM fuel cells: Focusing on mass transport," Energy, Elsevier, vol. 254(PA).
    17. Zhang, Xin & Li, Jingwen & Xiong, Yi & Ang, Yee Sin, 2022. "Efficient harvesting of low-grade waste heat from proton exchange membrane fuel cells via thermoradiative power devices," Energy, Elsevier, vol. 258(C).
    18. Huang, Ruchen & He, Hongwen & Gao, Miaojue, 2023. "Training-efficient and cost-optimal energy management for fuel cell hybrid electric bus based on a novel distributed deep reinforcement learning framework," Applied Energy, Elsevier, vol. 346(C).
    19. Lu, Guolong & Fan, Wenxuan & Lu, Dafeng & Zhao, Taotao & Wu, Qianqian & Liu, Mingxin & Liu, Zhenning, 2024. "Lung-inspired hybrid flow field to enhance PEMFC performance: A case of dual optimization by response surface and artificial intelligence," Applied Energy, Elsevier, vol. 355(C).
    20. Yunjie Yang & Minli Bai & Laisuo Su & Jizu Lv & Chengzhi Hu & Linsong Gao & Yang Li & Yubai Li & Yongchen Song, 2022. "One-Dimensional Numerical Simulation of Pt-Co Alloy Catalyst Aging for Proton Exchange Membrane Fuel Cells," Sustainability, MDPI, vol. 14(18), pages 1-23, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:301:y:2024:i:c:s0360544224013355. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.