IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v306y2024ics0360544224023053.html
   My bibliography  Save this article

Deep reinforcement learning with deep-Q-network based energy management for fuel cell hybrid electric truck

Author

Listed:
  • Wang, Zhifu
  • Zhang, Shunshun
  • Luo, Wei
  • Xu, Song

Abstract

To better utilize hydrogen energy from fuel cell electric vehicles (FCEVs), an investigation is conducted into how a proton exchange membrane fuel cell (PEMFC) hybrid vehicle's 100-km hydrogen consumption rate is affected by a dynamic programming-based energy management method. The Dueling Deep Q-Network (Dueling DQN) algorithm's energy management approach is proposed. The DQN algorithm is optimized to increase the method's stability and quicken its pace of convergence. The simulation results show that the fuel economy of DQN and Dueling DQN algorithm are 94.28 % and 95.7 % respectively, both of which are improved, while the Dueling DQN algorithm converges at 480 steps higher than the DQN algorithm converges at 800 steps, and the algorithm comparison verifies the two algorithms' The validity of both algorithms was verified by algorithm comparison. In China Truck Driving Conditions (CHTC-LT), the hardware-in-the-loop simulation of CAN communication protocol using NI-PXI hardware-in-the-loop test system achieves the target vehicle speed following and verifies the real-time performance of the strategy.

Suggested Citation

  • Wang, Zhifu & Zhang, Shunshun & Luo, Wei & Xu, Song, 2024. "Deep reinforcement learning with deep-Q-network based energy management for fuel cell hybrid electric truck," Energy, Elsevier, vol. 306(C).
    • Handle: RePEc:eee:energy:v:306:y:2024:i:c:s0360544224023053
    DOI: 10.1016/j.energy.2024.132531
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544224023053
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2024.132531?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ganesh, Akhil Hannegudda & Xu, Bin, 2022. "A review of reinforcement learning based energy management systems for electrified powertrains: Progress, challenge, and potential solution," Renewable and Sustainable Energy Reviews, Elsevier, vol. 154(C).
    2. İnci, Mustafa & Büyük, Mehmet & Demir, Mehmet Hakan & İlbey, Göktürk, 2021. "A review and research on fuel cell electric vehicles: Topologies, power electronic converters, energy management methods, technical challenges, marketing and future aspects," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    3. Renzhi Lyu & Zhenpo Wang & Zhaosheng Zhang, 2024. "Multi-Objective Optimization Strategy for Fuel Cell Hybrid Electric Trucks Based on Driving Patern Recognition," Energies, MDPI, vol. 17(6), pages 1-15, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. He, Hongwen & Meng, Xiangfei & Wang, Yong & Khajepour, Amir & An, Xiaowen & Wang, Renguang & Sun, Fengchun, 2024. "Deep reinforcement learning based energy management strategies for electrified vehicles: Recent advances and perspectives," Renewable and Sustainable Energy Reviews, Elsevier, vol. 192(C).
    2. Sofiane Bacha & Ramzi Saadi & Mohamed Yacine Ayad & Mohamed Sahraoui & Khaled Laadjal & Antonio J. Marques Cardoso, 2023. "Autonomous Electric-Vehicle Control Using Speed Planning Algorithm and Back-Stepping Approach," Energies, MDPI, vol. 16(5), pages 1-26, March.
    3. Sumitkumar, Rathor & Al-Sumaiti, Ameena Saad, 2024. "Shared autonomous electric vehicle: Towards social economy of energy and mobility from power-transportation nexus perspective," Renewable and Sustainable Energy Reviews, Elsevier, vol. 197(C).
    4. Matteo Acquarone & Claudio Maino & Daniela Misul & Ezio Spessa & Antonio Mastropietro & Luca Sorrentino & Enrico Busto, 2023. "Influence of the Reward Function on the Selection of Reinforcement Learning Agents for Hybrid Electric Vehicles Real-Time Control," Energies, MDPI, vol. 16(6), pages 1-22, March.
    5. Huang, Ruchen & He, Hongwen & Gao, Miaojue, 2023. "Training-efficient and cost-optimal energy management for fuel cell hybrid electric bus based on a novel distributed deep reinforcement learning framework," Applied Energy, Elsevier, vol. 346(C).
    6. Youssef Amry & Elhoussin Elbouchikhi & Franck Le Gall & Mounir Ghogho & Soumia El Hani, 2022. "Electric Vehicle Traction Drives and Charging Station Power Electronics: Current Status and Challenges," Energies, MDPI, vol. 15(16), pages 1-30, August.
    7. Ma, Yan & Hu, Fuyuan & Hu, Yunfeng, 2023. "Energy efficiency improvement of intelligent fuel cell/battery hybrid vehicles through an integrated management strategy," Energy, Elsevier, vol. 263(PE).
    8. Gao, Sichen & Zong, Yuhua & Ju, Fei & Wang, Qun & Huo, Weiwei & Wang, Liangmo & Wang, Tao, 2024. "Scenario-oriented adaptive ECMS using speed prediction for fuel cell vehicles in real-world driving," Energy, Elsevier, vol. 304(C).
    9. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    10. Georgios Varvoutis & Athanasios Lampropoulos & Evridiki Mandela & Michalis Konsolakis & George E. Marnellos, 2022. "Recent Advances on CO 2 Mitigation Technologies: On the Role of Hydrogenation Route via Green H 2," Energies, MDPI, vol. 15(13), pages 1-38, June.
    11. Chen, Kui & Badji, Abderrezak & Laghrouche, Salah & Djerdir, Abdesslem, 2022. "Polymer electrolyte membrane fuel cells degradation prediction using multi-kernel relevance vector regression and whale optimization algorithm," Applied Energy, Elsevier, vol. 318(C).
    12. Pang, Kexin & Zhou, Jian & Tsianikas, Stamatis & Coit, David W. & Ma, Yizhong, 2024. "Long-term microgrid expansion planning with resilience and environmental benefits using deep reinforcement learning," Renewable and Sustainable Energy Reviews, Elsevier, vol. 191(C).
    13. Benmouna, A. & Becherif, M. & Boulon, L. & Dépature, C. & Ramadan, Haitham S., 2021. "Efficient experimental energy management operating for FC/battery/SC vehicles via hybrid Artificial Neural Networks-Passivity Based Control," Renewable Energy, Elsevier, vol. 178(C), pages 1291-1302.
    14. Tanveer, Waqas Hassan & Abdelkareem, Mohammad Ali & Kolosz, Ben W. & Rezk, Hegazy & Andresen, John & Cha, Suk Won & Sayed, Enas Taha, 2021. "The role of vacuum based technologies in solid oxide fuel cell development to utilize industrial waste carbon for power production," Renewable and Sustainable Energy Reviews, Elsevier, vol. 142(C).
    15. Xia, Zhifeng & Chen, Huicui & Zhang, Ruirui & Weng, Qianyao & Zhang, Tong & Pei, Pucheng, 2023. "Behavior analysis of PEMFC with geometric configuration variation during multiple-step loading reduction process," Applied Energy, Elsevier, vol. 349(C).
    16. Badji, Abderrezak & Abdeslam, Djaffar Ould & Chabane, Djafar & Benamrouche, Nacereddine, 2022. "Real-time implementation of improved power frequency approach based energy management of fuel cell electric vehicle considering storage limitations," Energy, Elsevier, vol. 249(C).
    17. Hu, Dong & Huang, Chao & Yin, Guodong & Li, Yangmin & Huang, Yue & Huang, Hailong & Wu, Jingda & Li, Wenfei & Xie, Hui, 2024. "A transfer-based reinforcement learning collaborative energy management strategy for extended-range electric buses with cabin temperature comfort consideration," Energy, Elsevier, vol. 290(C).
    18. Büyük, Mehmet & İnci, Mustafa, 2023. "Improved drift-free P&O MPPT method to enhance energy harvesting capability for dynamic operating conditions of fuel cells," Energy, Elsevier, vol. 267(C).
    19. Zhou, Hongxu & Yu, Zhongwei & Wu, Xiaohua & Fan, Zhanfeng & Yin, Xiaofeng & Zhou, Lingxue, 2023. "Dynamic programming improved online fuzzy power distribution in a demonstration fuel cell hybrid bus," Energy, Elsevier, vol. 284(C).
    20. Hao, Zhaojun & Di Maio, Francesco & Zio, Enrico, 2023. "A sequential decision problem formulation and deep reinforcement learning solution of the optimization of O&M of cyber-physical energy systems (CPESs) for reliable and safe power production and supply," Reliability Engineering and System Safety, Elsevier, vol. 235(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:306:y:2024:i:c:s0360544224023053. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.