IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v14y2021i17p5270-d621674.html
   My bibliography  Save this article

Real-Time HEV Energy Management Strategy Considering Road Congestion Based on Deep Reinforcement Learning

Author

Listed:
  • Shota Inuzuka

    (Faculty of Science and Technology, Sophia University, Tokyo 102-8554, Japan)

  • Bo Zhang

    (Faculty of Science and Technology, Sophia University, Tokyo 102-8554, Japan)

  • Tielong Shen

    (Faculty of Science and Technology, Sophia University, Tokyo 102-8554, Japan)

Abstract

This paper deals with the HEV real-time energy management problem using deep reinforcement learning with connected technologies such as Vehicle to Vehicle (V2V) and Vehicle to Infrastructure (V2I). In the HEV energy management problem, it is important to run the engine efficiently in order to minimize its total energy cost. This research proposes a policy model that takes into account road congestion and aims to learn the optimal system mode selection and power distribution when considering the far future by policy-based reinforcement learning. In the simulation, a traffic environment is generated in a virtual space by IPG CarMaker and a HEV model is prepared in MATLAB/Simulink to calculate the energy cost while driving on the road environment. The simulation validation shows the versatility of the proposed method for the test data, and in addition, it shows that considering road congestion reduces the total cost and improves the learning speed. Furthermore, we compare the proposed method with model predictive control (MPC) under the same conditions and show that the proposed method obtains more global optimal solutions.

Suggested Citation

  • Shota Inuzuka & Bo Zhang & Tielong Shen, 2021. "Real-Time HEV Energy Management Strategy Considering Road Congestion Based on Deep Reinforcement Learning," Energies, MDPI, vol. 14(17), pages 1-20, August.
    • Handle: RePEc:gam:jeners:v:14:y:2021:i:17:p:5270-:d:621674
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/14/17/5270/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/14/17/5270/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zhang, Bo & Zhang, Jiangyan & Xu, Fuguo & Shen, Tielong, 2020. "Optimal control of power-split hybrid electric powertrains with minimization of energy consumption," Applied Energy, Elsevier, vol. 266(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yaqian Wang & Xiaohong Jiao, 2022. "Dual Heuristic Dynamic Programming Based Energy Management Control for Hybrid Electric Vehicles," Energies, MDPI, vol. 15(9), pages 1-19, April.
    2. Gwanggil Jeon, 2022. "Artificial Intelligence Approaches for Energies," Energies, MDPI, vol. 15(18), pages 1-3, September.
    3. Umberto Previti & Sebastian Brusca & Antonio Galvagno & Fabio Famoso, 2022. "Influence of Energy Management System Control Strategies on the Battery State of Health in Hybrid Electric Vehicles," Sustainability, MDPI, vol. 14(19), pages 1-20, September.
    4. He, Hongwen & Meng, Xiangfei & Wang, Yong & Khajepour, Amir & An, Xiaowen & Wang, Renguang & Sun, Fengchun, 2024. "Deep reinforcement learning based energy management strategies for electrified vehicles: Recent advances and perspectives," Renewable and Sustainable Energy Reviews, Elsevier, vol. 192(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhang, Bo & Zhang, Jiangyan & Shen, Tielong, 2022. "Optimal control design for comfortable-driving of hybrid electric vehicles in acceleration mode," Applied Energy, Elsevier, vol. 305(C).
    2. Singh, Somendra Pratap & Hanif, Athar & Ahmed, Qadeer & Meijer, Maarten & Lahti, John, 2022. "Optimal management of electric hotel loads in mild hybrid heavy duty truck," Applied Energy, Elsevier, vol. 326(C).
    3. García, Antonio & Monsalve-Serrano, Javier & Martinez-Boggio, Santiago & Gaillard, Patrick, 2021. "Impact of the hybrid electric architecture on the performance and emissions of a delivery truck with a dual-fuel RCCI engine," Applied Energy, Elsevier, vol. 301(C).
    4. Hou, Shengyan & Yin, Hai & Xu, Fuguo & Benjamín, Pla & Gao, Jinwu & Chen, Hong, 2023. "Multihorizon predictive energy optimization and lifetime management for connected fuel cell electric vehicles," Energy, Elsevier, vol. 266(C).
    5. Dapai Shi & Junjie Guo & Kangjie Liu & Qingling Cai & Zhenghong Wang & Xudong Qu, 2023. "Research on an Improved Rule-Based Energy Management Strategy Enlightened by the DP Optimization Results," Sustainability, MDPI, vol. 15(13), pages 1-13, July.
    6. Li, Guozhen & Zhang, Zhenyu & Shi, Wankai & Li, Wenyong, 2023. "Energy management strategy and simulation analysis of a hybrid train based on a comprehensive efficiency optimization," Applied Energy, Elsevier, vol. 349(C).
    7. Li, Qi & Wang, Tianhong & Li, Shihan & Chen, Weirong & Liu, Hong & Breaz, Elena & Gao, Fei, 2021. "Online extremum seeking-based optimized energy management strategy for hybrid electric tram considering fuel cell degradation," Applied Energy, Elsevier, vol. 285(C).
    8. Guo, Lingxiong & Zhang, Xudong & Zou, Yuan & Han, Lijin & Du, Guodong & Guo, Ningyuan & Xiang, Changle, 2022. "Co-optimization strategy of unmanned hybrid electric tracked vehicle combining eco-driving and simultaneous energy management," Energy, Elsevier, vol. 246(C).
    9. Zhang, Yahui & Wei, Zeyi & Wang, Zhong & Tian, Yang & Wang, Jizhe & Tian, Zhikun & Xu, Fuguo & Jiao, Xiaohong & Li, Liang & Wen, Guilin, 2024. "Hierarchical eco-driving control strategy for connected automated fuel cell hybrid vehicles and scenario-/hardware-in-the loop validation," Energy, Elsevier, vol. 292(C).
    10. Alberto Broatch & Pablo Olmeda & Benjamín Plá & Amin Dreif, 2022. "Novel Energy Management Control Strategy for Improving Efficiency in Hybrid Powertrains," Energies, MDPI, vol. 16(1), pages 1-21, December.
    11. Anselma, Pier Giuseppe, 2022. "Electrified powertrain sizing for vehicle fleets of car makers considering total ownership costs and CO2 emission legislation scenarios," Applied Energy, Elsevier, vol. 314(C).
    12. Zhang, Hao & Liu, Shang & Lei, Nuo & Fan, Qinhao & Wang, Zhi, 2022. "Leveraging the benefits of ethanol-fueled advanced combustion and supervisory control optimization in hybrid biofuel-electric vehicles," Applied Energy, Elsevier, vol. 326(C).
    13. Chen, Zheng & Wu, Simin & Shen, Shiquan & Liu, Yonggang & Guo, Fengxiang & Zhang, Yuanjian, 2023. "Co-optimization of velocity planning and energy management for autonomous plug-in hybrid electric vehicles in urban driving scenarios," Energy, Elsevier, vol. 263(PF).
    14. Nie, Zhigen & Jia, Yuan & Wang, Wanqiong & Chen, Zheng & Outbib, Rachid, 2022. "Co-optimization of speed planning and energy management for intelligent fuel cell hybrid vehicle considering complex traffic conditions," Energy, Elsevier, vol. 247(C).
    15. Zhang, Hao & Fan, Qinhao & Liu, Shang & Li, Shengbo Eben & Huang, Jin & Wang, Zhi, 2021. "Hierarchical energy management strategy for plug-in hybrid electric powertrain integrated with dual-mode combustion engine," Applied Energy, Elsevier, vol. 304(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:14:y:2021:i:17:p:5270-:d:621674. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.