IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v245y2022ics0360544222000858.html
   My bibliography  Save this article

Energy management strategy for power-split plug-in hybrid electric vehicle based on MPC and double Q-learning

Author

Listed:
  • Chen, Zheng
  • Gu, Hongji
  • Shen, Shiquan
  • Shen, Jiangwei

Abstract

Energy management strategy (EMS) for plug-in hybrid electric vehicle (PHEV) is the key to improve the energy utilization efficiency and vehicle fuel economy. In this paper, a model predictive control (MPC) based on EMS coupled with double Q-learning (DQL) is presented to allocate the power between multiple power sources for PHEV. Firstly, the powertrain framework of the PHEV and its mathematical models were analyzed in detail. Then, based on the required power and speed, an effective convergent offline learning controller was established based on DQL algorithm. Subsequently, the multi-feature input Elman neural network was implemented to predict vehicle speed in MPC, and the trained DQL controller was applied to solve the rolling optimization process in MPC to find the optimal battery output in the prediction horizon. Finally, the proposed strategy was verified in Autonomie software, and the simulation results show that the proposed strategy can achieve a superior fuel economy close to that of the offline stochastic dynamic planning strategy, meanwhile with a perfect adaptability for different state of charge (SOC) reference trajectories.

Suggested Citation

  • Chen, Zheng & Gu, Hongji & Shen, Shiquan & Shen, Jiangwei, 2022. "Energy management strategy for power-split plug-in hybrid electric vehicle based on MPC and double Q-learning," Energy, Elsevier, vol. 245(C).
    • Handle: RePEc:eee:energy:v:245:y:2022:i:c:s0360544222000858
    DOI: 10.1016/j.energy.2022.123182
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544222000858
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2022.123182?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wu, Yitao & Zhang, Yuanjian & Li, Guang & Shen, Jiangwei & Chen, Zheng & Liu, Yonggang, 2020. "A predictive energy management strategy for multi-mode plug-in hybrid electric vehicles based on multi neural networks," Energy, Elsevier, vol. 208(C).
    2. Xie, Shaobo & Hu, Xiaosong & Xin, Zongke & Brighton, James, 2019. "Pontryagin’s Minimum Principle based model predictive control of energy management for a plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 236(C), pages 893-905.
    3. Tian, He & Li, Shengbo Eben & Wang, Xu & Huang, Yong & Tian, Guangyu, 2018. "Data-driven hierarchical control for online energy management of plug-in hybrid electric city bus," Energy, Elsevier, vol. 142(C), pages 55-67.
    4. Guo, Ningyuan & Zhang, Xudong & Zou, Yuan & Guo, Lingxiong & Du, Guodong, 2021. "Real-time predictive energy management of plug-in hybrid electric vehicles for coordination of fuel economy and battery degradation," Energy, Elsevier, vol. 214(C).
    5. Yang, Chao & Du, Siyu & Li, Liang & You, Sixong & Yang, Yiyong & Zhao, Yue, 2017. "Adaptive real-time optimal energy management strategy based on equivalent factors optimization for plug-in hybrid electric vehicle," Applied Energy, Elsevier, vol. 203(C), pages 883-896.
    6. Peng, Jiankun & He, Hongwen & Xiong, Rui, 2017. "Rule based energy management strategy for a series–parallel plug-in hybrid electric bus optimized by dynamic programming," Applied Energy, Elsevier, vol. 185(P2), pages 1633-1643.
    7. Wang, Feng & Zhang, Jian & Xu, Xing & Cai, Yingfeng & Zhou, Zhiguang & Sun, Xiaoqiang, 2019. "A comprehensive dynamic efficiency-enhanced energy management strategy for plug-in hybrid electric vehicles," Applied Energy, Elsevier, vol. 247(C), pages 657-669.
    8. Sun, Chao & Sun, Fengchun & He, Hongwen, 2017. "Investigating adaptive-ECMS with velocity forecast ability for hybrid electric vehicles," Applied Energy, Elsevier, vol. 185(P2), pages 1644-1653.
    9. Chen, Zheng & Hu, Hengjie & Wu, Yitao & Zhang, Yuanjian & Li, Guang & Liu, Yonggang, 2020. "Stochastic model predictive control for energy management of power-split plug-in hybrid electric vehicles based on reinforcement learning," Energy, Elsevier, vol. 211(C).
    10. Wu, Jingda & He, Hongwen & Peng, Jiankun & Li, Yuecheng & Li, Zhanjiang, 2018. "Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus," Applied Energy, Elsevier, vol. 222(C), pages 799-811.
    11. Xiong, Rui & Cao, Jiayi & Yu, Quanqing, 2018. "Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle," Applied Energy, Elsevier, vol. 211(C), pages 538-548.
    12. Xie, Shanshan & He, Hongwen & Peng, Jiankun, 2017. "An energy management strategy based on stochastic model predictive control for plug-in hybrid electric buses," Applied Energy, Elsevier, vol. 196(C), pages 279-288.
    13. He, Hongwen & Wang, Yunlong & Han, Ruoyan & Han, Mo & Bai, Yunfei & Liu, Qingwu, 2021. "An improved MPC-based energy management strategy for hybrid vehicles using V2V and V2I communications," Energy, Elsevier, vol. 225(C).
    14. Shangguan, Jinyong & Guo, Hongqiang & Yue, Ming, 2020. "Robust energy management of plug-in hybrid electric bus considering the uncertainties of driving cycles and vehicle mass," Energy, Elsevier, vol. 203(C).
    15. Zhang, Yuanjian & Chu, Liang & Fu, Zicheng & Xu, Nan & Guo, Chong & Zhao, Di & Ou, Yang & Xu, Lei, 2020. "Energy management strategy for plug-in hybrid electric vehicle integrated with vehicle-environment cooperation control," Energy, Elsevier, vol. 197(C).
    16. Sánchez, Marcelino & Delprat, Sébastien & Hofman, Theo, 2020. "Energy management of hybrid vehicles with state constraints: A penalty and implicit Hamiltonian minimization approach," Applied Energy, Elsevier, vol. 260(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Iqbal, Najam & Wang, Hu & Zheng, Zunqing & Yao, Mingfa, 2024. "Reinforcement learning-based heuristic planning for optimized energy management in power-split hybrid electric heavy duty vehicles," Energy, Elsevier, vol. 302(C).
    2. Sun, Xilei & Fu, Jianqin & Yang, Huiyong & Xie, Mingke & Liu, Jingping, 2023. "An energy management strategy for plug-in hybrid electric vehicles based on deep learning and improved model predictive control," Energy, Elsevier, vol. 269(C).
    3. Li, Mingyang & Tang, Jinjun, 2023. "Simulation-based optimization considering energy consumption for assisted station locations to enhance flex-route transit," Energy, Elsevier, vol. 277(C).
    4. Penghui Qiang & Peng Wu & Tao Pan & Huaiquan Zang, 2022. "Real-Time Energy Management Strategy Based on Driving Conditions Using a Feature Fusion Extreme Learning Machine," Energies, MDPI, vol. 15(12), pages 1-22, June.
    5. Badji, Abderrezak & Abdeslam, Djaffar Ould & Chabane, Djafar & Benamrouche, Nacereddine, 2022. "Real-time implementation of improved power frequency approach based energy management of fuel cell electric vehicle considering storage limitations," Energy, Elsevier, vol. 249(C).
    6. Wang, Xiaoyu & Jia, Hongjie & Jin, Xiaolong & Mu, Yunfei & Wei, Wei & Yu, Xiaodan & Liang, Shuo, 2024. "Bi-level optimal operations for grid operator and low-carbon building prosumers with peer-to-peer energy sharing," Applied Energy, Elsevier, vol. 359(C).
    7. Cui, Wei & Cui, Naxin & Li, Tao & Cui, Zhongrui & Du, Yi & Zhang, Chenghui, 2022. "An efficient multi-objective hierarchical energy management strategy for plug-in hybrid electric vehicle in connected scenario," Energy, Elsevier, vol. 257(C).
    8. Mudhafar Al-Saadi & Maher Al-Greer & Michael Short, 2023. "Reinforcement Learning-Based Intelligent Control Strategies for Optimal Power Management in Advanced Power Distribution Systems: A Survey," Energies, MDPI, vol. 16(4), pages 1-38, February.
    9. Ruan, Shumin & Ma, Yue & Yang, Ningkang & Yan, Qi & Xiang, Changle, 2023. "Multiobjective optimization of longitudinal dynamics and energy management for HEVs based on nash bargaining game," Energy, Elsevier, vol. 262(PA).
    10. Ren, Xiaoxia & Ye, Jinze & Xie, Liping & Lin, Xinyou, 2024. "Battery longevity-conscious energy management predictive control strategy optimized by using deep reinforcement learning algorithm for a fuel cell hybrid electric vehicle," Energy, Elsevier, vol. 286(C).
    11. Chen, Zheng & Wu, Simin & Shen, Shiquan & Liu, Yonggang & Guo, Fengxiang & Zhang, Yuanjian, 2023. "Co-optimization of velocity planning and energy management for autonomous plug-in hybrid electric vehicles in urban driving scenarios," Energy, Elsevier, vol. 263(PF).
    12. Abd-Elhaleem, Sameh & Shoeib, Walaa & Sobaih, Abdel Azim, 2023. "A new power management strategy for plug-in hybrid electric vehicles based on an intelligent controller integrated with CIGPSO algorithm," Energy, Elsevier, vol. 265(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhuang, Weichao & Li (Eben), Shengbo & Zhang, Xiaowu & Kum, Dongsuk & Song, Ziyou & Yin, Guodong & Ju, Fei, 2020. "A survey of powertrain configuration studies on hybrid electric vehicles," Applied Energy, Elsevier, vol. 262(C).
    2. Dong, Peng & Zhao, Junwei & Liu, Xuewu & Wu, Jian & Xu, Xiangyang & Liu, Yanfang & Wang, Shuhan & Guo, Wei, 2022. "Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies: Development stages, challenges, and future trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    3. Geng, Wenran & Lou, Diming & Wang, Chen & Zhang, Tong, 2020. "A cascaded energy management optimization method of multimode power-split hybrid electric vehicles," Energy, Elsevier, vol. 199(C).
    4. Chen, Z. & Liu, Y. & Ye, M. & Zhang, Y. & Chen, Z. & Li, G., 2021. "A survey on key techniques and development perspectives of equivalent consumption minimisation strategy for hybrid electric vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 151(C).
    5. Fengqi Zhang & Lihua Wang & Serdar Coskun & Hui Pang & Yahui Cui & Junqiang Xi, 2020. "Energy Management Strategies for Hybrid Electric Vehicles: Review, Classification, Comparison, and Outlook," Energies, MDPI, vol. 13(13), pages 1-35, June.
    6. Cui, Wei & Cui, Naxin & Li, Tao & Cui, Zhongrui & Du, Yi & Zhang, Chenghui, 2022. "An efficient multi-objective hierarchical energy management strategy for plug-in hybrid electric vehicle in connected scenario," Energy, Elsevier, vol. 257(C).
    7. Iqbal, Najam & Wang, Hu & Zheng, Zunqing & Yao, Mingfa, 2024. "Reinforcement learning-based heuristic planning for optimized energy management in power-split hybrid electric heavy duty vehicles," Energy, Elsevier, vol. 302(C).
    8. Lei, Zhenzhen & Qin, Datong & Hou, Liliang & Peng, Jingyu & Liu, Yonggang & Chen, Zheng, 2020. "An adaptive equivalent consumption minimization strategy for plug-in hybrid electric vehicles based on traffic information," Energy, Elsevier, vol. 190(C).
    9. Lin, Xinyou & Zeng, Songrong & Li, Xuefan, 2021. "Online correction predictive energy management strategy using the Q-learning based swarm optimization with fuzzy neural network," Energy, Elsevier, vol. 223(C).
    10. Chen, Zheng & Hu, Hengjie & Wu, Yitao & Zhang, Yuanjian & Li, Guang & Liu, Yonggang, 2020. "Stochastic model predictive control for energy management of power-split plug-in hybrid electric vehicles based on reinforcement learning," Energy, Elsevier, vol. 211(C).
    11. Wu, Yuankai & Tan, Huachun & Peng, Jiankun & Zhang, Hailong & He, Hongwen, 2019. "Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 247(C), pages 454-466.
    12. Hou, Shengyan & Yin, Hai & Xu, Fuguo & Benjamín, Pla & Gao, Jinwu & Chen, Hong, 2023. "Multihorizon predictive energy optimization and lifetime management for connected fuel cell electric vehicles," Energy, Elsevier, vol. 266(C).
    13. Kong, Yan & Xu, Nan & Zhang, Yuanjian & Sui, Yan & Ju, Hao & Liu, Heng & Xu, Zhe, 2021. "Acquisition of full-factor trip information for global optimization energy management in multi-energy source vehicles and the measure of the amount of information to be transmitted," Energy, Elsevier, vol. 236(C).
    14. Md. Sazal Miah & Molla Shahadat Hossain Lipu & Sheikh Tanzim Meraj & Kamrul Hasan & Shaheer Ansari & Taskin Jamal & Hasan Masrur & Rajvikram Madurai Elavarasan & Aini Hussain, 2021. "Optimized Energy Management Schemes for Electric Vehicle Applications: A Bibliometric Analysis towards Future Trends," Sustainability, MDPI, vol. 13(22), pages 1-38, November.
    15. Yaqian Wang & Xiaohong Jiao, 2022. "Dual Heuristic Dynamic Programming Based Energy Management Control for Hybrid Electric Vehicles," Energies, MDPI, vol. 15(9), pages 1-19, April.
    16. Zhang, Shuo & Hu, Xiaosong & Xie, Shaobo & Song, Ziyou & Hu, Lin & Hou, Cong, 2019. "Adaptively coordinated optimization of battery aging and energy management in plug-in hybrid electric buses," Applied Energy, Elsevier, vol. 256(C).
    17. Yang, Chao & Du, Xuelong & Wang, Weida & Yuan, Lijuan & Yang, Liuquan, 2024. "Variable optimization domain-based cooperative energy management strategy for connected plug-in hybrid electric vehicles," Energy, Elsevier, vol. 290(C).
    18. Penghui Qiang & Peng Wu & Tao Pan & Huaiquan Zang, 2022. "Real-Time Energy Management Strategy Based on Driving Conditions Using a Feature Fusion Extreme Learning Machine," Energies, MDPI, vol. 15(12), pages 1-22, June.
    19. Manzolli, Jônatas Augusto & Trovão, João Pedro & Antunes, Carlos Henggeler, 2022. "A review of electric bus vehicles research topics – Methods and trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 159(C).
    20. Xiaodong Liu & Hongqiang Guo & Xingqun Cheng & Juan Du & Jian Ma, 2022. "A Robust Design of the Model-Free-Adaptive-Control-Based Energy Management for Plug-In Hybrid Electric Vehicle," Energies, MDPI, vol. 15(20), pages 1-24, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:245:y:2022:i:c:s0360544222000858. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.