IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v15y2023i18p13325-d1233575.html
   My bibliography  Save this article

Research on Ecological Driving Following Strategy Based on Deep Reinforcement Learning

Author

Listed:
  • Weiqi Zhou

    (Automotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, China
    Research Institute of Engineering Technology, Jiangsu University, Zhenjiang 212013, China)

  • Nanchi Wu

    (Automotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, China)

  • Qingchao Liu

    (Automotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, China
    Research Institute of Engineering Technology, Jiangsu University, Zhenjiang 212013, China)

  • Chaofeng Pan

    (Automotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, China)

  • Long Chen

    (Automotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, China)

Abstract

Traditional car-following models usually prioritize minimizing inter-vehicle distance error when tracking the preceding vehicle, often neglecting crucial factors like driving economy and passenger ride comfort. To address this limitation, this paper integrates the concept of eco-driving and formulates a multi-objective function that encompasses economy, comfort, and safety. A novel eco-driving car-following strategy based on the deep deterministic policy gradient (DDPG) is proposed, employing the vehicle’s state, including data from the preceding vehicle and the ego vehicle, as the state space, and the desired time headway from the intelligent driver model (IDM) as the action space. The DDPG agent is trained to dynamically adjust the following vehicle’s speed in real-time, striking a balance between driving economy, comfort, and safety. The results reveal that the proposed DDPG-based IDM model significantly enhances comfort, safety, and economy when compared to the fixed-time headway IDM model, achieving an economy improvement of 2.66% along with enhanced comfort. Moreover, the proposed approach maintains a relatively stable following distance under medium-speed conditions, ensuring driving safety. Additionally, the comprehensive performance of the proposed method is analyzed under three typical scenarios, confirming its generalization capability. The DDPG-enhanced IDM car-following model aligns with eco-driving principles, offering novel insights for advancing IDM-based car-following models.

Suggested Citation

  • Weiqi Zhou & Nanchi Wu & Qingchao Liu & Chaofeng Pan & Long Chen, 2023. "Research on Ecological Driving Following Strategy Based on Deep Reinforcement Learning," Sustainability, MDPI, vol. 15(18), pages 1-14, September.
  • Handle: RePEc:gam:jsusta:v:15:y:2023:i:18:p:13325-:d:1233575
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/15/18/13325/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/15/18/13325/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Junyan Han & Xiaoyuan Wang & Gang Wang, 2022. "Modeling the Car-Following Behavior with Consideration of Driver, Vehicle, and Environment Factors: A Historical Review," Sustainability, MDPI, vol. 14(13), pages 1-27, July.
    2. Qu, Xiaobo & Yu, Yang & Zhou, Mofan & Lin, Chin-Teng & Wang, Xiangyu, 2020. "Jointly dampening traffic oscillations and improving energy consumption with electric, connected and automated vehicles: A reinforcement learning based approach," Applied Energy, Elsevier, vol. 257(C).
    3. Li, Zhenhe & Khajepour, Amir & Song, Jinchun, 2019. "A comprehensive review of the key technologies for pure electric vehicles," Energy, Elsevier, vol. 182(C), pages 824-839.
    4. Ran Tu & Junshi Xu & Tiezhu Li & Haibo Chen, 2022. "Effective and Acceptable Eco-Driving Guidance for Human-Driving Vehicles: A Review," IJERPH, MDPI, vol. 19(12), pages 1-14, June.
    5. Yu, Shaowei & Shi, Zhongke, 2015. "An improved car-following model considering headway changes with memory," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 421(C), pages 1-14.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xie, Yunkun & Li, Yangyang & Zhao, Zhichao & Dong, Hao & Wang, Shuqian & Liu, Jingping & Guan, Jinhuan & Duan, Xiongbo, 2020. "Microsimulation of electric vehicle energy consumption and driving range," Applied Energy, Elsevier, vol. 267(C).
    2. Yin, Yu-Hang & Lü, Xing & Jiang, Rui & Jia, Bin & Gao, Ziyou, 2024. "Kinetic analysis and numerical tests of an adaptive car-following model for real-time traffic in ITS," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 635(C).
    3. Sofiane Bacha & Ramzi Saadi & Mohamed Yacine Ayad & Mohamed Sahraoui & Khaled Laadjal & Antonio J. Marques Cardoso, 2023. "Autonomous Electric-Vehicle Control Using Speed Planning Algorithm and Back-Stepping Approach," Energies, MDPI, vol. 16(5), pages 1-26, March.
    4. Chen, Jie & Wang, Ruochen & Ding, Renkai & Luo, Ding, 2024. "Matching design and numerical optimization of automotive thermoelectric generator system applied to range-extended electric vehicle," Applied Energy, Elsevier, vol. 370(C).
    5. Feiyu Hou & Fei Yao & Zheng Li, 2022. "A Torque-Compensated Fault-Tolerant Control Method for Electric Vehicle Traction Motor with Short-Circuit Fault," Sustainability, MDPI, vol. 14(21), pages 1-17, October.
    6. Jie Hu & Wentong Cao & Feng Jiang & Lingling Hu & Qian Chen & Weiguang Zheng & Junming Zhou, 2023. "Study on Multi-Objective Optimization of Power System Parameters of Battery Electric Vehicles," Sustainability, MDPI, vol. 15(10), pages 1-23, May.
    7. Yossi Hadad & Baruch Keren & Dima Alberg, 2023. "An Expert System for Ranking and Matching Electric Vehicles to Customer Specifications and Requirements," Energies, MDPI, vol. 16(11), pages 1-18, May.
    8. Hicham El Hadraoui & Mourad Zegrari & Fatima-Ezzahra Hammouch & Nasr Guennouni & Oussama Laayati & Ahmed Chebak, 2022. "Design of a Customizable Test Bench of an Electric Vehicle Powertrain for Learning Purposes Using Model-Based System Engineering," Sustainability, MDPI, vol. 14(17), pages 1-22, September.
    9. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    10. Tao Wang & Dayi Qu & Hui Song & Shouchen Dai, 2023. "A Hierarchical Framework of Decision Making and Trajectory Tracking Control for Autonomous Vehicles," Sustainability, MDPI, vol. 15(8), pages 1-28, April.
    11. Wen, Shuang & Lin, Ni & Huang, Shengxu & Wang, Zhenpo & Zhang, Zhaosheng, 2023. "Lithium battery health state assessment based on vehicle-to-grid (V2G) real-world data and natural gradient boosting model," Energy, Elsevier, vol. 284(C).
    12. He, Jia & He, Zhengbing & Fan, Bo & Chen, Yanyan, 2020. "Optimal location of lane-changing warning point in a two-lane road considering different traffic flows," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 540(C).
    13. Chen, Mingyi & Yu, Yue & Ouyang, Dongxu & Weng, Jingwen & Zhao, Luyao & Wang, Jian & Chen, Yin, 2024. "Research progress of enhancing battery safety with phase change materials," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PA).
    14. Theo Lieven & Beatrice Hügler, 2021. "Did Electric Vehicle Sales Skyrocket Due to Increased Environmental Awareness While Total Vehicle Sales Declined during COVID-19?," Sustainability, MDPI, vol. 13(24), pages 1-19, December.
    15. Wu, Yingwen & Ji, Yangjian, 2023. "Identifying firm-specific technology opportunities from the perspective of competitors by using association rule mining," Journal of Informetrics, Elsevier, vol. 17(2).
    16. Diana Lemian & Florin Bode, 2022. "Battery-Supercapacitor Energy Storage Systems for Electrical Vehicles: A Review," Energies, MDPI, vol. 15(15), pages 1-13, August.
    17. Yang, Xiong & Peng, Zhenhan & Wang, Pinxi & Zhuge, Chengxiang, 2023. "Seasonal variance in electric vehicle charging demand and its impacts on infrastructure deployment: A big data approach," Energy, Elsevier, vol. 280(C).
    18. Hongxing Zhao & Ruichun He & Xiaoyan Jia, 2019. "Estimation and Analysis of Vehicle Exhaust Emissions at Signalized Intersections Using a Car-Following Model," Sustainability, MDPI, vol. 11(14), pages 1-25, July.
    19. Tingting Wang & Xin Liu & Dongchen Qin & Yuechen Duan, 2022. "Thermal Modeling and Prediction of The Lithium-ion Battery Based on Driving Behavior," Energies, MDPI, vol. 15(23), pages 1-22, November.
    20. Xiaomeng Wang & Ling Peng & Tianhe Chi & Mengzhu Li & Xiaojing Yao & Jing Shao, 2015. "A Hidden Markov Model for Urban-Scale Traffic Estimation Using Floating Car Data," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-20, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:15:y:2023:i:18:p:13325-:d:1233575. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.