IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v16y2024i3p965-d1324636.html
   My bibliography  Save this article

Variable Speed Limit Intelligent Decision-Making Control Strategy Based on Deep Reinforcement Learning under Emergencies

Author

Listed:
  • Jingwen Yang

    (School of Electronic and Control Engineering, Chang’an University, Xi’an 710054, China)

  • Ping Wang

    (School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen 510006, China)

  • Yongfeng Ju

    (School of Electronic and Control Engineering, Chang’an University, Xi’an 710054, China)

Abstract

Uncertain emergency events are inevitable and occur unpredictably on the highway. Emergencies with lane capacity drops cause local congestion and can even cause a second accident if the response is not timely. To address this problem, a self-triggered variable speed limit (VSL) intelligent decision-making control strategy based on the improved deep deterministic policy gradient (DDPG) algorithm is proposed, which can eliminate or alleviate congestion in a timely manner. The action noise parameter is introduced to improve exploration efficiency and stability in the early stage of the algorithm training and then maximizes differential traffic flow as the control objective, taking the real-time traffic state as the input. The reward function is constructed to explore the values of the speed limit. The results show that in terms of safety, under different traffic flow levels, the proposed strategy has improved by over 28.30% compared to other methods. In terms of efficiency, except for being inferior to the no-control condition during low-traffic-flow conditions, our strategy has improved over 7.21% compared to the others. The proposed strategy greatly benefits traffic sustainability in Intelligent Transport Systems (ITSs).

Suggested Citation

  • Jingwen Yang & Ping Wang & Yongfeng Ju, 2024. "Variable Speed Limit Intelligent Decision-Making Control Strategy Based on Deep Reinforcement Learning under Emergencies," Sustainability, MDPI, vol. 16(3), pages 1-19, January.
    • Handle: RePEc:gam:jsusta:v:16:y:2024:i:3:p:965-:d:1324636
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/16/3/965/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/16/3/965/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    2. Wei, Sen & Li, Yanping & Yang, Hanqing & Xie, Minghui & Wang, Yuanqing, 2023. "A comprehensive operation and maintenance assessment for intelligent highways: A case study in Hong Kong-Zhuhai-Macao bridge," Transport Policy, Elsevier, vol. 142(C), pages 84-98.
    3. Daganzo, Carlos F., 1994. "The cell transmission model: A dynamic representation of highway traffic consistent with the hydrodynamic theory," Transportation Research Part B: Methodological, Elsevier, vol. 28(4), pages 269-287, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kim, Sunwoo & Choi, Yechan & Park, Joungho & Adams, Derrick & Heo, Seongmin & Lee, Jay H., 2024. "Multi-period, multi-timescale stochastic optimization model for simultaneous capacity investment and energy management decisions for hybrid Micro-Grids with green hydrogen production under uncertainty," Renewable and Sustainable Energy Reviews, Elsevier, vol. 190(PA).
    2. Jin, Ruiyang & Zhou, Yuke & Lu, Chao & Song, Jie, 2022. "Deep reinforcement learning-based strategy for charging station participating in demand response," Applied Energy, Elsevier, vol. 328(C).
    3. Minh Sang Pham Do & Ketoma Vix Kemanji & Man Dinh Vinh Nguyen & Tuan Anh Vu & Gerrit Meixner, 2023. "The Action Point Angle of Sight: A Traffic Generation Method for Driving Simulation, as a Small Step to Safe, Sustainable and Smart Cities," Sustainability, MDPI, vol. 15(12), pages 1-27, June.
    4. Qixiu Cheng & Zhiyuan Liu & Feifei Liu & Ruo Jia, 2017. "Urban dynamic congestion pricing: an overview and emerging research needs," International Journal of Urban Sciences, Taylor & Francis Journals, vol. 21(0), pages 3-18, August.
    5. Guo, Yurun & Wang, Shugang & Wang, Jihong & Zhang, Tengfei & Ma, Zhenjun & Jiang, Shuang, 2024. "Key district heating technologies for building energy flexibility: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PB).
    6. Gentile, Guido & Meschini, Lorenzo & Papola, Natale, 2007. "Spillback congestion in dynamic traffic assignment: A macroscopic flow model with time-varying bottlenecks," Transportation Research Part B: Methodological, Elsevier, vol. 41(10), pages 1114-1138, December.
    7. Shafqat Jawad & Junyong Liu, 2020. "Electrical Vehicle Charging Services Planning and Operation with Interdependent Power Networks and Transportation Networks: A Review of the Current Scenario and Future Trends," Energies, MDPI, vol. 13(13), pages 1-24, July.
    8. Seo, Toru & Kawasaki, Yutaka & Kusakabe, Takahiko & Asakura, Yasuo, 2019. "Fundamental diagram estimation by using trajectories of probe vehicles," Transportation Research Part B: Methodological, Elsevier, vol. 122(C), pages 40-56.
    9. McCrea, Jennifer & Moutari, Salissou, 2010. "A hybrid macroscopic-based model for traffic flow in road networks," European Journal of Operational Research, Elsevier, vol. 207(2), pages 676-684, December.
    10. Chou, Chang-Chi & Chiang, Wen-Chu & Chen, Albert Y., 2022. "Emergency medical response in mass casualty incidents considering the traffic congestions in proximity on-site and hospital delays," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 158(C).
    11. Huanping Li & Jian Wang & Guopeng Bai & Xiaowei Hu, 2021. "Exploring the Distribution of Traffic Flow for Shared Human and Autonomous Vehicle Roads," Energies, MDPI, vol. 14(12), pages 1-21, June.
    12. Gokhale, Gargya & Claessens, Bert & Develder, Chris, 2022. "Physics informed neural networks for control oriented thermal modeling of buildings," Applied Energy, Elsevier, vol. 314(C).
    13. Wang, Tao & Liao, Peng & Tang, Tie-Qiao & Huang, Hai-Jun, 2022. "Deterministic capacity drop and morning commute in traffic corridor with tandem bottlenecks: A new manifestation of capacity expansion paradox," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 168(C).
    14. Herrera, Juan C. & Bayen, Alexandre M., 2010. "Incorporation of Lagrangian measurements in freeway traffic state estimation," Transportation Research Part B: Methodological, Elsevier, vol. 44(4), pages 460-481, May.
    15. Sun, Hongchang & Niu, Yanlei & Li, Chengdong & Zhou, Changgeng & Zhai, Wenwen & Chen, Zhe & Wu, Hao & Niu, Lanqiang, 2022. "Energy consumption optimization of building air conditioning system via combining the parallel temporal convolutional neural network and adaptive opposition-learning chimp algorithm," Energy, Elsevier, vol. 259(C).
    16. Jinxiao Duan & Guanwen Zeng & Nimrod Serok & Daqing Li & Efrat Blumenfeld Lieberthal & Hai-Jun Huang & Shlomo Havlin, 2023. "Spatiotemporal dynamics of traffic bottlenecks yields an early signal of heavy congestions," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    17. Langer, Lissy & Volling, Thomas, 2022. "A reinforcement learning approach to home energy management for modulating heat pumps and photovoltaic systems," Applied Energy, Elsevier, vol. 327(C).
    18. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    19. David Watling & Giulio Cantarella, 2015. "Model Representation & Decision-Making in an Ever-Changing World: The Role of Stochastic Process Models of Transportation Systems," Networks and Spatial Economics, Springer, vol. 15(3), pages 843-882, September.
    20. Ma, Tao & Zhou, Zhou & Antoniou, Constantinos, 2018. "Dynamic factor model for network traffic state forecast," Transportation Research Part B: Methodological, Elsevier, vol. 118(C), pages 281-317.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:16:y:2024:i:3:p:965-:d:1324636. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.