IDEAS home Printed from https://ideas.repec.org/a/eee/transb/v186y2024ics0191261524001401.html
   My bibliography  Save this article

Beyond centralization: Non-cooperative perimeter control with extended mean-field reinforcement learning in urban road networks

Author

Listed:
  • Li, Xinghua
  • Zhang, Xinyuan
  • Qian, Xinwu
  • Zhao, Cong
  • Guo, Yuntao
  • Peeta, Srinivas

Abstract

Perimeter control is a traffic management approach aimed at regulating vehicular accumulation within urban regional networks by managing flows on all border-crossing roads. Methods based on the macroscopic fundamental diagram (MFD) fall short in providing specific metering for individual roads. Recent advancements in the cell transmission model (CTM) have attempted to address this limitation but are hindered by their reliance on centralized control, which requires the availability of full information and authority over traffic generation sites. Our study proposes an innovative decentralized, game-theoretical framework for perimeter control to address these practical challenges. It is structured around two key groups of agents: perimeter agents, tasked with managing border roads, and interior agents, focused on traffic within generation sites. The framework also incorporates mechanisms for interactions between these agents and the road network, aiming to optimize their individual utilities. Additionally, we have developed a multi-agent reinforcement learning (RL) algorithm, extending the mean-field theory concept, to address the complexity of simultaneous learning by multiple agents.

Suggested Citation

  • Li, Xinghua & Zhang, Xinyuan & Qian, Xinwu & Zhao, Cong & Guo, Yuntao & Peeta, Srinivas, 2024. "Beyond centralization: Non-cooperative perimeter control with extended mean-field reinforcement learning in urban road networks," Transportation Research Part B: Methodological, Elsevier, vol. 186(C).
    • Handle: RePEc:eee:transb:v:186:y:2024:i:c:s0191261524001401
    DOI: 10.1016/j.trb.2024.103016
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0191261524001401
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.trb.2024.103016?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Guo, Qiangqiang & Ban, Xuegang (Jeff), 2020. "Macroscopic fundamental diagram based perimeter control considering dynamic user equilibrium," Transportation Research Part B: Methodological, Elsevier, vol. 136(C), pages 87-109.
    2. Gayah, Vikash V. & Daganzo, Carlos F., 2011. "Clockwise hysteresis loops in the Macroscopic Fundamental Diagram: An effect of network instability," Transportation Research Part B: Methodological, Elsevier, vol. 45(4), pages 643-655, May.
    3. Ramezani, Mohsen & Haddad, Jack & Geroliminis, Nikolas, 2015. "Dynamics of heterogeneity in urban networks: aggregated traffic modeling and hierarchical control," Transportation Research Part B: Methodological, Elsevier, vol. 74(C), pages 1-19.
    4. Paul I. Richards, 1956. "Shock Waves on the Highway," Operations Research, INFORMS, vol. 4(1), pages 42-51, February.
    5. Daganzo, Carlos F., 1994. "The cell transmission model: A dynamic representation of highway traffic consistent with the hydrodynamic theory," Transportation Research Part B: Methodological, Elsevier, vol. 28(4), pages 269-287, August.
    6. Saeedmanesh, Mohammadreza & Geroliminis, Nikolas, 2016. "Clustering of heterogeneous networks with directional flows based on “Snake” similarities," Transportation Research Part B: Methodological, Elsevier, vol. 91(C), pages 250-269.
    7. Hong K. Lo, 2001. "A Cell-Based Traffic Control Formulation: Strategies and Benefits of Dynamic Timing Plans," Transportation Science, INFORMS, vol. 35(2), pages 148-164, May.
    8. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    9. Daganzo, Carlos F., 2007. "Urban gridlock: Macroscopic modeling and mitigation approaches," Transportation Research Part B: Methodological, Elsevier, vol. 41(1), pages 49-62, January.
    10. Haddad, Jack, 2017. "Optimal perimeter control synthesis for two urban regions with aggregate boundary queue dynamics," Transportation Research Part B: Methodological, Elsevier, vol. 96(C), pages 1-25.
    11. Geroliminis, Nikolas & Sun, Jie, 2011. "Properties of a well-defined macroscopic fundamental diagram for urban traffic," Transportation Research Part B: Methodological, Elsevier, vol. 45(3), pages 605-617, March.
    12. Geroliminis, Nikolas & Daganzo, Carlos F., 2008. "Existence of urban-scale macroscopic fundamental diagrams: Some experimental findings," Transportation Research Part B: Methodological, Elsevier, vol. 42(9), pages 759-770, November.
    13. Srivastava, Anupam & Jin, Wen-Long & Lebacque, Jean-Patrick, 2015. "A modified Cell Transmission Model with realistic queue discharge features at signalized intersections," Transportation Research Part B: Methodological, Elsevier, vol. 81(P1), pages 302-315.
    14. Ukkusuri, Satish V. & Han, Lanshan & Doan, Kien, 2012. "Dynamic user equilibrium with a path based cell transmission model for general traffic networks," Transportation Research Part B: Methodological, Elsevier, vol. 46(10), pages 1657-1684.
    15. Daganzo, Carlos F., 1995. "The cell transmission model, part II: Network traffic," Transportation Research Part B: Methodological, Elsevier, vol. 29(2), pages 79-93, April.
    16. Ji, Yuxuan & Geroliminis, Nikolas, 2012. "On the spatial partitioning of urban transportation networks," Transportation Research Part B: Methodological, Elsevier, vol. 46(10), pages 1639-1656.
    17. Gao, Shengling & Li, Daqing & Zheng, Nan & Hu, Ruiqi & She, Zhikun, 2022. "Resilient perimeter control for hyper-congested two-region networks with MFD dynamics," Transportation Research Part B: Methodological, Elsevier, vol. 156(C), pages 50-75.
    18. Haddad, Jack & Shraiber, Arie, 2014. "Robust perimeter control design for an urban region," Transportation Research Part B: Methodological, Elsevier, vol. 68(C), pages 315-332.
    19. Kouvelas, Anastasios & Saeedmanesh, Mohammadreza & Geroliminis, Nikolas, 2017. "Enhancing model-based feedback perimeter control with data-driven online adaptive optimization," Transportation Research Part B: Methodological, Elsevier, vol. 96(C), pages 26-45.
    20. Mohebifard, Rasool & Hajbabaie, Ali, 2019. "Optimal network-level traffic signal control: A benders decomposition-based solution algorithm," Transportation Research Part B: Methodological, Elsevier, vol. 121(C), pages 252-274.
    21. Haddad, Jack & Geroliminis, Nikolas, 2012. "On the stability of traffic perimeter control in two-region urban cities," Transportation Research Part B: Methodological, Elsevier, vol. 46(9), pages 1159-1176.
    22. Keyvan-Ekbatani, Mehdi & Kouvelas, Anastasios & Papamichail, Ioannis & Papageorgiou, Markos, 2012. "Exploiting the fundamental diagram of urban networks for feedback-based gating," Transportation Research Part B: Methodological, Elsevier, vol. 46(10), pages 1393-1403.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ding, Heng & Di, Yunran & Feng, Zhongxiang & Zhang, Weihua & Zheng, Xiaoyan & Yang, Tao, 2022. "A perimeter control method for a congested urban road network with dynamic and variable ranges," Transportation Research Part B: Methodological, Elsevier, vol. 155(C), pages 160-187.
    2. Ampountolas, Konstantinos & Zheng, Nan & Geroliminis, Nikolas, 2017. "Macroscopic modelling and robust control of bi-modal multi-region urban road networks," Transportation Research Part B: Methodological, Elsevier, vol. 104(C), pages 616-637.
    3. Haddad, Jack & Zheng, Zhengfei, 2020. "Adaptive perimeter control for multi-region accumulation-based models with state delays," Transportation Research Part B: Methodological, Elsevier, vol. 137(C), pages 133-153.
    4. Kouvelas, Anastasios & Saeedmanesh, Mohammadreza & Geroliminis, Nikolas, 2017. "Enhancing model-based feedback perimeter control with data-driven online adaptive optimization," Transportation Research Part B: Methodological, Elsevier, vol. 96(C), pages 26-45.
    5. Gao, Shengling & Li, Daqing & Zheng, Nan & Hu, Ruiqi & She, Zhikun, 2022. "Resilient perimeter control for hyper-congested two-region networks with MFD dynamics," Transportation Research Part B: Methodological, Elsevier, vol. 156(C), pages 50-75.
    6. Yang, Lei & Yin, Suwan & Han, Ke & Haddad, Jack & Hu, Minghua, 2017. "Fundamental diagrams of airport surface traffic: Models and applications," Transportation Research Part B: Methodological, Elsevier, vol. 106(C), pages 29-51.
    7. Su, Z.C. & Chow, Andy H.F. & Fang, C.L. & Liang, E.M. & Zhong, R.X., 2023. "Hierarchical control for stochastic network traffic with reinforcement learning," Transportation Research Part B: Methodological, Elsevier, vol. 167(C), pages 196-216.
    8. Amirgholy, Mahyar & Shahabi, Mehrdad & Gao, H. Oliver, 2017. "Optimal design of sustainable transit systems in congested urban networks: A macroscopic approach," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 103(C), pages 261-285.
    9. Ramezani, Mohsen & Haddad, Jack & Geroliminis, Nikolas, 2015. "Dynamics of heterogeneity in urban networks: aggregated traffic modeling and hierarchical control," Transportation Research Part B: Methodological, Elsevier, vol. 74(C), pages 1-19.
    10. Zhong, R.X. & Chen, C. & Huang, Y.P. & Sumalee, A. & Lam, W.H.K. & Xu, D.B., 2018. "Robust perimeter control for two urban regions with macroscopic fundamental diagrams: A control-Lyapunov function approach," Transportation Research Part B: Methodological, Elsevier, vol. 117(PB), pages 687-707.
    11. Dantsuji, Takao & Takayama, Yuki & Fukuda, Daisuke, 2023. "Perimeter control in a mixed bimodal bathtub model," Transportation Research Part B: Methodological, Elsevier, vol. 173(C), pages 267-291.
    12. Ambühl, Lukas & Loder, Allister & Bliemer, Michiel C.J. & Menendez, Monica & Axhausen, Kay W., 2020. "A functional form with a physical meaning for the macroscopic fundamental diagram," Transportation Research Part B: Methodological, Elsevier, vol. 137(C), pages 119-132.
    13. Saeedmanesh, Mohammadreza & Geroliminis, Nikolas, 2017. "Dynamic clustering and propagation of congestion in heterogeneously congested urban traffic networks," Transportation Research Part B: Methodological, Elsevier, vol. 105(C), pages 193-211.
    14. Yildirimoglu, Mehmet & Sirmatel, Isik Ilber & Geroliminis, Nikolas, 2018. "Hierarchical control of heterogeneous large-scale urban road networks via path assignment and regional route guidance," Transportation Research Part B: Methodological, Elsevier, vol. 118(C), pages 106-123.
    15. Niu, Xiao-Jing & Zhao, Xiao-Mei & Xie, Dong-Fan & Liu, Feng & Bi, Jun & Lu, Chaoru, 2022. "Impact of large-scale activities on macroscopic fundamental diagram: Field data analysis and modeling," Transportation Research Part A: Policy and Practice, Elsevier, vol. 161(C), pages 241-268.
    16. Guo, Qiangqiang & Ban, Xuegang (Jeff), 2020. "Macroscopic fundamental diagram based perimeter control considering dynamic user equilibrium," Transportation Research Part B: Methodological, Elsevier, vol. 136(C), pages 87-109.
    17. Ding, Heng & Qian, Yu & Zheng, Xiaoyan & Bai, Haijian & Wang, Shiguang & Zhou, Jingwen, 2022. "Dynamic parking charge–perimeter control coupled method for a congested road network based on the aggregation degree characteristics of parking generation distribution," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 587(C).
    18. Amirgholy, Mahyar & Gao, H. Oliver, 2017. "Modeling the dynamics of congestion in large urban networks using the macroscopic fundamental diagram: User equilibrium, system optimum, and pricing strategies," Transportation Research Part B: Methodological, Elsevier, vol. 104(C), pages 215-237.
    19. Ge, Qian & Fukuda, Daisuke, 2019. "A macroscopic dynamic network loading model for multiple-reservoir system," Transportation Research Part B: Methodological, Elsevier, vol. 126(C), pages 502-527.
    20. Haddad, Jack, 2017. "Optimal perimeter control synthesis for two urban regions with aggregate boundary queue dynamics," Transportation Research Part B: Methodological, Elsevier, vol. 96(C), pages 1-25.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:transb:v:186:y:2024:i:c:s0191261524001401. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/548/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.