IDEAS home Printed from https://ideas.repec.org/a/eee/eejocm/v39y2021ics175553452100021x.html
   My bibliography  Save this article

A hierarchical agent-based approach to simulate a dynamic decision-making process of evacuees using reinforcement learning

Author

Listed:
  • Hassanpour, Sajjad
  • Rassafi, Amir Abbas
  • González, Vicente A.
  • Liu, Jiamou

Abstract

Simulation models are an undeniable tool to help researchers and designers forecast effects of definite policies regarding pedestrian social and collective movement behaviour. Considering both the environment's details and the complexity of human behaviour in choosing paths simultaneously is the main challenge in micro-simulation pedestrian dynamics models. This paper aims to present a novel comprehensive hierarchical agent-based simulation of pedestrian evacuation from a dynamic network of the environment using reinforcement learning, which is the closest to human behaviour among the other machine learning algorithms. In the approach, agents autonomously decide through a three-layer hierarchical model, including goal, node, and cell selection layers. A multinomial logit model is used to model the process of choosing the main movement direction at each time-step. The proposed model was successfully tested to simulate the pedestrian evacuation process from the Britomart Transport Centre platforms in Auckland during an abstract destructive event. Maximum evacuation flow, total evacuation time, average evacuation time, and average evacuation flow were investigated as dependent variables through different evacuation scenarios. The results from the approach can be used by designers and managers to optimise the quality of evacuation; also, the proposed model has the potential of becoming a potent tool for constructional management if coupled with other constructional tools.

Suggested Citation

  • Hassanpour, Sajjad & Rassafi, Amir Abbas & González, Vicente A. & Liu, Jiamou, 2021. "A hierarchical agent-based approach to simulate a dynamic decision-making process of evacuees using reinforcement learning," Journal of choice modelling, Elsevier, vol. 39(C).
    • Handle: RePEc:eee:eejocm:v:39:y:2021:i:c:s175553452100021x
    DOI: 10.1016/j.jocm.2021.100288
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S175553452100021X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jocm.2021.100288?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Haghani, Milad & Sarvi, Majid & Shahhoseini, Zahra, 2015. "Accommodating taste heterogeneity and desired substitution pattern in exit choices of pedestrian crowd evacuees using a mixed nested logit model," Journal of choice modelling, Elsevier, vol. 16(C), pages 58-68.
    2. Alessandro Pluchino & Cesare Garofalo & Giuseppe Inturri & Andrea Rapisarda & Matteo Ignaccolo, 2014. "Agent-Based Simulation of Pedestrian Behaviour in Closed Spaces: A Museum Case Study," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 17(1), pages 1-16.
    3. Izquierdo, J. & Montalvo, I. & Pérez, R. & Fuertes, V.S., 2009. "Forecasting pedestrian evacuation times by using swarm intelligence," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 388(7), pages 1213-1220.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wen, Chieh-Hua & Huang, Chia-Jung & Fu, Chiang, 2020. "Incorporating continuous representation of preferences for flight departure times into stated itinerary choice modeling," Transport Policy, Elsevier, vol. 98(C), pages 10-20.
    2. Zheng, Yaochen & Chen, Jianqiao & Wei, Junhong & Guo, Xiwei, 2012. "Modeling of pedestrian evacuation based on the particle swarm optimization algorithm," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(17), pages 4225-4233.
    3. Vásquez Lavin, Felipe & Barrientos, Manuel & Castillo, Álvaro & Herrera, Iván & Ponce Oliva, Roberto D., 2020. "Firewood certification programs: Key attributes and policy implications," Energy Policy, Elsevier, vol. 137(C).
    4. Li, Lin & Yu, Zhonghai & Chen, Yang, 2014. "Evacuation dynamic and exit optimization of a supermarket based on particle swarm optimization," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 416(C), pages 157-172.
    5. Haghani, Milad & Bliemer, Michiel C.J. & Rose, John M. & Oppewal, Harmen & Lancsar, Emily, 2021. "Hypothetical bias in stated choice experiments: Part I. Macro-scale analysis of literature and integrative synthesis of empirical evidence from applied economics, experimental psychology and neuroimag," Journal of choice modelling, Elsevier, vol. 41(C).
    6. Qi, Jie & Rong, Zhihai, 2013. "The emergence of scaling laws search dynamics in a particle swarm optimization," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(6), pages 1522-1531.
    7. Milad Haghani & Majid Sarvi & Zahra Shahhoseini & Maik Boltes, 2016. "How Simple Hypothetical-Choice Experiments Can Be Utilized to Learn Humans’ Navigational Escape Decisions in Emergencies," PLOS ONE, Public Library of Science, vol. 11(11), pages 1-24, November.
    8. Abdelghany, Ahmed & Abdelghany, Khaled & Mahmassani, Hani, 2016. "A hybrid simulation-assignment modeling framework for crowd dynamics in large-scale pedestrian facilities," Transportation Research Part A: Policy and Practice, Elsevier, vol. 86(C), pages 159-176.
    9. Zheng, Xiaoping & Li, Wei & Guan, Chao, 2010. "Simulation of evacuation processes in a square with a partition wall using a cellular automaton model for pedestrian dynamics," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 389(11), pages 2177-2188.
    10. Haghani, Milad & Sarvi, Majid, 2018. "Crowd behaviour and motion: Empirical methods," Transportation Research Part B: Methodological, Elsevier, vol. 107(C), pages 253-294.
    11. Rodriguez, E. & Espinosa-Paredes, G. & Alvarez-Ramirez, J., 2014. "Convection–diffusion effects in marathon race dynamics," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 393(C), pages 498-507.
    12. Haghani, Milad & Bliemer, Michiel C.J. & Rose, John M. & Oppewal, Harmen & Lancsar, Emily, 2021. "Hypothetical bias in stated choice experiments: Part II. Conceptualisation of external validity, sources and explanations of bias and effectiveness of mitigation methods," Journal of choice modelling, Elsevier, vol. 41(C).
    13. Haghani, Milad & Bliemer, Michiel C.J. & Hensher, David A., 2021. "The landscape of econometric discrete choice modelling research," Journal of choice modelling, Elsevier, vol. 40(C).
    14. Haghani, Milad & Sarvi, Majid, 2018. "Hypothetical bias and decision-rule effect in modelling discrete directional choices," Transportation Research Part A: Policy and Practice, Elsevier, vol. 116(C), pages 361-388.
    15. G. B. Korovin, 2020. "Architecture of the agent-based model for the region’s industrial complex digital transformation," Journal of New Economy, Ural State University of Economics, vol. 21(3), pages 158-174, October.
    16. Milad Haghani & Michiel C. J. Bliemer & John M. Rose & Harmen Oppewal & Emily Lancsar, 2021. "Hypothetical bias in stated choice experiments: Part I. Integrative synthesis of empirical evidence and conceptualisation of external validity," Papers 2102.02940, arXiv.org.
    17. Moonsoo Ko & Taewan Kim & Keemin Sohn, 2013. "Calibrating a social-force-based pedestrian walking model based on maximum likelihood estimation," Transportation, Springer, vol. 40(1), pages 91-107, January.
    18. Situngkir, Hokky, 2016. "Agent-Based Model for River-Side Land-living: Portrait of Bandung Indonesian Cikapundung Park Case Study," MPRA Paper 71078, University Library of Munich, Germany.
    19. Abdelghany, Ahmed & Abdelghany, Khaled & Mahmassani, Hani & Alhalabi, Wael, 2014. "Modeling framework for optimal evacuation of large-scale crowded pedestrian facilities," European Journal of Operational Research, Elsevier, vol. 237(3), pages 1105-1118.
    20. Haghani, Milad & Sarvi, Majid, 2019. "Laboratory experimentation and simulation of discrete direction choices: Investigating hypothetical bias, decision-rule effect and external validity based on aggregate prediction measures," Transportation Research Part A: Policy and Practice, Elsevier, vol. 130(C), pages 134-157.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:eejocm:v:39:y:2021:i:c:s175553452100021x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/journal-of-choice-modelling .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.