IDEAS home Printed from https://ideas.repec.org/a/eee/jomega/v101y2021ics0305048319308229.html
   My bibliography  Save this article

Determining ambulance destinations when facing offload delays using a Markov decision process

Author

Listed:
  • Li, Mengyu
  • Carter, Alix
  • Goldstein, Judah
  • Hawco, Terence
  • Jensen, Jan
  • Vanberkel, Peter

Abstract

When emergency departments (EDs) are crowded and cannot accept incoming ambulance patients immediately, paramedics commonly continue to provide patient care until an ED bed becomes available. This delay in transferring a patient to the ED is referred to as ambulance offload delay (AOD). AOD is a pressing problem for Emergency Medical Services (EMS) as it prolongs the time before paramedics are available to respond to other calls. This can negatively affect ambulance availability and patient safety. The objective of this study is to develop an ambulance destination policy to mitigate AOD, allowing patients to see physicians sooner, and returning ambulances to service more quickly. We formulate a discrete time, infinite-horizon, discounted Markov Decision Process (MDP) model to determine when it is advantageous to send appropriate patients to out-of-region EDs, which have longer transport times but shorter offload times. Based on the MDP model, an optimal ambulance destination policy is constructed using the policy iteration algorithm. A computational study is applied using 12-months of data from an EMS provider which experiences AOD regularly. We find that the optimal policies can significantly reduce AOD, time to bed for patients, and out-of-service time for paramedics at the expense of increased ambulances travel distances. The model can be generalized and used as a decision support tool for EMS systems to mitigate the impact of AOD on their operations.

Suggested Citation

  • Li, Mengyu & Carter, Alix & Goldstein, Judah & Hawco, Terence & Jensen, Jan & Vanberkel, Peter, 2021. "Determining ambulance destinations when facing offload delays using a Markov decision process," Omega, Elsevier, vol. 101(C).
  • Handle: RePEc:eee:jomega:v:101:y:2021:i:c:s0305048319308229
    DOI: 10.1016/j.omega.2020.102251
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0305048319308229
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.omega.2020.102251?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. S. Creemers & M. Lambrecht & N. Vandaele, 2007. "Queueing Models in Healthcare," Review of Business and Economic Literature, KU Leuven, Faculty of Economics and Business (FEB), Review of Business and Economic Literature, vol. 0(3), pages 471-498.
    2. Almehdawe, Eman & Jewkes, Beth & He, Qi-Ming, 2016. "Analysis and optimization of an ambulance offload delay and allocation problem," Omega, Elsevier, vol. 65(C), pages 148-158.
    3. Yu Jin Lee & Sang Do Shin & Eui Jung Lee & Jin Seong Cho & Won Chul Cha, 2015. "Emergency Department Overcrowding and Ambulance Turnaround Time," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-9, June.
    4. Brotcorne, Luce & Laporte, Gilbert & Semet, Frederic, 2003. "Ambulance location and relocation models," European Journal of Operational Research, Elsevier, vol. 147(3), pages 451-463, June.
    5. Mateo Restrepo & Shane Henderson & Huseyin Topaloglu, 2009. "Erlang loss models for the static deployment of ambulances," Health Care Management Science, Springer, vol. 12(1), pages 67-79, March.
    6. Susan Budge & Armann Ingolfsson & Dawit Zerom, 2010. "Empirical Analysis of Ambulance Travel Times: The Case of Calgary Emergency Medical Services," Management Science, INFORMS, vol. 56(4), pages 716-723, April.
    7. Peter Kolesar & Warren Walker & Jack Hausner, 1975. "Determining the Relation between Fire Engine Travel Times and Travel Distances in New York City," Operations Research, INFORMS, vol. 23(4), pages 614-627, August.
    8. Damitha Bandara & Maria E. Mayorga & Laura A. McLay, 2012. "Optimal dispatching strategies for emergency vehicles to increase patient survivability," International Journal of Operational Research, Inderscience Enterprises Ltd, vol. 15(2), pages 195-214.
    9. Almehdawe, Eman & Jewkes, Beth & He, Qi-Ming, 2013. "A Markovian queueing model for ambulance offload delays," European Journal of Operational Research, Elsevier, vol. 226(3), pages 602-614.
    10. Zied Jemai & L. Aboueljinane & E. Sahin, 2013. "A review on simulation models applied to emergency medical service operations," Post-Print hal-01672393, HAL.
    11. Laura A. McLay & Maria E. Mayorga, 2013. "A Dispatching Model for Server-to-Customer Systems That Balances Efficiency and Equity," Manufacturing & Service Operations Management, INFORMS, vol. 15(2), pages 205-220, May.
    12. Matthew S. Maxwell & Mateo Restrepo & Shane G. Henderson & Huseyin Topaloglu, 2010. "Approximate Dynamic Programming for Ambulance Redeployment," INFORMS Journal on Computing, INFORMS, vol. 22(2), pages 266-281, May.
    13. Oded Berman, 1981. "Dynamic Repositioning of Indistinguishable Service Units on Transportation Networks," Transportation Science, INFORMS, vol. 15(2), pages 115-136, May.
    14. Laura McLay & Maria Mayorga, 2013. "A model for optimally dispatching ambulances to emergency calls with classification errors in patient priorities," IISE Transactions, Taylor & Francis Journals, vol. 45(1), pages 1-24.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Duma, Davide & Aringhieri, Roberto, 2023. "Real-time resource allocation in the emergency department: A case study," Omega, Elsevier, vol. 117(C).
    2. Hashemi-Petroodi, S. Ehsan & Thevenin, Simon & Kovalev, Sergey & Dolgui, Alexandre, 2023. "Markov decision process for multi-manned mixed-model assembly lines with walking workers," International Journal of Production Economics, Elsevier, vol. 255(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mengyu Li & Peter Vanberkel & Alix J. E. Carter, 2019. "A review on ambulance offload delay literature," Health Care Management Science, Springer, vol. 22(4), pages 658-675, December.
    2. Bélanger, V. & Ruiz, A. & Soriano, P., 2019. "Recent optimization models and trends in location, relocation, and dispatching of emergency medical vehicles," European Journal of Operational Research, Elsevier, vol. 272(1), pages 1-23.
    3. Amir Ali Nasrollahzadeh & Amin Khademi & Maria E. Mayorga, 2018. "Real-Time Ambulance Dispatching and Relocation," Manufacturing & Service Operations Management, INFORMS, vol. 20(3), pages 467-480, July.
    4. Yoon, Soovin & Albert, Laura A., 2021. "Dynamic dispatch policies for emergency response with multiple types of vehicles," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 152(C).
    5. Soovin Yoon & Laura A. Albert, 2018. "An expected coverage model with a cutoff priority queue," Health Care Management Science, Springer, vol. 21(4), pages 517-533, December.
    6. Westgate, Bradford S. & Woodard, Dawn B. & Matteson, David S. & Henderson, Shane G., 2016. "Large-network travel time distribution estimation for ambulances," European Journal of Operational Research, Elsevier, vol. 252(1), pages 322-333.
    7. Bélanger, V. & Lanzarone, E. & Nicoletta, V. & Ruiz, A. & Soriano, P., 2020. "A recursive simulation-optimization framework for the ambulance location and dispatching problem," European Journal of Operational Research, Elsevier, vol. 286(2), pages 713-725.
    8. Sudtachat, Kanchala & Mayorga, Maria E. & Mclay, Laura A., 2016. "A nested-compliance table policy for emergency medical service systems under relocation," Omega, Elsevier, vol. 58(C), pages 154-168.
    9. C. J. Jagtenberg & S. Bhulai & R. D. Mei, 2017. "Dynamic ambulance dispatching: is the closest-idle policy always optimal?," Health Care Management Science, Springer, vol. 20(4), pages 517-531, December.
    10. Li, Mengyu & Vanberkel, Peter & Zhong, Xiang, 2022. "Predicting ambulance offload delay using a hybrid decision tree model," Socio-Economic Planning Sciences, Elsevier, vol. 80(C).
    11. Bertsimas, Dimitris & Ng, Yeesian, 2019. "Robust and stochastic formulations for ambulance deployment and dispatch," European Journal of Operational Research, Elsevier, vol. 279(2), pages 557-571.
    12. Rettke, Aaron J. & Robbins, Matthew J. & Lunday, Brian J., 2016. "Approximate dynamic programming for the dispatch of military medical evacuation assets," European Journal of Operational Research, Elsevier, vol. 254(3), pages 824-839.
    13. Matthew S. Maxwell & Mateo Restrepo & Shane G. Henderson & Huseyin Topaloglu, 2010. "Approximate Dynamic Programming for Ambulance Redeployment," INFORMS Journal on Computing, INFORMS, vol. 22(2), pages 266-281, May.
    14. Amir Rastpour & Armann Ingolfsson & Bora Kolfal, 2020. "Modeling Yellow and Red Alert Durations for Ambulance Systems," Production and Operations Management, Production and Operations Management Society, vol. 29(8), pages 1972-1991, August.
    15. Phillip R. Jenkins & Matthew J. Robbins & Brian J. Lunday, 2018. "Examining military medical evacuation dispatching policies utilizing a Markov decision process model of a controlled queueing system," Annals of Operations Research, Springer, vol. 271(2), pages 641-678, December.
    16. McCormack, Richard & Coates, Graham, 2015. "A simulation model to enable the optimization of ambulance fleet allocation and base station location for increased patient survival," European Journal of Operational Research, Elsevier, vol. 247(1), pages 294-309.
    17. Ľuboš Buzna & Peter Czimmermann, 2021. "On the Modelling of Emergency Ambulance Trips: The Case of the Žilina Region in Slovakia," Mathematics, MDPI, vol. 9(17), pages 1-30, September.
    18. Kenneth C. Chong & Shane G. Henderson & Mark E. Lewis, 2016. "The Vehicle Mix Decision in Emergency Medical Service Systems," Manufacturing & Service Operations Management, INFORMS, vol. 18(3), pages 347-360, July.
    19. Schmid, Verena & Doerner, Karl F., 2010. "Ambulance location and relocation problems with time-dependent travel times," European Journal of Operational Research, Elsevier, vol. 207(3), pages 1293-1303, December.
    20. Ridler, Samuel & Mason, Andrew J. & Raith, Andrea, 2022. "A simulation and optimisation package for emergency medical services," European Journal of Operational Research, Elsevier, vol. 298(3), pages 1101-1113.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jomega:v:101:y:2021:i:c:s0305048319308229. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/375/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.