IDEAS home Printed from https://ideas.repec.org/a/spr/queues/v108y2024i3d10.1007_s11134-024-09927-w.html
   My bibliography  Save this article

Applying queueing theory to evaluate wait-time-savings of triage algorithms

Author

Listed:
  • Yee Lam Elim Thompson

    (The U.S. Food and Drug Administration)

  • Gary M. Levine

    (The U.S. Food and Drug Administration)

  • Weijie Chen

    (The U.S. Food and Drug Administration)

  • Berkman Sahiner

    (The U.S. Food and Drug Administration)

  • Qin Li

    (The U.S. Food and Drug Administration)

  • Nicholas Petrick

    (The U.S. Food and Drug Administration)

  • Jana G. Delfino

    (The U.S. Food and Drug Administration)

  • Miguel A. Lago

    (The U.S. Food and Drug Administration)

  • Qian Cao

    (The U.S. Food and Drug Administration)

  • Frank W. Samuelson

    (The U.S. Food and Drug Administration)

Abstract

In the past decade, artificial intelligence (AI) algorithms have made promising impacts in many areas of healthcare. One application is AI-enabled prioritization software known as computer-aided triage and notification (CADt). This type of software as a medical device is intended to prioritize reviews of radiological images with time-sensitive findings, thus shortening the waiting time for patients with these findings. While many CADt devices have been deployed into clinical workflows and have been shown to improve patient treatment and clinical outcomes, quantitative methods to evaluate the wait-time-savings from their deployment are not yet available. In this paper, we apply queueing theory methods to evaluate the wait-time-savings of a CADt by calculating the average waiting time per patient image without and with a CADt device being deployed. We study two workflow models with one or multiple radiologists (servers) for a range of AI diagnostic performances, radiologist’s reading rates, and patient image (customer) arrival rates. To evaluate the time-saving performance of a CADt, we use the difference in the mean waiting time between the diseased patient images in the with-CADt scenario and that in the without-CADt scenario as our performance metric. As part of this effort, we have developed and also share a software tool to simulate the radiology workflow around medical image interpretation, to verify theoretical results, and to provide confidence intervals for the performance metric we defined. We show quantitatively that a CADt triage device is more effective in a busy, short-staffed reading setting, which is consistent with our clinical intuition and simulation results. Although this work is motivated by the need for evaluating CADt devices, the evaluation methodology presented in this paper can be applied to assess the time-saving performance of other types of algorithms that prioritize a subset of customers based on binary outputs.

Suggested Citation

  • Yee Lam Elim Thompson & Gary M. Levine & Weijie Chen & Berkman Sahiner & Qin Li & Nicholas Petrick & Jana G. Delfino & Miguel A. Lago & Qian Cao & Frank W. Samuelson, 2024. "Applying queueing theory to evaluate wait-time-savings of triage algorithms," Queueing Systems: Theory and Applications, Springer, vol. 108(3), pages 579-610, December.
    • Handle: RePEc:spr:queues:v:108:y:2024:i:3:d:10.1007_s11134-024-09927-w
    DOI: 10.1007/s11134-024-09927-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11134-024-09927-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11134-024-09927-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Pengyi Shi & Jonathan E. Helm & H. Sebastian Heese & Alice M. Mitchell, 2021. "An Operational Framework for the Adoption and Integration of New Diagnostic Tests," Production and Operations Management, Production and Operations Management Society, vol. 30(2), pages 330-354, February.
    2. Zhankun Sun & Nilay Tanık Argon & Serhan Ziya, 2022. "When to Triage in Service Systems with Hidden Customer Class Identities?," Production and Operations Management, Production and Operations Management Society, vol. 31(1), pages 172-193, January.
    3. Dimitris Bertsimas & Daisuke Nakazato, 1995. "The Distributional Little's Law and Its Applications," Operations Research, INFORMS, vol. 43(2), pages 298-310, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rouba Ibrahim & Mor Armony & Achal Bassamboo, 2017. "Does the Past Predict the Future? The Case of Delay Announcements in Service Systems," Management Science, INFORMS, vol. 63(6), pages 1762-1780, June.
    2. George C. Mytalas & Michael A. Zazanis, 2022. "Service with a queue and a random capacity cart: random processing batches and E-limited policies," Annals of Operations Research, Springer, vol. 317(1), pages 147-178, October.
    3. Opher Baron & Oded Berman & Dmitry Krass & Jianfu Wang, 2014. "Using Strategic Idleness to Improve Customer Service Experience in Service Networks," Operations Research, INFORMS, vol. 62(1), pages 123-140, February.
    4. Gérard Hébuterne & Catherine Rosenberg, 1999. "Arrival and departure state distributions in the general bulk‐service queue," Naval Research Logistics (NRL), John Wiley & Sons, vol. 46(1), pages 107-118, February.
    5. Bertsimas, Dimitris., 1995. "Transient laws of non-stationary queueing systems and their applications," Working papers 3836-95., Massachusetts Institute of Technology (MIT), Sloan School of Management.
    6. John D. C. Little, 2011. "OR FORUM---Little's Law as Viewed on Its 50th Anniversary," Operations Research, INFORMS, vol. 59(3), pages 536-549, June.
    7. Josef Zuk & David Kirszenblat, 2024. "Explicit results for the distributions of queue lengths for a non-preemptive two-level priority queue," Annals of Operations Research, Springer, vol. 341(2), pages 1223-1246, October.
    8. Mor Harchol-Balter & Takayuki Osogami & Alan Scheller-Wolf & Adam Wierman, 2005. "Multi-Server Queueing Systems with Multiple Priority Classes," Queueing Systems: Theory and Applications, Springer, vol. 51(3), pages 331-360, December.
    9. Hossein Abouee-Mehrizi & Opher Baron & Oded Berman, 2014. "Exact Analysis of Capacitated Two-Echelon Inventory Systems with Priorities," Manufacturing & Service Operations Management, INFORMS, vol. 16(4), pages 561-577, October.
    10. Nam K. Kim & Kyung C. Chae & Mohan L. Chaudhry, 2004. "An Invariance Relation and a Unified Method to Derive Stationary Queue-Length Distributions," Operations Research, INFORMS, vol. 52(5), pages 756-764, October.
    11. Hossein Abouee-Mehrizi & Opher Baron, 2016. "State-dependent M/G/1 queueing systems," Queueing Systems: Theory and Applications, Springer, vol. 82(1), pages 121-148, February.
    12. Kristen Gardner & Rhonda Righter, 2020. "Product forms for FCFS queueing models with arbitrary server-job compatibilities: an overview," Queueing Systems: Theory and Applications, Springer, vol. 96(1), pages 3-51, October.
    13. Jianfu Wang & Opher Baron & Alan Scheller-Wolf, 2015. "M/M/c Queue with Two Priority Classes," Operations Research, INFORMS, vol. 63(3), pages 733-749, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:queues:v:108:y:2024:i:3:d:10.1007_s11134-024-09927-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.