IDEAS home Printed from https://ideas.repec.org/a/kap/hcarem/v27y2024i4d10.1007_s10729-024-09692-5.html
   My bibliography  Save this article

A reinforcement learning approach for the online dynamic home health care scheduling problem

Author

Listed:
  • Quy Ta-Dinh

    (Phenikaa University)

  • Tu-San Pham

    (Polytechnique Montreal)

  • Minh Hoàng Hà

    (College of Technology, National Economics University)

  • Louis-Martin Rousseau

    (Polytechnique Montreal)

Abstract

Over recent years, home health care has gained significant attention as an efficient solution to the increasing demand for healthcare services. Home health care scheduling is a challenging problem involving multiple complicated assignments and routing decisions subject to various constraints. The problem becomes even more challenging when considered on a rolling horizon with stochastic patient requests. This paper discusses the Online Dynamic Home Health Care Scheduling Problem (ODHHCSP), in which a home health care agency has to decide whether to accept or reject a patient request and determine the visit schedule and routes in case of acceptance. The objective of the problem is to maximize the number of patients served, given the limited resources. When the agency receives a patient’s request, a decision must be made on the spot, which poses many challenges, such as stochastic future requests or a limited time budget for decision-making. In this paper, we model the problem as a Markov decision process and propose a reinforcement learning (RL) approach. The experimental results show that the proposed approach outperforms other algorithms in the literature in terms of solution quality. In addition, a constant runtime of less than 0.001 seconds for each decision makes the approach especially suitable for an online setting like our problem.

Suggested Citation

  • Quy Ta-Dinh & Tu-San Pham & Minh Hoàng Hà & Louis-Martin Rousseau, 2024. "A reinforcement learning approach for the online dynamic home health care scheduling problem," Health Care Management Science, Springer, vol. 27(4), pages 650-664, December.
  • Handle: RePEc:kap:hcarem:v:27:y:2024:i:4:d:10.1007_s10729-024-09692-5
    DOI: 10.1007/s10729-024-09692-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10729-024-09692-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10729-024-09692-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Aliza Heching & J. N. Hooker & Ryo Kimura, 2019. "A Logic-Based Benders Approach to Home Healthcare Delivery," Transportation Science, INFORMS, vol. 53(2), pages 510-522, March.
    2. C. Rodriguez & T. Garaix & X. Xie & V. Augusto, 2015. "Staff dimensioning in homecare services with uncertain demands," International Journal of Production Research, Taylor & Francis Journals, vol. 53(24), pages 7396-7410, December.
    3. Soeffker, Ninja & Ulmer, Marlin W. & Mattfeld, Dirk C., 2022. "Stochastic dynamic vehicle routing in the light of prescriptive analytics: A review," European Journal of Operational Research, Elsevier, vol. 298(3), pages 801-820.
    4. Nikzad, Erfaneh & Bashiri, Mahdi & Abbasi, Babak, 2021. "A matheuristic algorithm for stochastic home health care planning," European Journal of Operational Research, Elsevier, vol. 288(3), pages 753-774.
    5. Chen, Xinwei & Ulmer, Marlin W. & Thomas, Barrett W., 2022. "Deep Q-learning for same-day delivery with vehicles and drones," European Journal of Operational Research, Elsevier, vol. 298(3), pages 939-952.
    6. Carello, Giuliana & Lanzarone, Ettore, 2014. "A cardinality-constrained robust model for the assignment problem in Home Care services," European Journal of Operational Research, Elsevier, vol. 236(2), pages 748-762.
    7. Zhan, Yang & Wang, Zizhuo & Wan, Guohua, 2021. "Home service routing and appointment scheduling with stochastic service times," European Journal of Operational Research, Elsevier, vol. 288(1), pages 98-110.
    8. Sachidanand V. Begur & David M. Miller & Jerry R. Weaver, 1997. "An Integrated Spatial DSS for Scheduling and Routing Home-Health-Care Nurses," Interfaces, INFORMS, vol. 27(4), pages 35-48, August.
    9. Mustafa Demirbilek & Juergen Branke & Arne K. Strauss, 2021. "Home healthcare routing and scheduling of multiple nurses in a dynamic environment," Flexible Services and Manufacturing Journal, Springer, vol. 33(1), pages 253-280, March.
    10. Grenouilleau, Florian & Legrain, Antoine & Lahrichi, Nadia & Rousseau, Louis-Martin, 2019. "A set partitioning heuristic for the home health care routing and scheduling problem," European Journal of Operational Research, Elsevier, vol. 275(1), pages 295-303.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nasir, Jamal Abdul & Kuo, Yong-Hong, 2024. "Stochastic home care transportation with dynamically prioritized patients: An integrated facility location, fleet sizing, and routing approach," Transportation Research Part B: Methodological, Elsevier, vol. 184(C).
    2. Jalel Euchi & Malek Masmoudi & Patrick Siarry, 2022. "Home health care routing and scheduling problems: a literature review," 4OR, Springer, vol. 20(3), pages 351-389, September.
    3. Pahlevani, Delaram & Abbasi, Babak & Hearne, John W. & Eberhard, Andrew, 2022. "A cluster-based algorithm for home health care planning: A case study in Australia," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 166(C).
    4. Malek Masmoudi & Jalel Euchi & Patrick Siarry, 2024. "Home healthcare routing and scheduling: operations research approaches and contemporary challenges," Annals of Operations Research, Springer, vol. 343(2), pages 701-751, December.
    5. Zheng, Chenyang & Wang, Shuming & Li, Ningxin & Wu, Yuanhao, 2021. "Stochastic joint homecare service and capacity planning with nested decomposition approaches," European Journal of Operational Research, Elsevier, vol. 295(1), pages 203-222.
    6. Guo, Jia & Bard, Jonathan F., 2023. "A three-step optimization-based algorithm for home healthcare delivery," Socio-Economic Planning Sciences, Elsevier, vol. 87(PA).
    7. Nikzad, Erfaneh & Bashiri, Mahdi & Abbasi, Babak, 2021. "A matheuristic algorithm for stochastic home health care planning," European Journal of Operational Research, Elsevier, vol. 288(3), pages 753-774.
    8. Jamal Abdul Nasir & Chuangyin Dang, 2018. "Solving a More Flexible Home Health Care Scheduling and Routing Problem with Joint Patient and Nursing Staff Selection," Sustainability, MDPI, vol. 10(1), pages 1-22, January.
    9. Jean-François Cordeau & Manuel Iori & Dario Vezzali, 2024. "An updated survey of attended home delivery and service problems with a focus on applications," Annals of Operations Research, Springer, vol. 343(2), pages 885-922, December.
    10. Delaet, Arne & Ramaekers, Katrien & Hirsch, Patrick & Molenbruch, Yves & Braekers, Kris, 2024. "A matheuristic for integrated medium-term home healthcare planning," European Journal of Operational Research, Elsevier, vol. 319(2), pages 543-556.
    11. Gang Du & Luyao Zheng & Xiaoling Ouyang, 2019. "Real-time scheduling optimization considering the unexpected events in home health care," Journal of Combinatorial Optimization, Springer, vol. 37(1), pages 196-220, January.
    12. Cappanera, Paola & Visintin, Filippo & Vannelli, Sara, 2025. "Home-based care and center-based care: From being alternatives to being synergistic. Optimization models to support flexible care delivery," Omega, Elsevier, vol. 131(C).
    13. Chen, Xinwei & Wang, Tong & Thomas, Barrett W. & Ulmer, Marlin W., 2023. "Same-day delivery with fair customer service," European Journal of Operational Research, Elsevier, vol. 308(2), pages 738-751.
    14. Makboul, Salma & Kharraja, Said & Abbassi, Abderrahman & El Hilali Alaoui, Ahmed, 2024. "A multiobjective approach for weekly Green Home Health Care routing and scheduling problem with care continuity and synchronized services," Operations Research Perspectives, Elsevier, vol. 12(C).
    15. Zhang, Jian & Woensel, Tom Van, 2023. "Dynamic vehicle routing with random requests: A literature review," International Journal of Production Economics, Elsevier, vol. 256(C).
    16. Paola Cappanera & Maria Grazia Scutellà, 2022. "Addressing consistency and demand uncertainty in the Home Care planning problem," Flexible Services and Manufacturing Journal, Springer, vol. 34(1), pages 1-39, March.
    17. Carolin Bauerhenne & Jonathan Bard & Rainer Kolisch, 2024. "Robust Routing and Scheduling of Home Healthcare Workers: A Nested Branch-and-Price Approach," Papers 2407.06215, arXiv.org.
    18. Fleckenstein, David & Klein, Robert & Steinhardt, Claudius, 2023. "Recent advances in integrating demand management and vehicle routing: A methodological review," European Journal of Operational Research, Elsevier, vol. 306(2), pages 499-518.
    19. Bosse, Alexander & Ulmer, Marlin W. & Manni, Emanuele & Mattfeld, Dirk C., 2023. "Dynamic priority rules for combining on-demand passenger transportation and transportation of goods," European Journal of Operational Research, Elsevier, vol. 309(1), pages 399-408.
    20. Tan Yu & Yongpei Guan & Xiang Zhong, 2024. "Visiting nurses assignment and routing for decentralized telehealth service networks," Annals of Operations Research, Springer, vol. 341(2), pages 1191-1221, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:hcarem:v:27:y:2024:i:4:d:10.1007_s10729-024-09692-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.