IDEAS home Printed from https://ideas.repec.org/p/ecl/harjfk/rwp17-036.html
   My bibliography  Save this paper

Data-Driven Management of Post-transplant Medications: An APOMDP Approach

Author

Listed:
  • Boloori, Alireza

    (Arizona State University)

  • Saghafian, Soroush

    (Harvard University)

  • Chakkera, Harini A. A.

    (Mayo Clinic Hospital)

  • Cook, Curtiss B.

    (Mayo Clinic Hospital)

Abstract

Organ-transplanted patients typically receive high amounts of immunosuppressive drugs (e.g., tacrolimus) as a mechanism to reduce their risk of organ rejection. However, due to the diabetogenic effect of these drugs, this practice exposes them to greater risk of New-Onset Diabetes After Trans-plant (NODAT), and hence, becoming insulin-dependent. This common conundrum of balancing the risk of organ rejection versus that of NODAT is further complicated due to various factors that create ambiguity in quantifying risks: (1) false-positive and false-negative errors of medical tests,(2) inevitable estimation errors when data sets are used, (3) variability among physicians’ attitudes towards ambiguous outcomes, and (4) dynamic and patient risk-profile dependent progression of health conditions. To address these challenges, we propose an ambiguous partially observable Markov decision process (APOMDP) framework, where dynamic optimization with respect to a “cloud†of possible models allows us to make decisions that are robust to misspecifications of risks. We first provide various structural results that facilitate characterizing the optimal policy. Using a clinical data set, we then compare the optimal policy to the current practice as well as some other bench-marks, and discuss various implications for both policy makers and physicians. In particular, our results show that substantial improvements are achievable in two important dimensions: (a) the quality-adjusted life expectancy (QALE) of patients, and (b) medical expenditures.

Suggested Citation

  • Boloori, Alireza & Saghafian, Soroush & Chakkera, Harini A. A. & Cook, Curtiss B., 2017. "Data-Driven Management of Post-transplant Medications: An APOMDP Approach," Working Paper Series rwp17-036, Harvard University, John F. Kennedy School of Government.
    • Handle: RePEc:ecl:harjfk:rwp17-036
    as

    Download full text from publisher

    File URL: https://research.hks.harvard.edu/publications/getFile.aspx?Id=1576
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Turgay Ayer & Oguzhan Alagoz & Natasha K. Stout, 2012. "OR Forum---A POMDP Approach to Personalize Mammography Screening Decisions," Operations Research, INFORMS, vol. 60(5), pages 1019-1034, October.
    2. van Buuren, Stef & Groothuis-Oudshoorn, Karin, 2011. "mice: Multivariate Imputation by Chained Equations in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 45(i03).
    3. Oguzhan Alagoz & Cindy L. Bryce & Steven Shechter & Andrew Schaefer & Chung-Chou H. Chang & Derek C. Angus & Mark S. Roberts, 2005. "Incorporating Biological Natural History in Simulation Models: Empirical Estimates of the Progression of End-Stage Liver Disease," Medical Decision Making, , vol. 25(6), pages 620-632, November.
    4. Erick Delage & Shie Mannor, 2010. "Percentile Optimization for Markov Decision Processes with Parameter Uncertainty," Operations Research, INFORMS, vol. 58(1), pages 203-213, February.
    5. MEI, Xingxing & FENG, Zhongchao & HE, Pinghua & GAO, Yawen & DAI, Yuqin, 2015. "Further Understanding of the Food Safety Problem," Asian Agricultural Research, USA-China Science and Culture Media Corporation, vol. 7(07), pages 1-4, July.
    6. Xuanming Su & Stefanos A. Zenios, 2005. "Patient Choice in Kidney Allocation: A Sequential Stochastic Assignment Model," Operations Research, INFORMS, vol. 53(3), pages 443-455, June.
    7. Lisa M. Maillart & Julie Simmons Ivy & Scott Ransom & Kathleen Diehl, 2008. "Assessing Dynamic Breast Cancer Screening Policies," Operations Research, INFORMS, vol. 56(6), pages 1411-1427, December.
    8. Christos H. Papadimitriou & John N. Tsitsiklis, 1987. "The Complexity of Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 12(3), pages 441-450, August.
    9. Fatih Safa Erenay & Oguzhan Alagoz & Adnan Said, 2014. "Optimizing Colonoscopy Screening for Colorectal Cancer Prevention and Surveillance," Manufacturing & Service Operations Management, INFORMS, vol. 16(3), pages 381-400, July.
    10. Brian T. Denton & Murat Kurt & Nilay D. Shah & Sandra C. Bryant & Steven A. Smith, 2009. "Optimizing the Start Time of Statin Therapy for Patients with Diabetes," Medical Decision Making, , vol. 29(3), pages 351-367, May.
    11. Oguzhan Alagoz & Lisa M. Maillart & Andrew J. Schaefer & Mark S. Roberts, 2004. "The Optimal Timing of Living-Donor Liver Transplantation," Management Science, INFORMS, vol. 50(10), pages 1420-1430, October.
    12. Arnab Nilim & Laurent El Ghaoui, 2005. "Robust Control of Markov Decision Processes with Uncertain Transition Matrices," Operations Research, INFORMS, vol. 53(5), pages 780-798, October.
    13. Burhaneddin Sandıkçı & Lisa M. Maillart & Andrew J. Schaefer & Mark S. Roberts, 2013. "Alleviating the Patient's Price of Privacy Through a Partially Observable Waiting List," Management Science, INFORMS, vol. 59(8), pages 1836-1854, August.
    14. George E. Monahan, 1982. "State of the Art---A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms," Management Science, INFORMS, vol. 28(1), pages 1-16, January.
    15. Mason, J.E. & Denton, B.T. & Shah, N.D. & Smith, S.A., 2014. "Optimizing the simultaneous management of blood pressure and cholesterol for type 2 diabetes patients," European Journal of Operational Research, Elsevier, vol. 233(3), pages 727-738.
    16. Wolfram Wiesemann & Daniel Kuhn & Berç Rustem, 2013. "Robust Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 38(1), pages 153-183, February.
    17. Jennifer E. Mason & Darin A. England & Brian T. Denton & Steven A. Smith & Murat Kurt & Nilay D. Shah, 2012. "Optimizing Statin Treatment Decisions for Diabetes Patients in the Presence of Uncertain Future Adherence," Medical Decision Making, , vol. 32(1), pages 154-166, January.
    18. Karlin, Samuel & Rinott, Yosef, 1980. "Classes of orderings of measures and related correlation inequalities. I. Multivariate totally positive distributions," Journal of Multivariate Analysis, Elsevier, vol. 10(4), pages 467-498, December.
    19. Garud N. Iyengar, 2005. "Robust Dynamic Programming," Mathematics of Operations Research, INFORMS, vol. 30(2), pages 257-280, May.
    20. Chuanpu Hu & William S. Lovejoy & Steven L. Shafer, 1996. "Comparison of Some Suboptimal Control Policies in Medical Drug Therapy," Operations Research, INFORMS, vol. 44(5), pages 696-709, October.
    21. Huan Xu & Shie Mannor, 2012. "Distributionally Robust Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 37(2), pages 288-300, May.
    22. Goh, Joel & Bayati, Mohsen & Zenios, Stefanos A. & Singh, Sundeep & Moore, David, 2015. "Data Uncertainty in Markov Chains: Application to Cost-Effectiveness Analyses of Medical Innovations," Research Papers 3283, Stanford University, Graduate School of Business.
    23. Joseph Fairchild & Jun Ma & Shu Wu, 2015. "Understanding Housing Market Volatility," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 47(7), pages 1309-1337, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andrew J. Keith & Darryl K. Ahner, 2021. "A survey of decision making and optimization under uncertainty," Annals of Operations Research, Springer, vol. 300(2), pages 319-353, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alireza Boloori & Soroush Saghafian & Harini A. Chakkera & Curtiss B. Cook, 2020. "Data-Driven Management of Post-transplant Medications: An Ambiguous Partially Observable Markov Decision Process Approach," Manufacturing & Service Operations Management, INFORMS, vol. 22(5), pages 1066-1087, September.
    2. Saghafian, Soroush, 2018. "Ambiguous partially observable Markov decision processes: Structural results and applications," Journal of Economic Theory, Elsevier, vol. 178(C), pages 1-35.
    3. Zlatana Nenova & Jennifer Shang, 2022. "Personalized Chronic Disease Follow‐Up Appointments: Risk‐Stratified Care Through Big Data," Production and Operations Management, Production and Operations Management Society, vol. 31(2), pages 583-606, February.
    4. Shie Mannor & Ofir Mebel & Huan Xu, 2016. "Robust MDPs with k -Rectangular Uncertainty," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1484-1509, November.
    5. Andrew J. Keith & Darryl K. Ahner, 2021. "A survey of decision making and optimization under uncertainty," Annals of Operations Research, Springer, vol. 300(2), pages 319-353, May.
    6. Bren, Austin & Saghafian, Soroush, 2018. "Data-Driven Percentile Optimization for Multi-Class Queueing Systems with Model Ambiguity: Theory and Application," Working Paper Series rwp18-008, Harvard University, John F. Kennedy School of Government.
    7. Burhaneddin Sandıkçı & Lisa M. Maillart & Andrew J. Schaefer & Oguzhan Alagoz & Mark S. Roberts, 2008. "Estimating the Patient's Price of Privacy in Liver Transplantation," Operations Research, INFORMS, vol. 56(6), pages 1393-1410, December.
    8. Michael Jong Kim, 2016. "Robust Control of Partially Observable Failing Systems," Operations Research, INFORMS, vol. 64(4), pages 999-1014, August.
    9. Ting-Yu Ho & Shan Liu & Zelda B. Zabinsky, 2019. "A Multi-Fidelity Rollout Algorithm for Dynamic Resource Allocation in Population Disease Management," Health Care Management Science, Springer, vol. 22(4), pages 727-755, December.
    10. Jonathan E. Helm & Mariel S. Lavieri & Mark P. Van Oyen & Joshua D. Stein & David C. Musch, 2015. "Dynamic Forecasting and Control Algorithms of Glaucoma Progression for Clinician Decision Support," Operations Research, INFORMS, vol. 63(5), pages 979-999, October.
    11. Kotas, Jakob & Ghate, Archis, 2018. "Bayesian learning of dose–response parameters from a cohort under response-guided dosing," European Journal of Operational Research, Elsevier, vol. 265(1), pages 328-343.
    12. Burhaneddin Sandıkçı & Lisa M. Maillart & Andrew J. Schaefer & Mark S. Roberts, 2013. "Alleviating the Patient's Price of Privacy Through a Partially Observable Waiting List," Management Science, INFORMS, vol. 59(8), pages 1836-1854, August.
    13. Muge Capan & Julie S. Ivy & James R. Wilson & Jeanne M. Huddleston, 2017. "A stochastic model of acute-care decisions based on patient and provider heterogeneity," Health Care Management Science, Springer, vol. 20(2), pages 187-206, June.
    14. Maximilian Blesch & Philipp Eisenhauer, 2021. "Robust decision-making under risk and ambiguity," Papers 2104.12573, arXiv.org, revised Oct 2021.
    15. Zeynep Turgay & Fikri Karaesmen & Egemen Lerzan Örmeci, 2018. "Structural properties of a class of robust inventory and queueing control problems," Naval Research Logistics (NRL), John Wiley & Sons, vol. 65(8), pages 699-716, December.
    16. Varagapriya, V & Singh, Vikas Vikram & Lisser, Abdel, 2024. "Rank-1 transition uncertainties in constrained Markov decision processes," European Journal of Operational Research, Elsevier, vol. 318(1), pages 167-178.
    17. Maximilian Blesch & Philipp Eisenhauer, 2023. "Robust Decision-Making under Risk and Ambiguity," Rationality and Competition Discussion Paper Series 463, CRC TRR 190 Rationality and Competition.
    18. Wolfram Wiesemann & Daniel Kuhn & Berç Rustem, 2013. "Robust Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 38(1), pages 153-183, February.
    19. V Varagapriya & Vikas Vikram Singh & Abdel Lisser, 2023. "Joint chance-constrained Markov decision processes," Annals of Operations Research, Springer, vol. 322(2), pages 1013-1035, March.
    20. Zhu, Zhicheng & Xiang, Yisha & Zhao, Ming & Shi, Yue, 2023. "Data-driven remanufacturing planning with parameter uncertainty," European Journal of Operational Research, Elsevier, vol. 309(1), pages 102-116.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecl:harjfk:rwp17-036. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/ksharus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.