IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0234817.html
   My bibliography  Save this article

It’s how you say it: Systematic A/B testing of digital messaging cut hospital no-show rates

Author

Listed:
  • Adi Berliner Senderey
  • Tamar Kornitzer
  • Gabriella Lawrence
  • Hilla Zysman
  • Yael Hallak
  • Dan Ariely
  • Ran Balicer

Abstract

Failure to attend hospital appointments has a detrimental impact on care quality. Documented efforts to address this challenge have only modestly decreased no-show rates. Behavioral economics theory has suggested that more effective messages may lead to increased responsiveness. In complex, real-world settings, it has proven difficult to predict the optimal message composition. In this study, we aimed to systematically compare the effects of several pre-appointment message formats on no-show rates. We randomly assigned members from Clalit Health Services (CHS), the largest payer-provider healthcare organization in Israel, who had scheduled outpatient clinic appointments in 14 CHS hospitals, to one of nine groups. Each individual received a pre-appointment SMS text reminder five days before the appointment, which differed by group. No-show and advanced cancellation rates were compared between the eight alternative messages, with the previously used generic message serving as the control. There were 161,587 CHS members who received pre-appointment reminder messages who were included in this study. Five message frames significantly differed from the control group. Members who received a reminder designed to evoke emotional guilt had a no-show rates of 14.2%, compared with 21.1% in the control group (odds ratio [OR]: 0.69, 95% confidence interval [CI]: 0.67, 0.76), and an advanced cancellation rate of 26.3% compared with 17.2% in the control group (OR: 1.2, 95% CI: 1.19, 1.21). Four additional reminder formats demonstrated significantly improved impact on no-show rates, compared to the control, though not as effective as the best performing message format. Carefully selecting the narrative of pre-appointment SMS reminders can lead to a marked decrease in no-show rates. The process of a/b testing, selecting, and adopting optimal messages is a practical example of implementing the learning healthcare system paradigm, which could prevent up to one-third of the 352,000 annually unattended appointments in Israel.

Suggested Citation

  • Adi Berliner Senderey & Tamar Kornitzer & Gabriella Lawrence & Hilla Zysman & Yael Hallak & Dan Ariely & Ran Balicer, 2020. "It’s how you say it: Systematic A/B testing of digital messaging cut hospital no-show rates," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-13, June.
    • Handle: RePEc:plo:pone00:0234817
    DOI: 10.1371/journal.pone.0234817
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0234817
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0234817&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0234817?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bertoni, Marco & Corazzini, Luca & Robone, Silvana, 2019. "Promoting Breast Cancer Screening Take-Ups with Zero Cost: Evidence from an Experiment on Formatting Invitation Letters in Italy," IZA Discussion Papers 12193, Institute of Labor Economics (IZA).
    2. Shane Frederick & Nathan Novemsky & Jing Wang & Ravi Dhar & Stephen Nowlis, 2009. "Opportunity Cost Neglect," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 36(4), pages 553-561, December.
    3. Noah J. Goldstein & Robert B. Cialdini & Vladas Griskevicius, 2008. "A Room with a Viewpoint: Using Social Norms to Motivate Environmental Conservation in Hotels," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 35(3), pages 472-482, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Berman, Jonathan Z. & Bhattacharjee, Amit & Small, Deborah A. & Zauberman, Gal, 2020. "Passing the buck to the wealthier: Reference-dependent standards of generosity," Organizational Behavior and Human Decision Processes, Elsevier, vol. 157(C), pages 46-56.
    2. Kettle, Keri L. & Mantonakis, Antonia, 2024. "Look for the signature: Using personal signatures as extrinsic cues promotes identity-congruent behavior," Journal of Business Research, Elsevier, vol. 170(C).
    3. Todd D. Gerarden & Richard G. Newell & Robert N. Stavins, 2017. "Assessing the Energy-Efficiency Gap," Journal of Economic Literature, American Economic Association, vol. 55(4), pages 1486-1525, December.
    4. Sleesman, Dustin J., 2019. "Pushing through the tension while stuck in the mud: Paradox mindset and escalation of commitment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 155(C), pages 83-96.
    5. Alpízar, Francisco & Martinsson, Peter, 2010. "Don’t Tell Me What to Do, Tell Me Who to Follow! - Field Experiment Evidence on Voluntary Donations," Working Papers in Economics 452, University of Gothenburg, Department of Economics.
    6. Asuamah Yeboah, Samuel, 2023. "Sustaining Change: Unravelling the Socio-cultural Threads of Sustainable Consumption," MPRA Paper 117981, University Library of Munich, Germany, revised 10 Jun 2023.
    7. Nathaniel Geiger, 2022. "Perceptions of Self-Motives and Environmental Activists’ Motives for Pro-Environmental Behavior," Sustainability, MDPI, vol. 14(17), pages 1-12, August.
    8. Yizhao Jiang, 2022. "The Influence of Payment Method: Do Consumers Pay More with Mobile Payment?," Papers 2210.14631, arXiv.org.
    9. Ajla Cosic & Hana Cosic & Sebastian Ille, 2018. "Can nudges affect students' green behaviour? A field experiment," Journal of Behavioral Economics for Policy, Society for the Advancement of Behavioral Economics (SABE), vol. 2(1), pages 107-111, March.
    10. Sarah Verdonk & Keri Chiveralls & Drew Dawson, 2017. "Getting Wasted at WOMADelaide: The Effect of Signage on Waste Disposal," Sustainability, MDPI, vol. 9(3), pages 1-17, February.
    11. Wang, Le & Luo, Xin (Robert) & Li, Han, 2022. "Envy or conformity? An empirical investigation of peer influence on the purchase of non-functional items in mobile free-to-play games," Journal of Business Research, Elsevier, vol. 147(C), pages 308-324.
    12. Bartels, Lara & Kesternich, Martin, 2022. "Motivate the crowd or crowd- them out? The impact of local government spending on the voluntary provision of a green public good," ZEW Discussion Papers 22-040, ZEW - Leibniz Centre for European Economic Research.
    13. Le Thi Dieu Hien & Khuu Ngoc Huyen & Thi Hong Loc Hoang, 2023. "Factors Affecting Energy-Saving Intentions among Youth in Vietnam," International Journal of Energy Economics and Policy, Econjournals, vol. 13(6), pages 603-609, November.
    14. Ingo Balderjahn & Dennis Appenfeller, 2023. "A Social Marketing Approach to Voluntary Simplicity: Communicating to Consume Less," Sustainability, MDPI, vol. 15(3), pages 1-17, January.
    15. Gwen-Jiro Clochard & Guillaume Hollard & Julia Wirtz, 2022. "More effort or better technologies? On the effect of relative performance feedback," Bristol Economics Discussion Papers 22/767, School of Economics, University of Bristol, UK.
    16. Arne K. Albrecht & Gianfranco Walsh & Simon Brach & Dwayne D. Gremler & Erica Herpen, 2017. "The influence of service employees and other customers on customer unfriendliness: a social norms perspective," Journal of the Academy of Marketing Science, Springer, vol. 45(6), pages 827-847, November.
    17. Ganserer, Angelika, 2021. "Non-compliance with temporary agency work regulations: Initial evidence from Germany," ZEW Discussion Papers 21-057, ZEW - Leibniz Centre for European Economic Research.
    18. Ming, Yaxin & Deng, Huixin & Wu, Xiaoyue, 2022. "The negative effect of air pollution on people's pro-environmental behavior," Journal of Business Research, Elsevier, vol. 142(C), pages 72-87.
    19. Zou, Lili Wenli & Chan, Ricky Y.K., 2019. "Why and when do consumers perform green behaviors? An examination of regulatory focus and ethical ideology," Journal of Business Research, Elsevier, vol. 94(C), pages 113-127.
    20. Cloé Garnache & Scott M. Swinton & Joseph A. Herriges & Frank Lupi & R. Jan Stevenson, 2016. "Solving the Phosphorus Pollution Puzzle: Synthesis and Directions for Future Research," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 98(5), pages 1334-1359.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0234817. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.