IDEAS home Printed from https://ideas.repec.org/a/nat/nathum/v6y2022i11d10.1038_s41562-022-01437-0.html
   My bibliography  Save this article

A citywide experiment testing the impact of geographically targeted, high-pay-off vaccine lotteries

Author

Listed:
  • Katherine L. Milkman

    (University of Pennsylvania)

  • Linnea Gandhi

    (University of Pennsylvania)

  • Sean F. Ellis

    (University of Pennsylvania)

  • Heather N. Graci

    (University of Pennsylvania)

  • Dena M. Gromet

    (University of Pennsylvania)

  • Rayyan S. Mobarak

    (University of Pennsylvania)

  • Alison M. Buttenheim

    (University of Pennsylvania
    University of Pennsylvania)

  • Angela L. Duckworth

    (University of Pennsylvania
    University of Pennsylvania)

  • Devin Pope

    (University of Chicago)

  • Ala Stanford

    (US Department of Health and Human Services)

  • Richard Thaler

    (University of Chicago)

  • Kevin G. Volpp

    (University of Pennsylvania
    University of Pennsylvania)

Abstract

Lotteries have been shown to motivate behaviour change in many settings, but their value as a policy tool is relatively untested. We implemented a pre-registered, citywide experiment to test the effects of three high-pay-off, geographically targeted lotteries designed to motivate adult Philadelphians to get their COVID-19 vaccine. In each drawing, the residents of a randomly selected ‘treatment’ zip code received half the lottery prizes, boosting their chances of winning to 50×–100× those of other Philadelphians. The first treated zip code, which drew considerable media attention, may have experienced a small bump in vaccinations compared with the control zip codes: average weekly vaccinations rose by an estimated 61 per 100,000 people per week (+11%). After pooling the results from all three zip codes treated during our six-week experiment, however, we do not detect evidence of any overall benefits. Furthermore, our 95% confidence interval provides a 9% upper bound on the net benefits of treatment in our study.

Suggested Citation

  • Katherine L. Milkman & Linnea Gandhi & Sean F. Ellis & Heather N. Graci & Dena M. Gromet & Rayyan S. Mobarak & Alison M. Buttenheim & Angela L. Duckworth & Devin Pope & Ala Stanford & Richard Thaler &, 2022. "A citywide experiment testing the impact of geographically targeted, high-pay-off vaccine lotteries," Nature Human Behaviour, Nature, vol. 6(11), pages 1515-1524, November.
    • Handle: RePEc:nat:nathum:v:6:y:2022:i:11:d:10.1038_s41562-022-01437-0
    DOI: 10.1038/s41562-022-01437-0
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41562-022-01437-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1038/s41562-022-01437-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. MacKinnon, James G. & Webb, Matthew D., 2020. "Randomization inference for difference-in-differences with few treated clusters," Journal of Econometrics, Elsevier, vol. 218(2), pages 435-450.
    2. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    3. Margaret E. Brehm & Paul A. Brehm & Martin Saavedra, 2022. "The Ohio Vaccine Lottery and Starting Vaccination Rates," American Journal of Health Economics, University of Chicago Press, vol. 8(3), pages 387-411.
    4. Arindrajit Dube & Ethan Kaplan & Suresh Naidu, 2011. "Coups, Corporations, and Classified Information," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(3), pages 1375-1409.
    5. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    6. Eduardo Cavallo & Sebastian Galiani & Ilan Noy & Juan Pantano, 2013. "Catastrophic Natural Disasters and Economic Growth," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1549-1561, December.
    7. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    8. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    9. Nicola Lacetera & Devin G. Pope & Justin R. Sydnor, 2012. "Heuristic Thinking and Limited Attention in the Car Market," American Economic Review, American Economic Association, vol. 102(5), pages 2206-2236, August.
    10. Sebastian Galiani & Brian Quistorff, 2017. "The synth runner package: Utilities to automate synthetic control estimation using synth," Stata Journal, StataCorp LLC, vol. 17(4), pages 834-849, December.
    11. Drazen Prelec, 1998. "The Probability Weighting Function," Econometrica, Econometric Society, vol. 66(3), pages 497-528, May.
    12. Hengchen Dai & Silvia Saccardo & Maria A. Han & Lily Roh & Naveen Raja & Sitaram Vangala & Hardikkumar Modi & Shital Pandya & Michael Sloyan & Daniel M. Croymans, 2021. "Behavioural nudges increase COVID-19 vaccinations," Nature, Nature, vol. 597(7876), pages 404-409, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elinder, Mikael & Erixson, Oscar & Öhman, Mattias, 2023. "Cognitive ability, health policy, and the dynamics of COVID-19 vaccination," Journal of Health Economics, Elsevier, vol. 91(C).
    2. Upravitelev, A., 2023. "Efficacious methods of restraining COVID-19 through behavioral public policy," Journal of the New Economic Association, New Economic Association, vol. 61(4), pages 203-222.
    3. Pol Campos-Mercade & Armando N. Meier & Stephan Meier & Devin G. Pope & Florian H. Schneider & Erik Wengström, 2024. "Incentives to Vaccinate," NBER Working Papers 32899, National Bureau of Economic Research, Inc.
    4. Pol Campos-Mercade & Armando N. Meier & Stephan Meier & Devin Pope & Florian H. Schneider & Erik Wengstroem, 2025. "Incentives to Vaccinate," CEBI working paper series 24-15, University of Copenhagen. Department of Economics. The Center for Economic Behavior and Inequality (CEBI).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rodríguez-Puello, Gabriel, 2024. "Digging for Trouble? Uncovering the Link Between Mining Booms and Crime," OSF Preprints s8ayp_v1, Center for Open Science.
    2. Denti, Daria & Iammarino, Simona, 2022. "Coming Out of the Woods. Do local support services influence the propensity to report sexual violence?," Journal of Economic Behavior & Organization, Elsevier, vol. 193(C), pages 334-352.
    3. Rodríguez-Puello, Gabriel, 2024. "Digging for Trouble? Uncovering the Link Between Mining Booms and Crime," OSF Preprints s8ayp_v1, Center for Open Science.
    4. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    5. Joseph, Iverson-Love, 2022. "The effect of natural disaster on economic growth: Evidence from a major earthquake in Haiti," World Development, Elsevier, vol. 159(C).
    6. Matthew D. Webb, 2023. "Reworking wild bootstrap‐based inference for clustered errors," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 56(3), pages 839-858, August.
    7. Rok Spruk & Mitja Kovac, 2020. "Does a ban on trans fats improve public health: synthetic control evidence from Denmark," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 156(1), pages 1-32, December.
    8. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    9. Maximiliano Marzetti & Rok Spruk, 2023. "Long-Term Economic Effects of Populist Legal Reforms: Evidence from Argentina," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 65(1), pages 60-95, March.
    10. Nathan, Max, 2022. "Does light touch cluster policy work? Evaluating the tech city programme," Research Policy, Elsevier, vol. 51(9).
    11. Avdic, Daniel & von Hinke, Stephanie, 2021. "Extending alcohol retailers’ opening hours: Evidence from Sweden," European Economic Review, Elsevier, vol. 138(C).
    12. Almer, Christian & Winkler, Ralph, 2017. "Analyzing the effectiveness of international environmental policies: The case of the Kyoto Protocol," Journal of Environmental Economics and Management, Elsevier, vol. 82(C), pages 125-151.
    13. Victor Chernozhukov & Kaspar Wüthrich & Yinchu Zhu, 2021. "An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1849-1864, October.
    14. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    15. Jason Ward & Ben Ost, 2021. "The Effect of Large-scale Performance-Based Funding in Higher Education," Education Finance and Policy, MIT Press, vol. 16(1), pages 92-124, Winter.
    16. Belman, Dale. & Wolfson, Paul J., 2016. "What does the minimum wage do in developing countries? : A review of studies and methodologies," ILO Working Papers 994893283402676, International Labour Organization.
    17. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    18. Bonander, Carl & Ekman, Mats & Jakobsson, Niklas, 2022. "Vaccination nudges: A study of pre-booked COVID-19 vaccinations in Sweden," Social Science & Medicine, Elsevier, vol. 309(C).
    19. Amos Z. B. Flomo & Elissaios Papyrakis & Natascha Wagner, 2023. "Evaluating the economic effects of the Ebola virus disease in Liberia: A synthetic control approach," Journal of International Development, John Wiley & Sons, Ltd., vol. 35(6), pages 1478-1504, August.
    20. Victor Hugo de Oliveira & Ines Lee & Climent Quintana‐Domeque, 2022. "The effect of increasing Women's autonomy on primary and repeated caesarean sections in Brazil," Health Economics, John Wiley & Sons, Ltd., vol. 31(8), pages 1800-1804, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nathum:v:6:y:2022:i:11:d:10.1038_s41562-022-01437-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.