IDEAS home Printed from https://ideas.repec.org/a/nat/nathum/v6y2022i11d10.1038_s41562-022-01437-0.html
   My bibliography  Save this article

A citywide experiment testing the impact of geographically targeted, high-pay-off vaccine lotteries

Author

Listed:
  • Katherine L. Milkman

    (University of Pennsylvania)

  • Linnea Gandhi

    (University of Pennsylvania)

  • Sean F. Ellis

    (University of Pennsylvania)

  • Heather N. Graci

    (University of Pennsylvania)

  • Dena M. Gromet

    (University of Pennsylvania)

  • Rayyan S. Mobarak

    (University of Pennsylvania)

  • Alison M. Buttenheim

    (University of Pennsylvania
    University of Pennsylvania)

  • Angela L. Duckworth

    (University of Pennsylvania
    University of Pennsylvania)

  • Devin Pope

    (University of Chicago)

  • Ala Stanford

    (US Department of Health and Human Services)

  • Richard Thaler

    (University of Chicago)

  • Kevin G. Volpp

    (University of Pennsylvania
    University of Pennsylvania)

Abstract

Lotteries have been shown to motivate behaviour change in many settings, but their value as a policy tool is relatively untested. We implemented a pre-registered, citywide experiment to test the effects of three high-pay-off, geographically targeted lotteries designed to motivate adult Philadelphians to get their COVID-19 vaccine. In each drawing, the residents of a randomly selected ‘treatment’ zip code received half the lottery prizes, boosting their chances of winning to 50×–100× those of other Philadelphians. The first treated zip code, which drew considerable media attention, may have experienced a small bump in vaccinations compared with the control zip codes: average weekly vaccinations rose by an estimated 61 per 100,000 people per week (+11%). After pooling the results from all three zip codes treated during our six-week experiment, however, we do not detect evidence of any overall benefits. Furthermore, our 95% confidence interval provides a 9% upper bound on the net benefits of treatment in our study.

Suggested Citation

  • Katherine L. Milkman & Linnea Gandhi & Sean F. Ellis & Heather N. Graci & Dena M. Gromet & Rayyan S. Mobarak & Alison M. Buttenheim & Angela L. Duckworth & Devin Pope & Ala Stanford & Richard Thaler &, 2022. "A citywide experiment testing the impact of geographically targeted, high-pay-off vaccine lotteries," Nature Human Behaviour, Nature, vol. 6(11), pages 1515-1524, November.
    • Handle: RePEc:nat:nathum:v:6:y:2022:i:11:d:10.1038_s41562-022-01437-0
    DOI: 10.1038/s41562-022-01437-0
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41562-022-01437-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1038/s41562-022-01437-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. MacKinnon, James G. & Webb, Matthew D., 2020. "Randomization inference for difference-in-differences with few treated clusters," Journal of Econometrics, Elsevier, vol. 218(2), pages 435-450.
    2. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    3. Nicola Lacetera & Devin G. Pope & Justin R. Sydnor, 2012. "Heuristic Thinking and Limited Attention in the Car Market," American Economic Review, American Economic Association, vol. 102(5), pages 2206-2236, August.
    4. Sebastian Galiani & Brian Quistorff, 2017. "The synth runner package: Utilities to automate synthetic control estimation using synth," Stata Journal, StataCorp LP, vol. 17(4), pages 834-849, December.
    5. Eduardo Cavallo & Sebastian Galiani & Ilan Noy & Juan Pantano, 2013. "Catastrophic Natural Disasters and Economic Growth," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1549-1561, December.
    6. Margaret E. Brehm & Paul A. Brehm & Martin Saavedra, 2022. "The Ohio Vaccine Lottery and Starting Vaccination Rates," American Journal of Health Economics, University of Chicago Press, vol. 8(3), pages 387-411.
    7. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    8. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    9. Arindrajit Dube & Ethan Kaplan & Suresh Naidu, 2011. "Coups, Corporations, and Classified Information," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(3), pages 1375-1409.
    10. Drazen Prelec, 1998. "The Probability Weighting Function," Econometrica, Econometric Society, vol. 66(3), pages 497-528, May.
    11. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    12. Hengchen Dai & Silvia Saccardo & Maria A. Han & Lily Roh & Naveen Raja & Sitaram Vangala & Hardikkumar Modi & Shital Pandya & Michael Sloyan & Daniel M. Croymans, 2021. "Behavioural nudges increase COVID-19 vaccinations," Nature, Nature, vol. 597(7876), pages 404-409, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elinder, Mikael & Erixson, Oscar & Öhman, Mattias, 2023. "Cognitive ability, health policy, and the dynamics of COVID-19 vaccination," Journal of Health Economics, Elsevier, vol. 91(C).
    2. Upravitelev, A., 2023. "Efficacious methods of restraining COVID-19 through behavioral public policy," Journal of the New Economic Association, New Economic Association, vol. 61(4), pages 203-222.
    3. Pol Campos-Mercade & Armando N. Meier & Stephan Meier & Devin G. Pope & Florian H. Schneider & Erik Wengström, 2024. "Incentives to Vaccinate," NBER Working Papers 32899, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Denti, Daria & Iammarino, Simona, 2022. "Coming Out of the Woods. Do local support services influence the propensity to report sexual violence?," Journal of Economic Behavior & Organization, Elsevier, vol. 193(C), pages 334-352.
    2. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    3. Joseph, Iverson-Love, 2022. "The effect of natural disaster on economic growth: Evidence from a major earthquake in Haiti," World Development, Elsevier, vol. 159(C).
    4. Rodríguez-Puello, Gabriel, 2024. "Digging for Trouble? Uncovering the Link Between Mining Booms and Crime," OSF Preprints s8ayp, Center for Open Science.
    5. Matthew D. Webb, 2023. "Reworking wild bootstrap‐based inference for clustered errors," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 56(3), pages 839-858, August.
    6. Rok Spruk & Mitja Kovac, 2020. "Does a ban on trans fats improve public health: synthetic control evidence from Denmark," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 156(1), pages 1-32, December.
    7. Nicolaj N. Mühlbach, 2020. "Tree-based Synthetic Control Methods: Consequences of moving the US Embassy," CREATES Research Papers 2020-04, Department of Economics and Business Economics, Aarhus University.
    8. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    9. Maximiliano Marzetti & Rok Spruk, 2023. "Long-Term Economic Effects of Populist Legal Reforms: Evidence from Argentina," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 65(1), pages 60-95, March.
    10. Bibek Adhikari & Romain Duval & Bingjie Hu & Prakash Loungani, 2018. "Can Reform Waves Turn the Tide? Some Case Studies using the Synthetic Control Method," Open Economies Review, Springer, vol. 29(4), pages 879-910, September.
    11. Peter Backus & Thien Nguyen, 2021. "The Effect of the Sex Buyer Law on the Market for Sex, Sexual Health and Sexual Violence," Economics Discussion Paper Series 2106, Economics, The University of Manchester.
    12. Girma, Sourafel & Paton, David, 2022. "Is assisted suicide a substitute for unassisted suicide?," European Economic Review, Elsevier, vol. 145(C).
    13. Nathan, Max, 2022. "Does light touch cluster policy work? Evaluating the tech city programme," Research Policy, Elsevier, vol. 51(9).
    14. Avdic, Daniel & von Hinke, Stephanie, 2021. "Extending alcohol retailers’ opening hours: Evidence from Sweden," European Economic Review, Elsevier, vol. 138(C).
    15. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    16. Almer, Christian & Winkler, Ralph, 2017. "Analyzing the effectiveness of international environmental policies: The case of the Kyoto Protocol," Journal of Environmental Economics and Management, Elsevier, vol. 82(C), pages 125-151.
    17. Dale Belman & Paul Wolfson & Kritkorn Nawakitphaitoon, 2015. "Who Is Affected by the Minimum Wage?," Industrial Relations: A Journal of Economy and Society, Wiley Blackwell, vol. 54(4), pages 582-621, October.
    18. Victor Chernozhukov & Kaspar Wuthrich & Yinchu Zhu, 2019. "Distributional conformal prediction," Papers 1909.07889, arXiv.org, revised Aug 2021.
    19. Victor Chernozhukov & Kaspar Wüthrich & Yinchu Zhu, 2021. "An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1849-1864, October.
    20. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nathum:v:6:y:2022:i:11:d:10.1038_s41562-022-01437-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.