IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/30273.html
   My bibliography  Save this paper

Digital Public Health Interventions at Scale: The Impact of Social Media Advertising on Beliefs and Outcomes Related to COVID Vaccines

Author

Listed:
  • Susan Athey
  • Kristen Grabarz
  • Michael Luca
  • Nils C. Wernerfelt

Abstract

Public health organizations increasingly use social media advertising campaigns in pursuit of public health goals. In this paper, we evaluate the impact of about $40 million of social media advertisements that were run and experimentally tested on Facebook and Instagram, aimed at increasing COVID-19 vaccination rates in the first year of the vaccine roll-out. The 819 randomized experiments in our sample were run by 174 different public health organizations and collectively reached 2.1 billion individuals in 15 languages. We find that these campaigns are, on average, effective at influencing self-reported beliefs - shifting opinions close to 1% at baseline with a cost per influenced person of about $3.41. Combining this result with an estimate of the relationship between survey outcomes and vaccination rates derived from observational data yields an estimated cost per additional vaccination of about $5.68. There is further evidence that campaigns are especially effective at influencing users’ knowledge of how to get vaccines. Our results represent, to the best of our knowledge, the largest set of online public health interventions analyzed to date.

Suggested Citation

  • Susan Athey & Kristen Grabarz & Michael Luca & Nils C. Wernerfelt, 2022. "Digital Public Health Interventions at Scale: The Impact of Social Media Advertising on Beliefs and Outcomes Related to COVID Vaccines," NBER Working Papers 30273, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:30273
    Note: DEV EH LE LS PE POL PR
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w30273.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Shan Huang & Sinan Aral & Yu Jeffrey Hu & Erik Brynjolfsson, 2020. "Social Advertising Effectiveness Across Products: A Large-Scale Field Experiment," Marketing Science, INFORMS, vol. 39(6), pages 1142-1165, November.
    2. Stefano DellaVigna & Elizabeth Linos, 2022. "RCTs to Scale: Comprehensive Evidence From Two Nudge Units," Econometrica, Econometric Society, vol. 90(1), pages 81-116, January.
    3. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    4. Avi Goldfarb & Catherine E. Tucker, 2011. "Privacy Regulation and Online Advertising," Management Science, INFORMS, vol. 57(1), pages 57-71, January.
    5. Hummel, Dennis & Maedche, Alexander, 2019. "How effective is nudging? A quantitative review on the effect sizes and limits of empirical nudging studies," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 80(C), pages 47-58.
    6. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    7. Jay J. Van Bavel & Katherine Baicker & Paulo S. Boggio & Valerio Capraro & Aleksandra Cichocka & Mina Cikara & Molly J. Crockett & Alia J. Crum & Karen M. Douglas & James N. Druckman & John Drury & Oe, 2020. "Using social and behavioural science to support COVID-19 pandemic response," Nature Human Behaviour, Nature, vol. 4(5), pages 460-471, May.
    8. Marcella Alsan & Sarah Eichmeyer, 2024. "Experimental Evidence on the Effectiveness of Nonexperts for Improving Vaccine Demand," American Economic Journal: Economic Policy, American Economic Association, vol. 16(1), pages 394-414, February.
    9. Rachael Meager, 2019. "Understanding the Average Impact of Microcredit Expansions: A Bayesian Hierarchical Analysis of Seven Randomized Experiments," American Economic Journal: Applied Economics, American Economic Association, vol. 11(1), pages 57-91, January.
    10. repec:nas:journl:v:115:y:2018:p:e5254-e5255 is not listed on IDEAS
    11. Marcella Alsan & Amitabh Chandra & Kosali Simon, 2021. "The Great Unequalizer: Initial Health Effects of COVID-19 in the United States," Journal of Economic Perspectives, American Economic Association, vol. 35(3), pages 25-46, Summer.
    12. Sarah Moshary & Bradley T. Shapiro & Jihong Song, 2021. "How and When to Use the Political Cycle to Identify Advertising Effects," Marketing Science, INFORMS, vol. 40(2), pages 283-304, March.
    13. Christina M. Astley & Gaurav Tuli & Kimberly A. Mc Cord & Emily L. Cohn & Benjamin Rader & Tanner J. Varrelman & Samantha L. Chiu & Xiaoyi Deng & Kathleen Stewart & Tamer H. Farag & Kristina M. Barkum, 2021. "Global monitoring of the impact of the COVID-19 pandemic through online surveys sampled from the Facebook user base," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(51), pages 2111455118-, December.
    14. Meager, Rachael, 2019. "Understanding the average impact of microcredit expansions: a Bayesian hierarchical analysis of seven randomized experiments," LSE Research Online Documents on Economics 88190, London School of Economics and Political Science, LSE Library.
    15. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, November.
    16. Hengchen Dai & Silvia Saccardo & Maria A. Han & Lily Roh & Naveen Raja & Sitaram Vangala & Hardikkumar Modi & Shital Pandya & Michael Sloyan & Daniel M. Croymans, 2021. "Behavioural nudges increase COVID-19 vaccinations," Nature, Nature, vol. 597(7876), pages 404-409, September.
    17. Randall A. Lewis & Justin M. Rao, 2015. "The Unfavorable Economics of Measuring the Returns to Advertising," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(4), pages 1941-1973.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jose Maria Barrero & Nicholas Bloom & Steven J. Davis, 2023. "Long Social Distancing," Journal of Labor Economics, University of Chicago Press, vol. 41(S1), pages 129-172.
    2. Matilde Giaccherini & Joanna Kopinska & Gabriele Rovigatti, 2022. "Vax Populi: The Social Costs of Online Vaccine Skepticism," CESifo Working Paper Series 10184, CESifo.
    3. Bei Yan & Feng Mai & Chaojiang Wu & Rui Chen & Xiaolin Li, 2024. "A Computational Framework for Understanding Firm Communication During Disasters," Information Systems Research, INFORMS, vol. 35(2), pages 590-608, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Susan Athey & Kristen Grabarz & Michael Luca & Nils Wernerfelt, 2022. "The Effectiveness of Digital Interventions on COVID-19 Attitudes and Beliefs," Papers 2206.10214, arXiv.org.
    2. Weijia Dai & Hyunjin Kim & Michael Luca, 2023. "Frontiers: Which Firms Gain from Digital Advertising? Evidence from a Field Experiment," Marketing Science, INFORMS, vol. 42(3), pages 429-439, May.
    3. Garrett Johnson & Julian Runge & Eric Seufert, 2022. "Privacy-Centric Digital Advertising: Implications for Research," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 9(1), pages 49-54, June.
    4. Stefano DellaVigna & Elizabeth Linos, 2022. "RCTs to Scale: Comprehensive Evidence From Two Nudge Units," Econometrica, Econometric Society, vol. 90(1), pages 81-116, January.
    5. Randall Lewis & Dan Nguyen, 2015. "Display advertising’s competitive spillovers to consumer search," Quantitative Marketing and Economics (QME), Springer, vol. 13(2), pages 93-115, June.
    6. George Z. Gui, 2024. "Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments," Marketing Science, INFORMS, vol. 43(2), pages 378-391, March.
    7. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," Working Papers hal-03455978, HAL.
    8. Pietro Emilio Spini, 2021. "Robustness, Heterogeneous Treatment Effects and Covariate Shifts," Papers 2112.09259, arXiv.org, revised Aug 2024.
    9. Ron Berman & Christophe Van den Bulte, 2022. "False Discovery in A/B Testing," Management Science, INFORMS, vol. 68(9), pages 6762-6782, September.
    10. Ron Berman & Elea McDonnell Feit, 2024. "Latent Stratification for Incrementality Experiments," Marketing Science, INFORMS, vol. 43(4), pages 903-917, July.
    11. Bo, Hao & Galiani, Sebastian, 2021. "Assessing external validity," Research in Economics, Elsevier, vol. 75(3), pages 274-285.
    12. George Z. Gui, 2020. "Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments," Papers 2010.05117, arXiv.org, revised Dec 2023.
    13. Isaiah Andrews & Drew Fudenberg & Lihua Lei & Annie Liang & Chaofeng Wu, 2022. "The Transfer Performance of Economic Models," Papers 2202.04796, arXiv.org, revised Jul 2024.
    14. Ron Berman & Yuval Heller, 2020. "Naive analytics equilibrium," Papers 2010.15810, arXiv.org, revised Apr 2021.
    15. Islam, Asad & Kusnadi, Gita & Rezki, Jahen & Sim, Armand & van Empel, Giovanni & Vlassopoulos, Michael & Zenou, Yves, 2024. "Addressing vaccine hesitancy using local ambassadors: A randomized controlled trial in Indonesia," European Economic Review, Elsevier, vol. 163(C).
    16. Paul B. Ellickson & Wreetabrata Kar & James C. Reeder, 2023. "Estimating Marketing Component Effects: Double Machine Learning from Targeted Digital Promotions," Marketing Science, INFORMS, vol. 42(4), pages 704-728, July.
    17. Weijia Dai & Hyunjin Kim & Michael Luca, 2016. "Which Firms Gain from Digital Advertising? Evidence from a Field Experiment," Harvard Business School Working Papers 17-025, Harvard Business School, revised Jan 2023.
    18. Diane Pelly & Orla Doyle, 2022. "Nudging in the workplace: increasing participation in employee EDI wellness events," Working Papers 202208, Geary Institute, University College Dublin.
    19. Brett R. Gordon & Robert Moakler & Florian Zettelmeyer, 2023. "Predictive Incrementality by Experimentation (PIE) for Ad Measurement," Papers 2304.06828, arXiv.org.
    20. Neckermann, Susanne & Turmunkh, Uyanga & van Dolder, Dennie & Wang, Tong V., 2022. "Nudging student participation in online evaluations of teaching: Evidence from a field experiment," European Economic Review, Elsevier, vol. 141(C).

    More about this item

    JEL classification:

    • D0 - Microeconomics - - General
    • I0 - Health, Education, and Welfare - - General
    • M0 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:30273. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.