IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/k5uap.html
   My bibliography  Save this paper

Scaling Behavioral Interventions in the Presence of Spillover

Author

Listed:
  • Ternovski, John
  • Keppeler, Florian
  • Jilke, Sebastian

    (Georgetown University)

  • Vogel, Dominik

    (University of Hamburg)

Abstract

Randomized Controlled Trials (RCTs) are increasingly relied upon by policymakers as part of efforts to incorporate evidence into the policymaking process, a movement known as evidence-based policymaking, or EBPM. Testing possible policy interventions via RCTs before full rollout is commonly thought to be the gold standard of evidence in the EBPM process. However, real-world policy changes do not always scale up as expected. Even large-N RCTs targeting a random sample of policy beneficiaries do not capture the influence of social networks and risk missing consequential spillover effects. We illustrate this issue by assessing the efficacy of monetary incentives to increase COVID-19 vaccination in an RCT over the entire population of a medium-sized European town (~40,000 residents). We use administrative vaccination data as our primary outcome. Since the entire population was randomized, we are able to estimate spillover effects within households. There were significant negative spillover effects on booster vaccinations that we attribute to a displacement effect, potentially driven by long lines at the vaccination events. Our results illustrate that using a population-level RCT to test whether a policy scales can help avoid costly, ineffective, or even counterproductive policy outcomes.

Suggested Citation

  • Ternovski, John & Keppeler, Florian & Jilke, Sebastian & Vogel, Dominik, 2023. "Scaling Behavioral Interventions in the Presence of Spillover," OSF Preprints k5uap, Center for Open Science.
    • Handle: RePEc:osf:osfxxx:k5uap
    DOI: 10.31219/osf.io/k5uap
    as

    Download full text from publisher

    File URL: https://osf.io/download/653aa37528274513f5b864ed/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/k5uap?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Michael McGann & Emma Blomkamp & Jenny M. Lewis, 2018. "The rise of public sector innovation labs: experiments in design thinking for policy," Policy Sciences, Springer;Society of Policy Sciences, vol. 51(3), pages 249-267, September.
    2. Jon Baron, 2018. "A Brief History of Evidence-Based Policy," The ANNALS of the American Academy of Political and Social Science, , vol. 678(1), pages 40-50, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kawther Saeedi & Anna Visvizi & Dimah Alahmadi & Amal Babour, 2023. "Smart Cities and Households’ Recyclable Waste Management: The Case of Jeddah," Sustainability, MDPI, vol. 15(8), pages 1-23, April.
    2. Schiuma, Giovanni & Santarsiero, Francesco, 2023. "Innovation labs as organisational catalysts for innovation capacity development: A systematic literature review," Technovation, Elsevier, vol. 123(C).
    3. Ryan Anders Whitney & David López-García, 2023. "Fast-track institutionalization: The opening of urban planning best practice agencies in Mexico City," Environment and Planning C, , vol. 41(3), pages 600-616, May.
    4. Alfonso Unceta & Xabier Barandiaran & Natalia Restrepo, 2019. "The Role of Public Innovation Labs in Collaborative Governance—The Case of the Gipuzkoa Lab in the Basque Country, Spain," Sustainability, MDPI, vol. 11(21), pages 1-16, November.
    5. Alessandro Pollini & Alessandro Caforio, 2021. "Participation and Iterative Experiments: Designing Alternative Futures with Migrants and Service Providers," Social Sciences, MDPI, vol. 10(10), pages 1-13, September.
    6. Vasco Fonseca & Joaquim Caeiro & Fernanda Nogueira, 2021. "Social Model—Innovation and Behavioural Intervention as a Public Policy of Action within an Oncology and Loneliness Scope," Sustainability, MDPI, vol. 13(3), pages 1-7, February.
    7. Makoza, Frank, 2023. "Analyzing policy change of Malawi ICT and Digitalization policy: Policy Assemblage Perspective," EconStor Preprints 273309, ZBW - Leibniz Information Centre for Economics.
    8. Meister Broekema Peter & Bulder Elisabeth A. M. & Horlings Lummina G., 2022. "Same same, but different…? The emergence of Public Sector Innovation Labs in theory and practice," Management & Marketing, Sciendo, vol. 17(s1), pages 344-363, October.
    9. Vassallo, Jarrod P. & Banerjee, Sourindra & Zaman, Hasanuzzaman & Prabhu, Jaideep C., 2023. "Design thinking and public sector innovation: The divergent effects of risk-taking, cognitive empathy and emotional empathy on individual performance," Research Policy, Elsevier, vol. 52(6).
    10. Diana Pamela Villa Alvarez & Valentina Auricchio & Marzia Mortati, 2022. "Mapping design activities and methods of public sector innovation units through the policy cycle model," Policy Sciences, Springer;Society of Policy Sciences, vol. 55(1), pages 89-136, March.
    11. Strzałkowski, Andrzej, 2024. "Adaptation and operationalisation of sustainable degrowth for policy: Why we need to translate research papers into legislative drafts?," Ecological Economics, Elsevier, vol. 220(C).
    12. Simon Hull & Jennifer Whittal, 2021. "Do Design Science Research and Design Thinking Processes Improve the ‘Fit’ of the Fit-For-Purpose Approach to Securing Land Tenure for All in South Africa?," Land, MDPI, vol. 10(5), pages 1-26, May.
    13. Muñoz, Pablo & Dimov, Dimo, 2023. "A translational framework for entrepreneurship research," Journal of Business Venturing Insights, Elsevier, vol. 19(C).
    14. Lars Fuglsang & Anne Vorre Hansen & Ines Mergel & Maria Taivalsaari Røhnebæk, 2021. "Living Labs for Public Sector Innovation: An Integrative Literature Review," Administrative Sciences, MDPI, vol. 11(2), pages 1-19, June.
    15. Sojeong Kim & Adam M. Wellstead & Tanya Heikkila, 2023. "Policy capacity and rise of data‐based policy innovation labs," Review of Policy Research, Policy Studies Organization, vol. 40(3), pages 341-362, May.
    16. Yongjin Choi & Ashley M. Fox & Jennifer Dodge, 2022. "What counts? Policy evidence in public hearing testimonies: the case of single-payer healthcare in New York State," Policy Sciences, Springer;Society of Policy Sciences, vol. 55(4), pages 631-660, December.
    17. Nathalie Haug & Ines Mergel, 2021. "Public Value Co-Creation in Living Labs—Results from Three Case Studies," Administrative Sciences, MDPI, vol. 11(3), pages 1-22, July.
    18. Triyuth Promsiri & Krisakorn Sukavejworakit & Vasu Keerativutisest & Thanaphol Virasa & Krischanan Kampanthong, 2022. "Sustaining Thai Government Agency Innovation through Design Thinking Learning Effectiveness," Sustainability, MDPI, vol. 14(12), pages 1-13, June.
    19. Anamaria Vrabie & Rodica Ianole-Călin, 2020. "A Comparative Analysis of Municipal Public Innovation: Evidence from Romania and United States," JOItmC, MDPI, vol. 6(4), pages 1-21, October.
    20. Fuglsang, Lars & Hansen, Anne Vorre, 2022. "Framing improvements of public innovation in a living lab context: Processual learning, restrained space and democratic engagement," Research Policy, Elsevier, vol. 51(1).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:k5uap. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.