IDEAS home Printed from https://ideas.repec.org/a/oup/wbecrv/v29y2015isuppl_1ps217-s225..html
   My bibliography  Save this article

Causal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations

Author

Listed:
  • Seán M. Muller

Abstract

The ability to generalize effects estimated from randomized experiments is critical for their relevance to policy. Framing that problem in terms of causal interaction reveals the extent to which the literature to date has failed to adequately address external validity. An analogy with matching estimators illustrates the current inconsistency in approaches to estimating causal relationships and generalizing these estimates to other populations and contexts. Contrary to some claims, atheoretic replication is not a plausible solution. Better knowledge of, and more information on, interacting factors is required for credible, formal extrapolation. In the absence of that, modesty is recommended.

Suggested Citation

  • Seán M. Muller, 2015. "Causal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations," The World Bank Economic Review, World Bank, vol. 29(suppl_1), pages 217-225.
    • Handle: RePEc:oup:wbecrv:v:29:y:2015:i:suppl_1:p:s217-s225.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/wber/lhv027
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    2. Muller, Seán M., 2021. "The dangers of performative scientism as the alternative to anti-scientific policymaking: A critical, preliminary assessment of South Africa’s Covid-19 response and its consequences," World Development, Elsevier, vol. 140(C).
    3. Andor, Mark A. & Gerster, Andreas & Peters, Jörg & Schmidt, Christoph M., 2020. "Social Norms and Energy Conservation Beyond the US," Journal of Environmental Economics and Management, Elsevier, vol. 103(C).
    4. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    5. Tim Kaiser & Lukas Menkhoff, 2017. "Does Financial Education Impact Financial Literacy and Financial Behavior, and If So, When?," The World Bank Economic Review, World Bank, vol. 31(3), pages 611-630.
    6. Ankel-Peters, Jörg & Schmidt, Christoph M., 2023. "Rural electrification, the credibility revolution, and the limits of evidence-based policy," Ruhr Economic Papers 1051, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    7. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    8. Jeffrey Smith, 2022. "Treatment Effect Heterogeneity," Evaluation Review, , vol. 46(5), pages 652-677, October.
    9. Sebastian Galiani & Patrick J. McEwan & Brian Quistorff, 2017. "External and Internal Validity of a Geographic Quasi-Experiment Embedded in a Cluster-Randomized Experiment," Advances in Econometrics, in: Regression Discontinuity Designs, volume 38, pages 195-236, Emerald Group Publishing Limited.
    10. Esterling, Kevin & Brady, David & Schwitzgebel, Eric, 2021. "The Necessity of Construct and External Validity for Generalized Causal Claims," OSF Preprints 2s8w5, Center for Open Science.
    11. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    12. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    13. Petr Matous, 2023. "Male and stale? Questioning the role of “opinion leaders” in agricultural programs," Agriculture and Human Values, Springer;The Agriculture, Food, & Human Values Society (AFHVS), vol. 40(3), pages 1205-1220, September.
    14. Seán M. Muller, 2021. "Evidence for a YETI? A Cautionary Tale from South Africa's Youth Employment Tax Incentive," Development and Change, International Institute of Social Studies, vol. 52(6), pages 1301-1342, November.
    15. Ihsaan BASSIER & Joshua BUDLENDER, 2024. "Methods for Credible Evaluation of Programme Stimulus Effects in South Africa," Working Paper 53315978-8164-44f9-8141-5, Agence française de développement.
    16. Muller, Seán M., 2020. "The implications of a fundamental contradiction in advocating randomized trials for policy," World Development, Elsevier, vol. 127(C).
    17. Hoffmann, Nimi, 2020. "Involuntary experiments in former colonies: The case for a moratorium," World Development, Elsevier, vol. 127(C).
    18. Alejandro Noriega & Alex Pentland, 2020. "Representativity and Networked Interference in Data-Rich Field Experiments: A Large-Scale RCT in Rural Mexico," The World Bank Economic Review, World Bank, vol. 34(Supplemen), pages 35-39.
    19. Tim Kaiser & Lukas Menkhoff, 2018. "Active Learning Fosters Financial Behavior: Experimental Evidence," Discussion Papers of DIW Berlin 1743, DIW Berlin, German Institute for Economic Research.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:wbecrv:v:29:y:2015:i:suppl_1:p:s217-s225.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://edirc.repec.org/data/wrldbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.