The Necessity of Construct and External Validity for Generalized Causal Claims
Author
Abstract
Suggested Citation
DOI: 10.31219/osf.io/2s8w5
Download full text from publisher
References listed on IDEAS
- Keele, Luke & Minozzi, William, 2013. "How Much Is Minnesota Like Wisconsin? Assumptions and Counterfactuals in Causal Inference with Observational Data," Political Analysis, Cambridge University Press, vol. 21(2), pages 193-216, April.
- Guido W. Imbens, 2020.
"Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics,"
Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
- Guido Imbens, 2019. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," NBER Working Papers 26104, National Bureau of Economic Research, Inc.
- Joshua D. Angrist & Jörn-Steffen Pischke, 2010.
"The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics,"
Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
- Joshua Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics," NBER Working Papers 15794, National Bureau of Economic Research, Inc.
- Angrist, Joshua & Pischke, Jörn-Steffen, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," IZA Discussion Papers 4800, Institute of Labor Economics (IZA).
- Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics," RatSWD Working Papers 142, German Data Forum (RatSWD).
- Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design is taking the Con out of Econometrics," CEP Discussion Papers dp0976, Centre for Economic Performance, LSE.
- Angrist, Joshua D. & Pischke, Jörn-Steffen, 2010. "The credibility revolution in empirical economics: how better research design is taking the con out of econometrics," LSE Research Online Documents on Economics 48898, London School of Economics and Political Science, LSE Library.
- Angus Deaton, 2010.
"Instruments, Randomization, and Learning about Development,"
Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
- Angus Deaton, 2010. "Instruments, randomization, and learning about development," Working Papers 1224, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
- Robert B. Olsen & Larry L. Orr & Stephen H. Bell & Elizabeth A. Stuart, 2013. "External Validity in Policy Evaluations That Choose Sites Purposively," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 32(1), pages 107-121, January.
- Andrew Gelman & Guido Imbens, 2013. "Why ask Why? Forward Causal Inference and Reverse Causal Questions," NBER Working Papers 19614, National Bureau of Economic Research, Inc.
- Thomas D. Cook, 2014. "Generalizing Causal Knowledge In The Policy Sciences: External Validity As A Task Of Both Multiattribute Representation And Multiattribute Extrapolation," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(2), pages 527-536, March.
- Hunt Allcott, 2015. "Site Selection Bias in Program Evaluation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(3), pages 1117-1165.
- Eva Vivalt, 0. "How Much Can We Generalize From Impact Evaluations?," Journal of the European Economic Association, European Economic Association, vol. 18(6), pages 3045-3089.
- Maureen A. Pirog, 2014. "Internal Versus External Validity: Where Are Policy Analysts Going?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(2), pages 548-550, March.
- Humphreys, Macartan & Scacco, Alexandra, 2020. "The aggregation challenge," World Development, Elsevier, vol. 127(C).
- Adcock, Robert & Collier, David, 2001. "Measurement Validity: A Shared Standard for Qualitative and Quantitative Research," American Political Science Review, Cambridge University Press, vol. 95(3), pages 529-546, September.
- Jörg Peters & Jörg Langbein & Gareth Roberts, 2018.
"Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity,"
The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
- Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2017. "Generalization in the Tropics: Development policy, randomized controlled trials, and external validity," Ruhr Economic Papers 716, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Deaton, Angus & Cartwright, Nancy, 2018.
"Understanding and misunderstanding randomized controlled trials,"
Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
- Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," Working Papers august_25.pdf, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
- Angus Deaton & Nancy Cartwright, 2017. "Understanding and misunderstanding randomized controlled trials," Working Papers 2017-10, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
- Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," NBER Working Papers 22595, National Bureau of Economic Research, Inc.
- Humphreys, Macartan & Scacco, Alexandra, 2020. "The aggregation challenge," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 127, pages 1-3.
- Gerber, Alan S. & Green, Donald P., 2000. "The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment," American Political Science Review, Cambridge University Press, vol. 94(3), pages 653-663, September.
- Abhijit V. Banerjee & Esther Duflo, 2009.
"The Experimental Approach to Development Economics,"
Annual Review of Economics, Annual Reviews, vol. 1(1), pages 151-178, May.
- Abhijit V. Banerjee & Esther Duflo, 2008. "The Experimental Approach to Development Economics," NBER Working Papers 14467, National Bureau of Economic Research, Inc.
- Banerjee, Abhijit & Duflo, Esther, 2008. "The Experimental Approach to Development Economics," CEPR Discussion Papers 7037, C.E.P.R. Discussion Papers.
- Elizabeth A. Stuart & Stephen R. Cole & Catherine P. Bradshaw & Philip J. Leaf, 2011. "The use of propensity scores to assess the generalizability of results from randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 369-386, April.
- Joshua D. Angrist & Jörn-Steffen Pischke, 2015. "The path from cause to effect: mastering 'metrics," CentrePiece - The magazine for economic performance 442, Centre for Economic Performance, LSE.
- Eva Vivalt, 2020. "How Much Can We Generalize From Impact Evaluations?," Journal of the European Economic Association, European Economic Association, vol. 18(6), pages 3045-3089.
- Dafoe, Allan & Zhang, Baobao & Caughey, Devin, 2018. "Information Equivalence in Survey Experiments," Political Analysis, Cambridge University Press, vol. 26(4), pages 399-416, October.
- Joseph Henrich & Steve J. Heine & Ara Norenzayan, 2010. "The Weirdest People in the World?," RatSWD Working Papers 139, German Data Forum (RatSWD).
- Gerber, Alan S. & Green, Donald P. & Larimer, Christopher W., 2008. "Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment," American Political Science Review, Cambridge University Press, vol. 102(1), pages 33-48, February.
- Seán M. Muller, 2015. "Causal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations," The World Bank Economic Review, World Bank, vol. 29(suppl_1), pages 217-225.
- Guala,Francesco, 2005.
"The Methodology of Experimental Economics,"
Cambridge Books,
Cambridge University Press, number 9780521618618, November.
- Guala,Francesco, 2005. "The Methodology of Experimental Economics," Cambridge Books, Cambridge University Press, number 9780521853408, November.
- Lant Pritchett, Justin Sandefur, 2013. "Context Matters for Size: Why External Validity Claims and Development Practice Don't Mix-Working Paper 336," Working Papers 336, Center for Global Development.
- Lant Pritchett & Justin Sandefur, 2015. "Learning from Experiments When Context Matters," American Economic Review, American Economic Association, vol. 105(5), pages 471-475, May.
- Michael J. Weiss & Howard S. Bloom & Thomas Brock, 2014. "A Conceptual Framework For Studying The Sources Of Variation In Program Effects," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(3), pages 778-808, June.
- Alan Gerber & Donald Green, 2000. "The effects of canvassing, direct mail, and telephone contact on voter turnout: A field experiment," Natural Field Experiments 00248, The Field Experiments Website.
- Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, November.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Brick, Kerri & De Martino, Samantha & Visser, Martine, 2023.
"Behavioural nudges for water conservation in unequal settings: Experimental evidence from Cape Town,"
Journal of Environmental Economics and Management, Elsevier, vol. 121(C).
- Brick, Kerri & DeMartino, Samantha & Visser, Martine, 2019. "Behavioural Nudges for Water Conservation in Unequal Settings Experimental Evidence from Cape Town," EfD Discussion Paper 19-19, Environment for Development, University of Gothenburg.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Esterling, Kevin M. & Brady, David & Schwitzgebel, Eric, 2023. "The Necessity of Construct and External Validity for Generalized Causal Claims," I4R Discussion Paper Series 18, The Institute for Replication (I4R).
- Jörg Peters & Jörg Langbein & Gareth Roberts, 2018.
"Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity,"
The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
- Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2017. "Generalization in the Tropics: Development policy, randomized controlled trials, and external validity," Ruhr Economic Papers 716, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Denis Fougère & Nicolas Jacquemet, 2020.
"Policy Evaluation Using Causal Inference Methods,"
Working Papers
hal-03455978, HAL.
- Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) hal-03455978, HAL.
- Denis Fougère & Nicolas Jacquemet, 2021. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03098058, HAL.
- Fougère, Denis & Jacquemet, Nicolas, 2020. "Policy Evaluation Using Causal Inference Methods," IZA Discussion Papers 12922, Institute of Labor Economics (IZA).
- Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
- Denis Fougère & Nicolas Jacquemet, 2021. "Policy Evaluation Using Causal Inference Methods," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) hal-03098058, HAL.
- Denis Fougère & Nicolas Jacquemet, 2021. "Policy Evaluation Using Causal Inference Methods," PSE-Ecole d'économie de Paris (Postprint) hal-03098058, HAL.
- Denis Fougère & Nicolas Jacquemet, 2021. "Policy Evaluation Using Causal Inference Methods," Post-Print hal-03098058, HAL.
- Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019.
"All that Glitters is not Gold. The Political Economy of Randomized Evaluations in Development,"
Development and Change, International Institute of Social Studies, vol. 50(3), pages 735-762, May.
- Florent BEDECARRATS & Isabelle GUERIN & François ROUBAUD, 2017. "All That Glitters Is Not Gold: The Political Economy of Randomised Evaluations in Development," Working Paper 753120cd-506f-4c5f-80ed-7, Agence française de développement.
- Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019. "All that glitters is not gold : the political economy of randomized evaluations in development," Post-Print ird-02112849, HAL.
- Deaton, Angus & Cartwright, Nancy, 2018.
"Understanding and misunderstanding randomized controlled trials,"
Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
- Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," NBER Working Papers 22595, National Bureau of Economic Research, Inc.
- Angus Deaton & Nancy Cartwright, 2017. "Understanding and misunderstanding randomized controlled trials," Working Papers 2017-10, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
- Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," Working Papers august_25.pdf, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
- Ashis Das & Jed Friedman & Eeshani Kandpal, 2018. "Does involvement of local NGOs enhance public service delivery? Cautionary evidence from a malaria‐prevention program in India," Health Economics, John Wiley & Sons, Ltd., vol. 27(1), pages 172-188, January.
- Andor, Mark A. & Gerster, Andreas & Peters, Jörg & Schmidt, Christoph M., 2020.
"Social Norms and Energy Conservation Beyond the US,"
Journal of Environmental Economics and Management, Elsevier, vol. 103(C).
- Andor, Mark Andreas & Gerster, Andreas & Peters, Jörg & Schmidt, Christoph M., 2017. "Social norms and energy conservation beyond the US," Ruhr Economic Papers 714, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Andor, Mark A. & Gerster, Andreas & Peters, Jörg & Schmidt, Christoph M., 2018. "Social Norms and Energy Conservation Beyond the US," EfD Discussion Paper 18-16, Environment for Development, University of Gothenburg.
- Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017.
"From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application,"
Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
- Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan, 2016. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Working Papers id:11516, eSocialSciences.
- Banerjee, Abhijit V. & Banerji, Rukmini & Berry, James & Duflo, Esther & Kannan, Harini & Mukerji, Shobhini & Shotland, Marc & Walton, Michael, 2016. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Working Paper Series rwp16-057, Harvard University, John F. Kennedy School of Government.
- Duflo, Esther & Banerjee, Abhijit & Banerji, Rukmini & Berry, James & Kannan, Harini & Mukerji, Shobhini & Walton, Michael & Shotland, Marc, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," CEPR Discussion Papers 11762, C.E.P.R. Discussion Papers.
- Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukherji & Marc Shotland & Michael Walton, 2016. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," NBER Working Papers 22931, National Bureau of Economic Research, Inc.
- Cristina Corduneanu-Huci & Michael T. Dorsch & Paul Maarek, 2017. "Learning to constrain: Political competition and randomized controlled trials in development," THEMA Working Papers 2017-24, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
- Guido W. Imbens, 2020.
"Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics,"
Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
- Guido Imbens, 2019. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," NBER Working Papers 26104, National Bureau of Economic Research, Inc.
- Masselus, Lise & Petrik, Christina & Ankel-Peters, Jörg, 2024.
"Lost in the design space? Construct validity in the microfinance literature,"
Ruhr Economic Papers
1097, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Masselus, Lise & Petrik, Christina & Ankel-Peters, Jörg, 2024. "Lost in the Design Space? Construct Validity in the Microfinance Literature," I4R Discussion Paper Series 184, The Institute for Replication (I4R).
- Masselus, Lise & Petrik, Christina & Ankel-Peters, Jörg, 2024. "Lost in the Design Space? Construct Validity in the Microfinance Literature," OSF Preprints nwp8k, Center for Open Science.
- Faraz Usmani & Marc Jeuland & Subhrendu K. Pattanayak, 2024.
"NGOs and the Effectiveness of Interventions,"
The Review of Economics and Statistics, MIT Press, vol. 106(6), pages 1690-1708, November.
- Faraz Usmani & Marc Jeuland & Subhrendu K. Pattanayak, 2018. "NGOs and the effectiveness of interventions," WIDER Working Paper Series wp-2018-59, World Institute for Development Economic Research (UNU-WIDER).
- Usmani, Faraz & Jeuland, Marc & Pattanayak, Subhrendu, 2021. "NGOs and the effectiveness of interventions," Ruhr Economic Papers 902, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Corduneanu-Huci, Cristina & Dorsch, Michael T. & Maarek, Paul, 2021.
"The politics of experimentation: Political competition and randomized controlled trials,"
Journal of Comparative Economics, Elsevier, vol. 49(1), pages 1-21.
- Cristina Corduneanu-Huci & Michael T. Dorsch & Paul Maarek, 2021. "The politics of experimentation: Political competition and randomized controlled trials," Post-Print hal-04120428, HAL.
- Andrews, Isaiah & Oster, Emily, 2019. "A simple approximation for evaluating external validity bias," Economics Letters, Elsevier, vol. 178(C), pages 58-62.
- Alex Eble & Peter Boone & Diana Elbourne, 2017.
"On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics,"
The World Bank Economic Review, World Bank, vol. 31(3), pages 687-707.
- Eble,Alex & Boone,Peter & Elbourne,Diana, 2016. "On minimizing the risk of bias in randomized controlled trials in economics," Policy Research Working Paper Series 7746, The World Bank.
- Naoki Egami & Erin Hartman, 2021. "Covariate selection for generalizing experimental results: Application to a large‐scale development program in Uganda," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(4), pages 1524-1548, October.
- Omar Al-Ubaydli & John List & Claire Mackevicius & Min Sok Lee & Dana Suskind, 2019. "How Can Experiments Play a Greater Role in Public Policy? 12 Proposals from an Economic Model of Scaling," Artefactual Field Experiments 00679, The Field Experiments Website.
- Stefano DellaVigna & Elizabeth Linos, 2022.
"RCTs to Scale: Comprehensive Evidence From Two Nudge Units,"
Econometrica, Econometric Society, vol. 90(1), pages 81-116, January.
- Stefano DellaVigna & Elizabeth Linos, 2020. "RCTs to Scale: Comprehensive Evidence from Two Nudge Units," NBER Working Papers 27594, National Bureau of Economic Research, Inc.
- Bayer, Patrick & Kennedy, Ryan & Yang, Joonseok & Urpelainen, Johannes, 2020. "The need for impact evaluation in electricity access research," Energy Policy, Elsevier, vol. 137(C).
- Jeffrey Smith & Arthur Sweetman, 2016.
"Viewpoint: Estimating the causal effects of policies and programs,"
Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
- Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 49(3), pages 871-905, August.
- Smith, Jeffrey A. & Sweetman, Arthur, 2016. "Viewpoint: Estimating the Causal Effects of Policies and Programs," IZA Discussion Papers 10108, Institute of Labor Economics (IZA).
Corrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:2s8w5. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.