IDEAS home Printed from https://ideas.repec.org/a/sae/anname/v628y2010i1p132-147.html
   My bibliography  Save this article

Theory, External Validity, and Experimental Inference: Some Conjectures

Author

Listed:
  • Fernando Martel Garcia

    (New York University's Wilf Family Department of Politics, (fmg229@nyu.edu)

  • Leonard Wantchekon

    (New York University, (leonard.wantchekon@nyu.edu)

Abstract

It is often argued that experiments are strong on causal identification (internal validity) but weak on generalizability (external validity). One widely accepted way to limit threats to external validity is to incorporate as much variation in the background conditions and in the covariates as possible through replication. Another strategy is to make the theoretical foundations of the experiment more explicit. The latter requires that we develop trajectories of experiments that are consistent with a theoretical argument. In other words, new experiments should not simply consist of changing the context of old ones, but do so in ways that explicitly test various aspects of a theory in a coherent way.

Suggested Citation

  • Fernando Martel Garcia & Leonard Wantchekon, 2010. "Theory, External Validity, and Experimental Inference: Some Conjectures," The ANNALS of the American Academy of Political and Social Science, , vol. 628(1), pages 132-147, March.
    • Handle: RePEc:sae:anname:v:628:y:2010:i:1:p:132-147
    DOI: 10.1177/0002716209351519
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0002716209351519
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0002716209351519?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Michelle Morais de Sa e Silva, 2008. "Opportunity NYC: a Performance-Based conditional Cash Transfer Programme. A Qualitative Analysis," Working Papers 49, International Policy Centre for Inclusive Growth.
    2. Leonard Wantchekon, 2003. "Clientelism and voting behavior: Evidence from a field experiment in benin," Natural Field Experiments 00339, The Field Experiments Website.
    3. Mahoney, James & Goertz, Gary, 2006. "A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research," Political Analysis, Cambridge University Press, vol. 14(3), pages 227-249, July.
    4. James J. Heckman, 2008. "Econometric Causality," International Statistical Review, International Statistical Institute, vol. 76(1), pages 1-27, April.
    5. Francesco Guala & Luigi Mittone, 2005. "Experiments in economics: External validity and the robustness of phenomena," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(4), pages 495-515.
    6. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
    7. Gerber, Alan S. & Green, Donald P., 2000. "The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment," American Political Science Review, Cambridge University Press, vol. 94(3), pages 653-663, September.
    8. Alan Gerber, 2004. "Does campaign spending work?," Natural Field Experiments 00246, The Field Experiments Website.
    9. Granger, Clive W.J. & Machina, Mark J., 2006. "Forecasting and Decision Theory," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 2, pages 81-98, Elsevier.
    10. Druckman, James N. & Green, Donald P. & Kuklinski, James H. & Lupia, Arthur, 2006. "The Growth and Development of Experimental Research in Political Science," American Political Science Review, Cambridge University Press, vol. 100(4), pages 627-635, November.
    11. Arthur Schram, 2005. "Artificiality: The tension between internal and external validity in economic experiments," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 225-237.
    12. Laura B. Rawlings, 2005. "Evaluating the Impact of Conditional Cash Transfer Programs," The World Bank Research Observer, World Bank, vol. 20(1), pages 29-55.
    13. Yusaku Horiuchi & Kosuke Imai & Naoko Taniguchi, 2007. "Designing and Analyzing Randomized Experiments: Application to a Japanese Election Survey Experiment," American Journal of Political Science, John Wiley & Sons, vol. 51(3), pages 669-687, July.
    14. Alan Gerber & Donald Green, 2000. "The effects of canvassing, direct mail, and telephone contact on voter turnout: A field experiment," Natural Field Experiments 00248, The Field Experiments Website.
    15. Steven D. Levitt & John A. List, 2007. "What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?," Journal of Economic Perspectives, American Economic Association, vol. 21(2), pages 153-174, Spring.
    16. James J. Heckman, 2008. "Causalidad econométrica," Monetaria, CEMLA, vol. 0(3), pages 291-338, julio-sep.
    17. McDermott, Rose, 2002. "Experimental Methodology in Political Science," Political Analysis, Cambridge University Press, vol. 10(4), pages 325-342.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Keenan, Michael & Fort, Ricardo & Vargas, Ricardo, 2024. "Shocked into side-selling? Production shocks and organic coffee farmers’ marketing decisions," Food Policy, Elsevier, vol. 125(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gerry Stoker, 2010. "Exploring the Promise of Experimentation in Political Science: Micro‐Foundational Insights and Policy Relevance," Political Studies, Political Studies Association, vol. 58(2), pages 300-319, March.
    2. Benno Torgler, 2022. "The power of public choice in law and economics," Journal of Economic Surveys, Wiley Blackwell, vol. 36(5), pages 1410-1453, December.
    3. Olken, Benjamin A., 2010. "Direct Democracy and Local Public Goods: Evidence from a Field Experiment in Indonesia," American Political Science Review, Cambridge University Press, vol. 104(2), pages 243-267, May.
    4. KAMKOUM, Arnaud Cedric, 2023. "The Federal Reserve’s Response to the Global Financial Crisis and its Effects: An Interrupted Time-Series Analysis of the Impact of its Quantitative Easing Programs," Thesis Commons d7pvg, Center for Open Science.
    5. Julia Cage & Edgard Dewitte, 2021. "It Takes Money to Make MPs: Evidence from 150 Years of British Campaign Spending," SciencePo Working papers Main hal-03384143, HAL.
    6. Grácio, Matilde & Vicente, Pedro C., 2021. "Information, get-out-the-vote messages, and peer influence: Causal effects on political behavior in Mozambique," Journal of Development Economics, Elsevier, vol. 151(C).
    7. Julia Cage & Edgard Dewitte, 2021. "It Takes Money to Make MPs: Evidence from 150 Years of British Campaign Spending," SciencePo Working papers hal-03384143, HAL.
    8. Julia Cage & Yasmine Bekkouche, 2018. "The Price of a Vote: Evidence from France, 1993-2014," Working Papers hal-03393149, HAL.
    9. Bekkouche, Yasmine & Cagé, Julia & Dewitte, Edgard, 2022. "The heterogeneous price of a vote: Evidence from multiparty systems, 1993–2017," Journal of Public Economics, Elsevier, vol. 206(C).
    10. repec:hal:spmain:info:hdl:2441/1dp7827s4n8ht8fk3qhmeuvd0o is not listed on IDEAS
    11. Rogers, Todd T & Middleton, Joel A., 2012. "Are Ballot Initiative Outcomes Influenced by the Campaigns of Independent Groups? A Precinct-Randomized Field Experiment," Scholarly Articles 9830357, Harvard Kennedy School of Government.
    12. Xavier Giné & Ghazala Mansuri, 2018. "Together We Will: Experimental Evidence on Female Voting Behavior in Pakistan," American Economic Journal: Applied Economics, American Economic Association, vol. 10(1), pages 207-235, January.
    13. Little, Andrew T., 2022. "Bayesian Explanations for Persuasion," OSF Preprints ygw8e, Center for Open Science.
    14. Bratton, Michael, 2013. "Measuring Government Performance in Public Opinion Surveys in Africa: Towards Experiments?," WIDER Working Paper Series 023, World Institute for Development Economic Research (UNU-WIDER).
    15. Kosuke Imai, 2005. "Do get-out-the-vote calls reduce turnout? The importance of statistical methods for field experiments," Natural Field Experiments 00272, The Field Experiments Website.
    16. Bekkouche, Yasmine & Cagé, Julia & Dewitte, Edgard, 2022. "The heterogeneous price of a vote: Evidence from multiparty systems, 1993–2017," Journal of Public Economics, Elsevier, vol. 206(C).
    17. Yasmine Bekkouche & Julia Cage, 2019. "The Heterogeneous Price of a Vote: Evidence from France, 1993-2014," SciencePo Working papers Main hal-03393084, HAL.
    18. Chad Kendall & Tommaso Nannicini & Francesco Trebbi, 2015. "How Do Voters Respond to Information? Evidence from a Randomized Campaign," American Economic Review, American Economic Association, vol. 105(1), pages 322-353, January.
    19. repec:spo:wpecon:info:hdl:2441/2ahul47tb09rvqfl9eelv7o5ca is not listed on IDEAS
    20. Pereira dos Santos, João & Tavares, José & Vicente, Pedro C., 2021. "Can ATMs get out the vote? Evidence from a nationwide field experiment," European Economic Review, Elsevier, vol. 134(C).
    21. Arnaud Cedric Kamkoum, 2023. "The Federal Reserve's Response to the Global Financial Crisis and Its Long-Term Impact: An Interrupted Time-Series Natural Experimental Analysis," Papers 2305.12318, arXiv.org.
    22. Jared Barton & Marco Castillo & Ragan Petrie, 2014. "What Persuades Voters? A Field Experiment on Political Campaigning," Economic Journal, Royal Economic Society, vol. 124(574), pages 293-326, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:anname:v:628:y:2010:i:1:p:132-147. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.