Addressing validity and generalizability concerns in field experiments
Author
Abstract
Suggested Citation
Download full text from publisher
Other versions of this item:
- Gerhard Riener & Sebastian O. Schneider & Valentin Wagner, 2020. "Addressing Validity and Generalizability Concerns in Field Experiments," Working Papers 2019, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz.
- Gerhard Riener & Sebastian Schneider & Valentin Wagner, 2020. "Addressing Validity and Generalizability Concerns in Field Experiments," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2020_16, Max Planck Institute for Research on Collective Goods.
References listed on IDEAS
- Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005.
"Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools,"
Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
- Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2000. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," NBER Working Papers 7831, National Bureau of Economic Research, Inc.
- Asako Ohinata & Jan C. van Ours, 2013.
"How Immigrant Children Affect the Academic Achievement of Native Dutch Children,"
Economic Journal, Royal Economic Society, vol. 0, pages 308-331, August.
- Ohinata, A. & van Ours, J.C., 2011. "How Immigrant Children Affect the Academic Achievement of Native Dutch Children," Other publications TiSEM 8064d408-d22c-4831-9605-8, Tilburg University, School of Economics and Management.
- Asako Ohinata & Jan C van Ours, 2012. "How immigrant children affect the academic achievement of native Dutch children," Norface Discussion Paper Series 2012012, Norface Research Programme on Migration, Department of Economics, University College London.
- Ohinata, A. & van Ours, J.C., 2011. "How Immigrant Children Affect the Academic Achievement of Native Dutch Children," Discussion Paper 2011-136, Tilburg University, Center for Economic Research.
- Ohinata, Asako & van Ours, Jan C., 2011. "How Immigrant Children Affect the Academic Achievement of Native Dutch Children," IZA Discussion Papers 6212, Institute of Labor Economics (IZA).
- Francesco Avvisati & Marc Gurgand & Nina Guyon & Eric Maurin, 2014.
"Getting Parents Involved: A Field Experiment in Deprived Schools,"
The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(1), pages 57-83.
- Maurin, Eric & Gurgand, Marc & Guyon, Nina & Avvisati, Francesco, 2010. "Getting Parents Involved: A Field Experiment in Deprived Schools," CEPR Discussion Papers 8020, C.E.P.R. Discussion Papers.
- Francesco Avvisati & Marc Gurgand & Nina Guyon & Eric Maurin, 2014. "Getting Parents Involved: A Field Experiment in Deprived Schools," Post-Print halshs-00942662, HAL.
- Francesco Avvisati & Marc Gurgand & Nina Guyon & Eric Maurin, 2014. "Getting Parents Involved: A Field Experiment in Deprived Schools," PSE-Ecole d'économie de Paris (Postprint) halshs-00942662, HAL.
- Omar Al-Ubaydli & John List & Dana Suskind, 2019.
"The science of using science: Towards an understanding of the threats to scaling experiments,"
Artefactual Field Experiments
00670, The Field Experiments Website.
- Omar Al-Ubaydli & John A. List & Dana Suskind, 2019. "The Science of Using Science: Towards an Understanding of the Threats to Scaling Experiments," NBER Working Papers 25848, National Bureau of Economic Research, Inc.
- Kraft, Matthew A. & Rogers, Todd, 2015.
"The underutilized potential of teacher-to-parent communication: Evidence from a field experiment,"
Economics of Education Review, Elsevier, vol. 47(C), pages 49-63.
- Kraft, Matthew A. & Rogers, Todd, 2015. "The Underutilized Potential of Teacher-to-Parent Communication: Evidence from a Field Experiment," Working Paper Series rwp14-049, Harvard University, John F. Kennedy School of Government.
- Miriam Bruhn & David McKenzie, 2009.
"In Pursuit of Balance: Randomization in Practice in Development Field Experiments,"
American Economic Journal: Applied Economics, American Economic Association, vol. 1(4), pages 200-232, October.
- Bruhn, Miriam & McKenzie, David, 2008. "In pursuit of balance : randomization in practice in development field experiments," Policy Research Working Paper Series 4752, The World Bank.
- Christoph Rothe, 2017.
"Robust Confidence Intervals for Average Treatment Effects Under Limited Overlap,"
Econometrica, Econometric Society, vol. 85, pages 645-660, March.
- Rothe, Christoph, 2015. "Robust Confidence Intervals for Average Treatment Effects under Limited Overlap," IZA Discussion Papers 8758, Institute of Labor Economics (IZA).
- Kasy, Maximilian, 2016.
"Why Experimenters Might Not Always Want to Randomize, and What They Could Do Instead,"
Political Analysis, Cambridge University Press, vol. 24(3), pages 324-338, July.
- Kasy, Maximilian, "undated". "Why Experimenters Might Not Always Want to Randomize, and What They Could Do Instead," Working Paper 36154, Harvard University OpenScholar.
- Gerhard Riener & Valentin Wagner, 2019. "On the design of non-monetary incentives in schools," Education Economics, Taylor & Francis Journals, vol. 27(3), pages 223-240, May.
- Fischer, Mira & Wagner, Valentin, 2018.
"Effects of timing and reference frame of feedback: Evidence from a field experiment,"
Discussion Papers, Research Unit: Market Behavior
SP II 2018-206, WZB Berlin Social Science Center.
- Fischer, Mira & Wagner, Valentin, 2018. "Effects of Timing and Reference Frame of Feedback: Evidence from a Field Experiment," IZA Discussion Papers 11970, Institute of Labor Economics (IZA).
- Mira Fischer & Valentin Wagner, 2018. "Effects of Timing and Reference Frame of Feedback: Evidence from a Field Experiment," Working Papers 1820, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz.
- Joseph Hotz, V. & Imbens, Guido W. & Mortimer, Julie H., 2005. "Predicting the efficacy of future training programs using past experiences at other locations," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 241-270.
- Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013.
"Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment,"
The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(2), pages 531-580.
- Crépon, Bruno & Duflo, Esther & Gurgand, Marc & Zamora, Philippe & Rathelot, Roland, 2012. "Do Labor Market Policies Have Displacement Effects? Evidence from a Clustered Randomized Experiment," CEPR Discussion Papers 9251, C.E.P.R. Discussion Papers.
- Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2012. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," Working Papers 2012-28, Center for Research in Economics and Statistics.
- Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," Post-Print halshs-00840901, HAL.
- Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2012. "Do Labor Market Policies Have Displacement Effects? Evidence from a Clustered Randomized Experiment," NBER Working Papers 18597, National Bureau of Economic Research, Inc.
- Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," PSE-Ecole d'économie de Paris (Postprint) halshs-00840901, HAL.
- Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," PSE - Labex "OSE-Ouvrir la Science Economique" halshs-00840901, HAL.
- Wagner, Valentin & Riener, Gerhard, 2015. "Peers or parents? On non-monetary incentives in schools," DICE Discussion Papers 203, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
- Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019.
"The Dozen Things Experimental Economists Should Do (More of),"
Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
- Eszter Czibor & David Jimenez-Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," NBER Working Papers 25451, National Bureau of Economic Research, Inc.
- Eszter Czibor & David Jimenez-Gomez & John List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Artefactual Field Experiments 00648, The Field Experiments Website.
- Francesco Avvisati & Marc Gurgand & Nina Guyon & Eric Maurin, 2014.
"Getting Parents Involved: A Field Experiment in Deprived Schools,"
The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(1), pages 57-83.
- Maurin, Eric & Gurgand, Marc & Guyon, Nina & Avvisati, Francesco, 2010. "Getting Parents Involved: A Field Experiment in Deprived Schools," CEPR Discussion Papers 8020, C.E.P.R. Discussion Papers.
- Francesco Avvisati & Marc Gurgand & Nina Guyon & Eric Maurin, 2014. "Getting Parents Involved: A Field Experiment in Deprived Schools," PSE - Labex "OSE-Ouvrir la Science Economique" halshs-00942662, HAL.
- Francesco Avvisati & Marc Gurgand & Nina Guyon & Eric Maurin, 2014. "Getting Parents Involved: A Field Experiment in Deprived Schools," PSE-Ecole d'économie de Paris (Postprint) halshs-00942662, HAL.
- Francesco Avvisati & Marc Gurgand & Nina Guyon & Eric Maurin, 2014. "Getting Parents Involved: A Field Experiment in Deprived Schools," Post-Print halshs-00942662, HAL.
- Johannes Abeler & Daniele Nosenzo, 2015. "Self-selection into laboratory experiments: pro-social motives versus monetary incentives," Experimental Economics, Springer;Economic Science Association, vol. 18(2), pages 195-214, June.
- Edward P. Lazear & Ulrike Malmendier & Roberto A. Weber, 2012. "Sorting in Experiments with Application to Social Preferences," American Economic Journal: Applied Economics, American Economic Association, vol. 4(1), pages 136-163, January.
- Jonathan Schulz & Uwe Sunde & Petra Thiemann & Christian Thoeni, 2019.
"Selection into Experiments: Evidence from a Population of Students,"
Discussion Papers
2019-09, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
- Schulz, Jonathan F. & Sunde, Uwe & Thiemann, Petra & Thöni, Christian, 2019. "Selection into Experiments: Evidence from a Population of Students," IZA Discussion Papers 12807, Institute of Labor Economics (IZA).
- Schulz, Jonathan & Sunde, Uwe & Thiemann, Petra & Thöni, Christian, 2019. "Selection into Experiments: Evidence from a Population of Students," Working Papers 2019:18, Lund University, Department of Economics.
- Ohinata, Asako & van Ours, Jan C., 2011.
"How Immigrant Children Affect the Academic Achievement of Native Dutch Children,"
IZA Discussion Papers
6212, Institute of Labor Economics (IZA).
- Asako Ohinata & Jan C van Ours, 2012. "How immigrant children affect the academic achievement of native Dutch children," Norface Discussion Paper Series 2012012, Norface Research Programme on Migration, Department of Economics, University College London.
- van Ours, Jan C. & Ohinata, Asako, 2011. "How immigrant children affect the academic achievement of native Dutch children," CEPR Discussion Papers 8718, C.E.P.R. Discussion Papers.
- Fischer, Mira & Wagner, Valentin, 2019. "Effects of Timing and Reference Frame of Feedback," Rationality and Competition Discussion Paper Series 150, CRC TRR 190 Rationality and Competition.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018.
"Nudging in education,"
Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
- Mette Trier Damgaard & Helena Skyt Nielsen, 2017. "Nudging in education," Economics Working Papers 2017-05, Department of Economics and Business Economics, Aarhus University.
- Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in Education," IZA Discussion Papers 11454, Institute of Labor Economics (IZA).
- Agostinelli, Francesco & Avitabile, Ciro & Bobba, Matteo, 2021.
"Enhancing Human Capital in Children: A Case Study on Scaling,"
TSE Working Papers
21-1196, Toulouse School of Economics (TSE), revised Oct 2023.
- Francesco Agostinelli & Ciro Avitabile & Matteo Bobba, 2023. "Enhancing Human Capital in Children: A Case Study on Scaling," NBER Working Papers 31407, National Bureau of Economic Research, Inc.
- Francesco Agostinelli & Ciro Avitable & Matteo Bobba, 2021. "Enhancing Human Capital in Children: A Case Study on Scaling," Working Papers 2021-012, Human Capital and Economic Opportunity Working Group.
- Agostinelli,Francesco & Avitabile,Ciro & Bobba,Matteo, 2021. "Enhancing Human Capital in Children : A Case Study on Scaling," Policy Research Working Paper Series 9647, The World Bank.
- Joana Elisa Maldonado & Kristof De Witte & Koen Declercq, 2022.
"The effects of parental involvement in homework: two randomised controlled trials in financial education,"
Empirical Economics, Springer, vol. 62(3), pages 1439-1464, March.
- Joana Elisa Maldonado & Kristof De Witte & Koen Declercq, 2019. "The effects of parental involvement in homework. Two randomised controlled trials in financial education," Working Papers of Department of Economics, Leuven 645427, KU Leuven, Faculty of Economics and Business (FEB), Department of Economics, Leuven.
- Joana Elisa Maldonado & Kristof De Witte & Koen Declercq, 2019. "The effects of parental involvement in homework. Two randomised controlled trials in financial education," Working Papers of LEER - Leuven Economics of Education Research 645427, KU Leuven, Faculty of Economics and Business (FEB), LEER - Leuven Economics of Education Research.
- Jeffrey Smith & Arthur Sweetman, 2016.
"Viewpoint: Estimating the causal effects of policies and programs,"
Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
- Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 49(3), pages 871-905, August.
- Smith, Jeffrey A. & Sweetman, Arthur, 2016. "Viewpoint: Estimating the Causal Effects of Policies and Programs," IZA Discussion Papers 10108, Institute of Labor Economics (IZA).
- Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019.
"The Dozen Things Experimental Economists Should Do (More of),"
Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
- Eszter Czibor & David Jimenez-Gomez & John List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Artefactual Field Experiments 00648, The Field Experiments Website.
- Eszter Czibor & David Jimenez-Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," NBER Working Papers 25451, National Bureau of Economic Research, Inc.
- Belot, Michèle & James, Jonathan, 2016.
"Partner selection into policy relevant field experiments,"
Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
- Michele Belot & Jonathan James, 2013. "Partner Selection into Policy Relevant Field Experiments," Edinburgh School of Economics Discussion Paper Series 236, Edinburgh School of Economics, University of Edinburgh.
- Belot, Michele & James, Jonathan, 2013. "Partner Selection into Policy Relevant Field Experiments," SIRE Discussion Papers 2013-112, Scottish Institute for Research in Economics (SIRE).
- Benjamin A. Olken, 2015. "Promises and Perils of Pre-analysis Plans," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 61-80, Summer.
- Luc Behaghel & Karen Macours & Julie Subervie, 2019.
"How can randomised controlled trials help improve the design of the common agricultural policy?,"
European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 46(3), pages 473-493.
- Luc Behaghel & Karen Macours & Julie Subervie, 2019. "How can randomised controlled trials help improve the design of the common agricultural policy?," Post-Print halshs-02297689, HAL.
- Luc Behaghel & Karen Macours & Julie Subervie, 2019. "How can randomised controlled trials help improve the design of the common agricultural policy?," PSE-Ecole d'économie de Paris (Postprint) halshs-02297689, HAL.
- Thiemann, Petra & Schulz, Jonathan & Sunde, Uwe & Thöni, Christian, 2022. "Selection into experiments: New evidence on the role of preferences, cognition, and recruitment protocols," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 98(C).
- Hunt Allcott, 2012. "Site Selection Bias in Program Evaluation," NBER Working Papers 18373, National Bureau of Economic Research, Inc.
- Luc Behaghel & Karen Macours & Julie Subervie, 2018.
"Can RCTs help improve the design of CAP,"
Working Papers
hal-01974425, HAL.
- Luc Behaghel & Karen Macours & Julie Subervie, 2018. "Can RCTs help improve the design of CAP," Post-Print hal-02112625, HAL.
- Luc Behaghel & Karen Macours & Julie Subervie, 2018. "Can RCTs help improve the design of CAP," CEE-M Working Papers hal-01974425, CEE-M, Universtiy of Montpellier, CNRS, INRA, Montpellier SupAgro.
- Abhijit Banerjee & Sylvain Chassang & Erik Snowberg, 2016. "Decision Theoretic Approaches to Experiment Design and External Validity," NBER Working Papers 22167, National Bureau of Economic Research, Inc.
- Denis Fougère & Nicolas Jacquemet, 2019.
"Causal Inference and Impact Evaluation,"
Economie et Statistique / Economics and Statistics, Institut National de la Statistique et des Etudes Economiques (INSEE), issue 510-511-5, pages 181-200.
- Denis Fougère & Nicolas Jacquemet, 2019. "Causal Inference and Impact Evaluation," SciencePo Working papers Main hal-02866828, HAL.
- Denis Fougère & Nicolas Jacquemet, 2019. "Causal Inference and Impact Evaluation," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) hal-02866828, HAL.
- Denis Fougère & Nicolas Jacquemet, 2019. "Causal Inference and Impact Evaluation," Post-Print hal-02866828, HAL.
- Denis Fougère & Nicolas Jacquemet, 2019. "Causal Inference and Impact Evaluation," PSE-Ecole d'économie de Paris (Postprint) hal-02866828, HAL.
- Asadul Islam, 2017. "Parental Involvement in Education: Evidence from Field Experiments in Developing Countries," Monash Economics Working Papers 02-17, Monash University, Department of Economics.
- Omar Al-Ubaydli & John List & Claire Mackevicius & Min Sok Lee & Dana Suskind, 2019. "How Can Experiments Play a Greater Role in Public Policy? 12 Proposals from an Economic Model of Scaling," Artefactual Field Experiments 00679, The Field Experiments Website.
- Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016.
"Policy evaluation, randomized controlled trials, and external validity—A systematic review,"
Economics Letters, Elsevier, vol. 147(C), pages 51-54.
- Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2015. "Policy evaluation, randomized controlled trials, and external validity: A systematic review," Ruhr Economic Papers 589, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Siebert, W. Stanley & Wei, Xiangdong & Wong, Ho Lun & Zhou, Xiang, 2018. "Student Feedback, Parent-Teacher Communication, and Academic Performance: Experimental Evidence from Rural China," IZA Discussion Papers 11347, Institute of Labor Economics (IZA).
- Agostinelli, Francesco & Avitabile, Ciro & Bobba, Matteo, 2021. "Enhancing Human Capital at Scale," IZA Discussion Papers 14192, Institute of Labor Economics (IZA).
- Eric Bettinger & Nina Cunha & Guilherme Lichand & Ricardo Madeira, 2020. "Are the effects of informational interventions driven by salience?," ECON - Working Papers 350, Department of Economics - University of Zurich, revised May 2021.
- Wagner, Valentin & Riener, Gerhard, 2015. "Peers or parents? On non-monetary incentives in schools," DICE Discussion Papers 203, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
More about this item
Keywords
external validity; field experiments; generalizability; treatment effect; overlap; balance; precision; treatment assignment; unconfoundedness; self-selection bias; site-selection bias;All these keywords.
JEL classification:
- C9 - Mathematical and Quantitative Methods - - Design of Experiments
- C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
- C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
- D04 - Microeconomics - - General - - - Microeconomic Policy: Formulation; Implementation; Evaluation
NEP fields
This paper has been announced in the following NEP Reports:- NEP-EXP-2020-08-10 (Experimental Economics)
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:dicedp:345. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/diduede.html .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.