IDEAS home Printed from https://ideas.repec.org/p/zbw/rwirep/716.html
   My bibliography  Save this paper

Generalization in the Tropics: Development policy, randomized controlled trials, and external validity

Author

Listed:
  • Peters, Jörg
  • Langbein, Jörg
  • Roberts, Gareth

Abstract

When properly implemented, Randomized Controlled Trials (RCT) achieve a high degree of internal validity. Yet, if an RCT is to inform policy, it is critical to establish external validity. This paper systematically reviews all RCTs conducted in developing countries and published in leading economic journals between 2009 and 2014 with respect to how they deal with external validity. Following Duflo, Glennerster, and Kremer (2008), the hazards to external validity we scrutinize are Hawthorne effects, general equilibrium effects, specific sample problems, and special care in treatment provision. Based on a set of objective indicators, we find that the majority of published RCTs does not discuss these hazards and many do not provide the necessary information to assess potential problems. The paper calls for including external validity dimensions in a more systematic reporting on the results of RCTs. This may create incentives to avoid overgeneralizing findings and help policy makers to interpret results appropriately.

Suggested Citation

  • Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2017. "Generalization in the Tropics: Development policy, randomized controlled trials, and external validity," Ruhr Economic Papers 716, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    • Handle: RePEc:zbw:rwirep:716
    DOI: 10.4419/86788835
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/170702/1/100071845X.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.4419/86788835?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Pedro C. Vicente, 2014. "Is Vote Buying Effective? Evidence from a Field Experiment in West Africa," Economic Journal, Royal Economic Society, vol. 124(574), pages 356-387, February.
    2. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    3. Karen Macours & Norbert Schady & Renos Vakis, 2012. "Cash Transfers, Behavioral Changes, and Cognitive Development in Early Childhood: Evidence from a Randomized Experiment," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 247-273, April.
    4. Adrienne M. Lucas & Isaac M. Mbiti, 2014. "Effects of School Quality on Student Achievement: Discontinuity Evidence from Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 6(3), pages 234-263, July.
    5. Alex Eble & Peter Boone & Diana Elbourne, 2017. "On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics," The World Bank Economic Review, World Bank, vol. 31(3), pages 687-707.
    6. Dean Karlan & Martin Valdivia, 2011. "Teaching Entrepreneurship: Impact of Business Training on Microfinance Clients and Institutions," The Review of Economics and Statistics, MIT Press, vol. 93(2), pages 510-527, May.
    7. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    8. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1128, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
    9. Lori Beaman & Jeremy Magruder, 2012. "Who Gets the Job Referral? Evidence from a Social Networks Experiment," American Economic Review, American Economic Association, vol. 102(7), pages 3574-3593, December.
    10. Paul Collier & Pedro C. Vicente, 2014. "Votes and Violence: Evidence from a Field Experiment in Nigeria," Economic Journal, Royal Economic Society, vol. 124(574), pages 327-355, February.
    11. Bertrand, Marianne & Karlan, Dean S. & Mullainathan, Sendhil & Shafir, Eldar & Zinman, Jonathan, 2005. "What's Psychology Worth? A Field Experiment in the Consumer Credit Market," Center Discussion Papers 28441, Yale University, Economic Growth Center.
    12. repec:hal:pseose:halshs-00840901 is not listed on IDEAS
    13. Dean Karlan & Robert Osei & Isaac Osei-Akoto & Christopher Udry, 2014. "Agricultural Decisions after Relaxing Credit and Risk Constraints," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(2), pages 597-652.
    14. Karen Macours & Renos Vakis, 2014. "Changing Households' Investment Behaviour through Social Interactions with Local Leaders: Evidence from a Randomised Transfer Programme," Economic Journal, Royal Economic Society, vol. 0(576), pages 607-633, May.
    15. Alejandro Drexler & Greg Fischer & Antoinette Schoar, 2014. "Keeping It Simple: Financial Literacy and Rules of Thumb," American Economic Journal: Applied Economics, American Economic Association, vol. 6(2), pages 1-31, April.
    16. Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(2), pages 531-580.
    17. David K. Evans & Anna Popova, 2016. "What Really Works to Improve Learning in Developing Countries? An Analysis of Divergent Findings in Systematic Reviews," The World Bank Research Observer, World Bank, vol. 31(2), pages 242-270.
    18. Michael Woolcock, 2013. "Using Case Studies to Explore the External Validity of ‘Complex’ Development Interventions," CID Working Papers 270, Center for International Development at Harvard University.
    19. Esther Duflo & Michael Greenstone & Nicholas Ryan, 2013. "Truth-telling by Third-party Auditors and the Response of Polluting Firms: Experimental Evidence from India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(4), pages 1499-1545.
    20. Orazio Attanasio & Abigail Barr & Juan Camilo Cardenas & Garance Genicot & Costas Meghir, 2012. "Risk Pooling, Risk Preferences, and Social Networks," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 134-167, April.
    21. Vivi Alatas & Abhijit Banerjee & Rema Hanna & Benjamin A. Olken & Julia Tobias, 2012. "Targeting the Poor: Evidence from a Field Experiment in Indonesia," American Economic Review, American Economic Association, vol. 102(4), pages 1206-1240, June.
    22. repec:hal:pseose:halshs-00754593 is not listed on IDEAS
    23. Martina Björkman & Jakob Svensson, 2009. "Power to the People: Evidence from a Randomized Field Experiment on Community-Based Monitoring in Uganda," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 124(2), pages 735-769.
    24. Suresh de Mel & David McKenzie & Christopher Woodruff, 2013. "The Demand for, and Consequences of, Formalization among Informal Firms in Sri Lanka," American Economic Journal: Applied Economics, American Economic Association, vol. 5(2), pages 122-150, April.
    25. David P. Coady & Rebecca L. Harris, 2004. "Evaluating transfer programmes within a general equilibrium framework," Economic Journal, Royal Economic Society, vol. 114(498), pages 778-799, October.
    26. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    27. Jonathan Robinson, 2012. "Limited Insurance within the Household: Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 4(4), pages 140-164, October.
    28. Jonas Hjort, 2013. "Ethnic Divisions and Production in Firms," CESifo Working Paper Series 4449, CESifo.
    29. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    30. Lori Beaman & Raghabendra Chattopadhyay & Esther Duflo & Rohini Pande & Petia Topalova, 2009. "Powerful Women: Does Exposure Reduce Bias?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 124(4), pages 1497-1540.
    31. David McKenzie & Christopher Woodruff, 2014. "What Are We Learning from Business Training and Entrepreneurship Evaluations around the Developing World?," The World Bank Research Observer, World Bank, vol. 29(1), pages 48-82.
    32. Pascaline Dupas & Jonathan Robinson, 2013. "Savings Constraints and Microenterprise Development: Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 5(1), pages 163-192, January.
    33. Nicholas Bloom & Benn Eifert & Aprajit Mahajan & David McKenzie & John Roberts, 2013. "Does Management Matter? Evidence from India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(1), pages 1-51.
    34. Xavier Gine & Jessica Goldberg & Dean Yang, 2012. "Credit Market Consequences of Improved Personal Identification: Field Experimental Evidence from Malawi," American Economic Review, American Economic Association, vol. 102(6), pages 2923-2954, October.
    35. David K. Evans & Anna Popova, 2017. "Cash Transfers and Temptation Goods," Economic Development and Cultural Change, University of Chicago Press, vol. 65(2), pages 189-221.
    36. Jishnu Das & Stefan Dercon & James Habyarimana & Pramila Krishnan & Karthik Muralidharan & Venkatesh Sundararaman, 2013. "School Inputs, Household Substitution, and Test Scores," American Economic Journal: Applied Economics, American Economic Association, vol. 5(2), pages 29-57, April.
    37. Banerjee, Abhijit & Bardhan, Pranab & Basu, Kaushik, 2005. "New Directions in Development Economics: Theory or Empirics? A Symposium in Economic and Political Weekly," Working Papers 127128, Cornell University, Department of Applied Economics and Management.
    38. Pascaline Dupas, 2011. "Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 1-34, January.
    39. Sylvain Chassang & Gerard Padro I Miquel & Erik Snowberg, 2012. "Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments," American Economic Review, American Economic Association, vol. 102(4), pages 1279-1309, June.
    40. Erica Field & Rohini Pande & John Papp & Natalia Rigol, 2013. "Does the Classic Microfinance Model Discourage Entrepreneurship among the Poor? Experimental Evidence from India," American Economic Review, American Economic Association, vol. 103(6), pages 2196-2226, October.
    41. Michal Bauer & Julie Chytilova & Jonathan Morduch, 2012. "Behavioral Foundations of Microcredit: Experimental and Survey Evidence from Rural India," American Economic Review, American Economic Association, vol. 102(2), pages 1118-1139, April.
    42. Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," Working Papers august_25.pdf, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    43. repec:idb:brikps:77778 is not listed on IDEAS
    44. Timothy J. Besley & Konrad B. Burchardi & Maitreesh Ghatak, 2012. "Incentives and the De Soto Effect," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(1), pages 237-282.
    45. Rafael Di Tella & Ernesto Schargrodsky, 2013. "Criminal Recidivism after Prison and Electronic Monitoring," Journal of Political Economy, University of Chicago Press, vol. 121(1), pages 28-73.
    46. Woolcock, Michael, 2013. "Using Case Studies to Explore the External Validity of 'Complex' Development Interventions," Working Paper Series rwp13-048, Harvard University, John F. Kennedy School of Government.
    47. Robert Jensen, 2012. "Do Labor Market Opportunities Affect Young Women's Work and Family Decisions? Experimental Evidence from India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(2), pages 753-792.
    48. Dean Karlan & Jonathan Zinman, 2009. "Observing Unobservables: Identifying Information Asymmetries With a Consumer Credit Field Experiment," Econometrica, Econometric Society, vol. 77(6), pages 1993-2008, November.
    49. Karthik Muralidharan & Venkatesh Sundararaman, 2010. "The Impact of Diagnostic Feedback to Teachers on Student Learning: Experimental Evidence from India," Economic Journal, Royal Economic Society, vol. 120(546), pages 187-203, August.
    50. Marianne Bertrand & Dean Karlan & Sendhil Mullainathan & Eldar Shafir & Jonathan Zinman, 2010. "What's Advertising Content Worth? Evidence from a Consumer Credit Marketing Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 263-306.
    51. Karthik Muralidharan & Paul Niehaus & Sandip Sukhtankar, 2023. "General Equilibrium Effects of (Improving) Public Employment Programs: Experimental Evidence From India," Econometrica, Econometric Society, vol. 91(4), pages 1261-1295, July.
    52. Xavier Giné & Dean Karlan & Jonathan Zinman, 2010. "Put Your Money Where Your Butt Is: A Commitment Contract for Smoking Cessation," American Economic Journal: Applied Economics, American Economic Association, vol. 2(4), pages 213-235, October.
    53. Menno Pradhan & Daniel Suryadarma & Amanda Beatty & Maisy Wong & Arya Gaduh & Armida Alisjahbana & Rima Prama Artha, 2014. "Improving Educational Quality through Enhancing Community Participation: Results from a Randomized Field Experiment in Indonesia," American Economic Journal: Applied Economics, American Economic Association, vol. 6(2), pages 105-126, April.
    54. Nava Ashraf & James Berry & Jesse M. Shapiro, 2010. "Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia," American Economic Review, American Economic Association, vol. 100(5), pages 2383-2413, December.
    55. Paul Glewwe & Nauman Ilias & Michael Kremer, 2010. "Teacher Incentives," American Economic Journal: Applied Economics, American Economic Association, vol. 2(3), pages 205-227, July.
    56. Seán M. Muller, 2015. "Causal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations," The World Bank Economic Review, World Bank, vol. 29(suppl_1), pages 217-225.
    57. Levitt, Steven D. & List, John A., 2009. "Field experiments in economics: The past, the present, and the future," European Economic Review, Elsevier, vol. 53(1), pages 1-18, January.
    58. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009.pdf is not listed on IDEAS
    59. Sylvie Moulin & Michael Kremer & Paul Glewwe, 2009. "Many Children Left Behind? Textbooks and Test Scores in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 1(1), pages 112-135, January.
    60. Alessandro Tarozzi & Aprajit Mahajan & Brian Blackburn & Dan Kopf & Lakshmi Krishnan & Joanne Yoong, 2014. "Micro-loans, Insecticide-Treated Bednets, and Malaria: Evidence from a Randomized Controlled Trial in Orissa, India," American Economic Review, American Economic Association, vol. 104(7), pages 1909-1941, July.
    61. Michael Kremer & Edward Miguel & Rebecca Thornton, 2009. "Incentives to Learn," The Review of Economics and Statistics, MIT Press, vol. 91(3), pages 437-456, August.
    62. Kaushik Basu, 2014. "Randomisation, Causality and the Role of Reasoned Intuition," Oxford Development Studies, Taylor & Francis Journals, vol. 42(4), pages 455-472, December.
    63. Hongbin Cai & Yuyu Chen & Hanming Fang, 2009. "Observational Learning: Evidence from a Randomized Natural Field Experiment," American Economic Review, American Economic Association, vol. 99(3), pages 864-882, June.
    64. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    65. Elizabeth M. King & Jere R. Behrman, 2009. "Timing and Duration of Exposure in Evaluations of Social Programs," The World Bank Research Observer, World Bank, vol. 24(1), pages 55-82, February.
    66. Maarten J. Voors & Eleonora E. M. Nillesen & Philip Verwimp & Erwin H. Bulte & Robert Lensink & Daan P. Van Soest, 2012. "Violent Conflict and Behavior: A Field Experiment in Burundi," American Economic Review, American Economic Association, vol. 102(2), pages 941-964, April.
    67. Pascaline Dupas, 2014. "Short‐Run Subsidies and Long‐Run Adoption of New Health Products: Evidence From a Field Experiment," Econometrica, Econometric Society, vol. 82(1), pages 197-228, January.
    68. Temple, Jonathan R.W., 2010. "Aid and Conditionality," Handbook of Development Economics, in: Dani Rodrik & Mark Rosenzweig (ed.), Handbook of Development Economics, edition 1, volume 5, chapter 0, pages 4415-4523, Elsevier.
    69. Pascaline Dupas & Jonathan Robinson, 2013. "Why Don't the Poor Save More? Evidence from Health Savings Experiments," American Economic Review, American Economic Association, vol. 103(4), pages 1138-1171, June.
    70. Leonardo Bursztyn & Lucas C. Coffman, 2012. "The Schooling Decision: Family Preferences, Intergenerational Conflict, and Moral Hazard in the Brazilian Favelas," Journal of Political Economy, University of Chicago Press, vol. 120(3), pages 359-397.
    71. Paul J. Gertler & Sebastian W. Martinez & Marta Rubio-Codina, 2012. "Investing Cash Transfers to Raise Long-Term Living Standards," American Economic Journal: Applied Economics, American Economic Association, vol. 4(1), pages 164-192, January.
    72. Benjamin A. Olken & Junko Onishi & Susan Wong, 2014. "Should Aid Reward Performance? Evidence from a Field Experiment on Health and Education in Indonesia," American Economic Journal: Applied Economics, American Economic Association, vol. 6(4), pages 1-34, October.
    73. Olivier Armantier & Amadou Boly, 2013. "Comparing Corruption in the Laboratory and in the Field in Burkina Faso and in Canada," Economic Journal, Royal Economic Society, vol. 123(12), pages 1168-1187, December.
    74. Esther Duflo & Michael Kremer & Jonathan Robinson, 2011. "Nudging Farmers to Use Fertilizer: Theory and Experimental Evidence from Kenya," American Economic Review, American Economic Association, vol. 101(6), pages 2350-2390, October.
    75. Emily Oster & Rebecca Thornton, 2011. "Menstruation, Sanitary Products, and School Attendance: Evidence from a Randomized Evaluation," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 91-100, January.
    76. repec:hal:pseose:halshs-01053534 is not listed on IDEAS
    77. Cilliers, Jacobus & Dube, Oeindrila & Siddiqi, Bilal, 2015. "The white-man effect: How foreigner presence affects behavior in experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 118(C), pages 397-414.
    78. repec:pri:rpdevs:instruments_of_development.pdf is not listed on IDEAS
    79. Suresh de Mel & David McKenzie & Christopher Woodruff, 2009. "Are Women More Credit Constrained? Experimental Evidence on Gender and Microenterprise Returns," American Economic Journal: Applied Economics, American Economic Association, vol. 1(3), pages 1-32, July.
    80. Jobiba Chinkhumba & Susan Godlonton & Rebecca Thornton, 2014. "The Demand for Medical Male Circumcision," American Economic Journal: Applied Economics, American Economic Association, vol. 6(2), pages 152-177, April.
    81. Kenneth F Schulz & Douglas G Altman & David Moher & for the CONSORT Group, 2010. "CONSORT 2010 Statement: Updated Guidelines for Reporting Parallel Group Randomised Trials," PLOS Medicine, Public Library of Science, vol. 7(3), pages 1-7, March.
    82. Uri Gneezy & Kenneth L. Leonard & John A. List, 2009. "Gender Differences in Competition: Evidence From a Matrilineal and a Patriarchal Society," Econometrica, Econometric Society, vol. 77(5), pages 1637-1664, September.
    83. Benjamin Feigenberg & Erica Field & Rohini Pande, 2013. "The Economic Returns to Social Interaction: Experimental Evidence from Microfinance," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(4), pages 1459-1483.
    84. Nava Ashraf, 2009. "Spousal Control and Intra-household Decision Making: An Experimental Study in the Philippines," American Economic Review, American Economic Association, vol. 99(4), pages 1245-1277, September.
    85. Esther Duflo & Rema Hanna & Stephen P. Ryan, 2012. "Incentives Work: Getting Teachers to Come to School," American Economic Review, American Economic Association, vol. 102(4), pages 1241-1278, June.
    86. Hunt Allcott, 2015. "Site Selection Bias in Program Evaluation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(3), pages 1117-1165.
    87. Katherine Casey & Rachel Glennerster & Edward Miguel, 2012. "Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(4), pages 1755-1812.
    88. Jenny C. Aker & Christopher Ksoll & Travis J. Lybbert, 2012. "Can Mobile Phones Improve Learning? Evidence from a Field Experiment in Niger," American Economic Journal: Applied Economics, American Economic Association, vol. 4(4), pages 94-120, October.
    89. Eva Vivalt, 0. "How Much Can We Generalize From Impact Evaluations?," Journal of the European Economic Association, European Economic Association, vol. 18(6), pages 3045-3089.
    90. Nava Ashraf & Erica Field & Jean Lee, 2014. "Household Bargaining and Excess Fertility: An Experimental Study in Zambia," American Economic Review, American Economic Association, vol. 104(7), pages 2210-2237, July.
    91. Martin Ravallion, 2012. "Fighting Poverty One Experiment at a Time: Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty : Review Essay," Journal of Economic Literature, American Economic Association, vol. 50(1), pages 103-114, March.
    92. repec:oup:qjecon:v:129:y:2014:i:2:p:697-752. is not listed on IDEAS
    93. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    94. Gani Aldashev & Georg Kirchsteiger & Alexander Sebald, 2017. "Assignment Procedure Biases in Randomised Policy Experiments," Economic Journal, Royal Economic Society, vol. 127(602), pages 873-895, June.
    95. Felipe Barrera-Osorio & Marianne Bertrand & Leigh L. Linden & Francisco Perez-Calle, 2011. "Improving the Design of Conditional Transfer Programs: Evidence from a Randomized Education Experiment in Colombia," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 167-195, April.
    96. Brian E. Roe & David R. Just, 2009. "Internal and External Validity in Economics Research: Tradeoffs between Experiments, Field Experiments, Natural Experiments, and Field Data," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 91(5), pages 1266-1271.
    97. Sarah Baird & Craig McIntosh & Berk Özler, 2011. "Cash or Condition? Evidence from a Cash Transfer Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 1709-1753.
    98. Karen Macours & Renos Vakis, 2014. "Changing Households' Investment Behaviour through Social Interactions with Local Leaders: Evidence from a Randomised Transfer Programme," Post-Print halshs-01313720, HAL.
    99. Orazio Attanasio & Adriana Kugler & Costas Meghir, 2011. "Subsidizing Vocational Training for Disadvantaged Youth in Colombia: Evidence from a Randomized Trial," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 188-220, July.
    100. Dana Burde & Leigh L. Linden, 2013. "Bringing Education to Afghan Girls: A Randomized Controlled Trial of Village-Based Schools," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 27-40, July.
    101. Elizabeth A. Stuart & Stephen R. Cole & Catherine P. Bradshaw & Philip J. Leaf, 2011. "The use of propensity scores to assess the generalizability of results from randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 369-386, April.
    102. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009 is not listed on IDEAS
    103. Christopher Blattman & Nathan Fiala & Sebastian Martinez, 2014. "Generating Skilled Self-Employment in Developing Countries: Experimental Evidence from Uganda," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(2), pages 697-752.
    104. repec:idb:brikps:publication-detail,7101.html?id=32886 is not listed on IDEAS
    105. Moussa P. Blimpo, 2014. "Team Incentives for Education in Developing Countries: A Randomized Field Experiment in Benin," American Economic Journal: Applied Economics, American Economic Association, vol. 6(4), pages 90-109, October.
    106. Karen Macours & Norbert Schady & Renos Vakis, 2012. "Cash Transfers, Behavioral Changes, and Cognitive Development in Early Childhood: Evidence from a Randomized Experiment," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 247-273, April.
    107. repec:feb:artefa:0087 is not listed on IDEAS
    108. Kaushik Basu & Andrew Foster, 2015. "Development Economics and Method: A Quarter Century of ABCDE," The World Bank Economic Review, World Bank, vol. 29(suppl_1), pages 2-8.
    109. Jessica Cohen & Pascaline Dupas, 2010. "Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 1-45.
    110. Karthik Muralidharan & Venkatesh Sundararaman, 2011. "Teacher Performance Pay: Experimental Evidence from India," Journal of Political Economy, University of Chicago Press, vol. 119(1), pages 39-77.
    111. Rema Hanna & Sendhil Mullainathan & Joshua Schwartzstein, 2014. "Learning Through Noticing: Theory and Evidence from a Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(3), pages 1311-1353.
    112. Karthik Muralidharan & Venkatesh Sundararaman, 2015. "Editor's Choice The Aggregate Effect of School Choice: Evidence from a Two-Stage Experiment in India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(3), pages 1011-1066.
    113. Amanda E. Kowalski, 2018. "How to Examine External Validity Within an Experiment," NBER Working Papers 24834, National Bureau of Economic Research, Inc.
    114. Lant Pritchett & Justin Sandefur, 2015. "Learning from Experiments When Context Matters," American Economic Review, American Economic Association, vol. 105(5), pages 471-475, May.
    115. Suresh de Mel & David McKenzie & Christopher Woodruff, 2009. "Returns to Capital in Microenterprises: Evidence from a Field Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 124(1), pages 423-423.
    116. Robert T. Jensen & Nolan H. Miller, 2011. "Do Consumer Price Subsidies Really Improve Nutrition?," The Review of Economics and Statistics, MIT Press, vol. 93(4), pages 1205-1223, November.
    117. Robert Jensen, 2010. "The (Perceived) Returns to Education and the Demand for Schooling," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(2), pages 515-548.
    118. Thomas Fujiwara & Leonard Wantchekon, 2013. "Can Informed Public Deliberation Overcome Clientelism? Experimental Evidence from Benin," American Economic Journal: Applied Economics, American Economic Association, vol. 5(4), pages 241-255, October.
    119. Erwin Bulte & Gonne Beekman & Salvatore Di Falco & Joseph Hella & Pan Lei, 2014. "Behavioral Responses and the Impact of New Agricultural Technologies: Evidence from a Double-blind Field Experiment in Tanzania," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 96(3), pages 813-830.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    2. Karthik Muralidharan & Paul Niehaus, 2017. "Experimentation at Scale," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 103-124, Fall.
    3. Molina Millán, Teresa & Macours, Karen, 2017. "Attrition in Randomized Control Trials: Using Tracking Information to Correct Bias," IZA Discussion Papers 10711, Institute of Labor Economics (IZA).
    4. Teresa Molina Millán & Karen Macours, 2017. "Attrition in randomized control trials: Using tracking information to correct bias," FEUNL Working Paper Series novaf:wp1702, Universidade Nova de Lisboa, Faculdade de Economia.
    5. Benjamin A. Olken, 2020. "Banerjee, Duflo, Kremer, and the Rise of Modern Development Economics," Scandinavian Journal of Economics, Wiley Blackwell, vol. 122(3), pages 853-878, July.
    6. Pascaline Dupas & Edward Miguel, 2016. "Impacts and Determinants of Health Levels in Low-Income Countries," NBER Working Papers 22235, National Bureau of Economic Research, Inc.
    7. Benjamin A. Olken, 2015. "Promises and Perils of Pre-analysis Plans," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 61-80, Summer.
    8. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    9. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    10. B Kelsey Jack, "undated". "Market Inefficiencies and the Adoption of Agricultural Technologies in Developing Countries," CID Working Papers 50, Center for International Development at Harvard University.
    11. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    12. Eduard Marinov, 2019. "The 2019 Nobel Prize in Economics," Economic Thought journal, Bulgarian Academy of Sciences - Economic Research Institute, issue 6, pages 78-116.
    13. Harounan Kazianga & Leigh Linden & Ali Protik & Matt Sloan, 2015. "Impact Evaluation of Burkina Faso's BRIGHT Program: Design Report," Mathematica Policy Research Reports c0250cd3f27d448ea70d909c3, Mathematica Policy Research.
    14. Karthik Muralidharan & Mauricio Romero & Kaspar Wüthrich, 2019. "Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments," NBER Working Papers 26562, National Bureau of Economic Research, Inc.
    15. Dean Karlan & Aishwarya Lakshmi Ratan & Jonathan Zinman, 2014. "Savings by and for the Poor: A Research Review and Agenda," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 60(1), pages 36-78, March.
    16. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019. "All that Glitters is not Gold. The Political Economy of Randomized Evaluations in Development," Development and Change, International Institute of Social Studies, vol. 50(3), pages 735-762, May.
    17. Eric Bettinger & Nina Cunha & Guilherme Lichand & Ricardo Madeira, 2020. "Are the effects of informational interventions driven by salience?," ECON - Working Papers 350, Department of Economics - University of Zurich, revised May 2021.
    18. Francesca Marchetta & Tom Dilly, 2019. "Supporting Education in Africa: Opportunities and Challenges for an Impact Investor," Working Papers hal-02288103, HAL.
    19. Abhijit Banerjee & Sylvain Chassang & Erik Snowberg, 2016. "Decision Theoretic Approaches to Experiment Design and External Validity," NBER Working Papers 22167, National Bureau of Economic Research, Inc.
    20. Isaac Mbiti & Karthik Muralidharan & Mauricio Romero & Youdi Schipper & Constantine Manda & Rakesh Rajani, 2019. "Inputs, Incentives, and Complementarities in Education: Experimental Evidence from Tanzania," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(3), pages 1627-1673.

    More about this item

    Keywords

    policy evaluation; systematic review; internal validity; external validity; randomized controlled trials;
    All these keywords.

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:rwirep:716. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/rwiesde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.