IDEAS home Printed from https://ideas.repec.org/p/hal/wpaper/hal-01686672.html
   My bibliography  Save this paper

The gold standard for randomised evaluations: from discussion of method to political economy

Author

Listed:
  • Florent Bédécarrats

    (autre - AUTRES)

  • Isabelle Guérin

    (Institut de Recherche pour le Développement (IRD))

  • François Roubaud

    (LEDa - Laboratoire d'Economie de Dauphine - Université Paris Dauphine-PSL - PSL - Université Paris Sciences et Lettres)

Abstract

This last decade has seen the emergence of a new field of research in development economics: randomised control trials. This paper explores the contrast between the (many) limitations and (very narrow) real scope of these methods and their success in sheer number and media coverage. Our analysis suggests that the paradox is due to a particular economic and political mix driven by the innovative strategies used by this new school's researchers and by specific interests and preferences in the academic world and the donor community.

Suggested Citation

  • Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2018. "The gold standard for randomised evaluations: from discussion of method to political economy," Working Papers hal-01686672, HAL.
    • Handle: RePEc:hal:wpaper:hal-01686672
    Note: View the original document on HAL open archive server: https://hal.science/hal-01686672
    as

    Download full text from publisher

    File URL: https://hal.science/hal-01686672/document
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Carvalho, Luciano Castro de & Di Serio, Luiz Carlos & Vasconcellos, Marcos Augusto de, 2012. "Competitividade das nações: análise da métrica utilizada pelo World Economic Forum," RAE - Revista de Administração de Empresas, FGV-EAESP Escola de Administração de Empresas de São Paulo (Brazil), vol. 52(4), July.
    2. Daria Taglioni & Richard Baldwin, 2014. "Gravity chains: Estimating bilateral trade flows when parts and components trade is important," Journal of Banking and Financial Economics, University of Warsaw, Faculty of Management, vol. 2(2), pages 61-82, November.
    3. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1128, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
    4. James E. Anderson & Eric van Wincoop, 2003. "Gravity with Gravitas: A Solution to the Border Puzzle," American Economic Review, American Economic Association, vol. 93(1), pages 170-192, March.
    5. Tanguy Bernard & Jocelyne Delarue & Jean-David Naudet, 2012. "Impact evaluations: a tool for accountability? Lessons from experience at Agence Française de D�veloppement," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 4(2), pages 314-327, June.
    6. Westbrock, Bastian & Bosker, Maarten, 2014. "A theory of trade in a global production network," CEPR Discussion Papers 9870, C.E.P.R. Discussion Papers.
    7. Michael Rosholm & Lars Skipper, 2009. "Is labour market training a curse for the unemployed? Evidence from a social experiment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(2), pages 338-365, March.
    8. Paul Shaffer, 2011. "Against Excessive Rhetoric in Impact Assessment: Overstating the Case for Randomised Controlled Experiments," Journal of Development Studies, Taylor & Francis Journals, vol. 47(11), pages 1619-1635.
    9. van Bergeijk,Peter A. G. & Brakman,Steven (ed.), 2010. "The Gravity Model in International Trade," Cambridge Books, Cambridge University Press, number 9780521196154, November.
    10. Escaith, Hubert & Lindenberg, Nannette & Miroudot, Sébastien, 2010. "International supply chains and trade elasticity in times of global crisis," WTO Staff Working Papers ERSD-2010-08, World Trade Organization (WTO), Economic Research and Statistics Division.
    11. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    12. Guilhoto, J. J. M. & Sesso Filho, U. A., 2005. "Estimação da Matriz Insumo-Produto a Partir De Dados Preliminares das Contas Nacionais [Estimation of input-output matrix using preliminary data from national accounts]," MPRA Paper 38212, University Library of Munich, Germany.
    13. Rachel Glennerster, 2012. "The Power of Evidence: Improving the Effectiveness of Government by Investing in More Rigorous Evaluation," National Institute Economic Review, National Institute of Economic and Social Research, vol. 219(1), pages 4-14, January.
    14. Morvant-Roux, Solène & Guérin, Isabelle & Roesch, Marc & Moisseron, Jean-Yves, 2014. "Adding Value to Randomization with Qualitative Analysis: The Case of Microcredit in Rural Morocco," World Development, Elsevier, vol. 56(C), pages 302-312.
    15. Christopher B. Barrett & Michael R. Carter, 2010. "The Power and Pitfalls of Experiments in Development Economics: Some Non-random Reflections," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 32(4), pages 515-548.
    16. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    17. Howard White, 2014. "Current Challenges in Impact Evaluation," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 26(1), pages 18-30, January.
    18. Martin Ravallion, 2009. "Evaluation in the Practice of Development," The World Bank Research Observer, World Bank, vol. 24(1), pages 29-53, March.
    19. Labrousse, Agnès, 2010. "Nouvelle économie du développement et essais cliniques randomisés : une mise en perspective d’un outil de preuve et de gouvernement," Revue de la Régulation - Capitalisme, institutions, pouvoirs, Association Recherche et Régulation, vol. 7.
    20. Daron Acemoglu, 2010. "Theory, General Equilibrium, and Political Economy in Development Economics," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 17-32, Summer.
    21. James Heckman & Jeffrey Smith & Christopher Taber, 1998. "Accounting For Dropouts In Evaluations Of Social Programs," The Review of Economics and Statistics, MIT Press, vol. 80(1), pages 1-14, February.
    22. Rodrik, Dani, 2008. "The New Development Economics: We Shall Experiment, but How Shall We Learn?," Working Paper Series rwp08-055, Harvard University, John F. Kennedy School of Government.
    23. William Easterly (ed.), 2008. "Reinventing Foreign Aid," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262550660, December.
    24. Guilhoto, Joaquim José Martins & Sesso Filho, Umberto Antonio, 2010. "Estimação da matriz insumo-produto utilizando dados preliminares das contas nacionais: aplicação e análise de indicadores econômicos para o Brasil em 2005 [Estimation of input-output matrix using p," MPRA Paper 37539, University Library of Munich, Germany.
    25. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009 is not listed on IDEAS
    26. Kei-Mu Yi, 2003. "Can Vertical Specialization Explain the Growth of World Trade?," Journal of Political Economy, University of Chicago Press, vol. 111(1), pages 52-102, February.
    27. Kei-Mu Yi, 2010. "Can Multistage Production Explain the Home Bias in Trade?," American Economic Review, American Economic Association, vol. 100(1), pages 364-393, March.
    28. Tessa Bold & Mwangi Kimenyi & Germano Mwabu & Alice Ng'ang'a & Justin Sandefur, 2013. "Scaling-up What Works: Experimental Evidence on External Validity in Kenyan Education," CSAE Working Paper Series 2013-04, Centre for the Study of African Economies, University of Oxford.
    29. Ayçıl Yücer & Joaquim Guilhoto & Jean-Marc Siroën, 2014. "Internal and International Vertical Specialization of Brazilian states – An Input-Output analysis," Revue d'économie politique, Dalloz, vol. 124(4), pages 599-612.
    30. Jyrki Ali-Yrkkö & Petri Rouvinen & Timo Seppälä & Pekka Ylä-Anttila, 2011. "Who Captures Value in Global Supply Chains? Case Nokia N95 Smartphone," Journal of Industry, Competition and Trade, Springer, vol. 11(3), pages 263-278, September.
    31. Florent Bédécarrats, 2012. "L'impact de la microfinance : un enjeu politique au prisme de ses controverses scientifiques," Mondes en développement, De Boeck Université, vol. 0(2), pages 127-142.
    32. McCallum, John, 1995. "National Borders Matter: Canada-U.S. Regional Trade Patterns," American Economic Review, American Economic Association, vol. 85(3), pages 615-623, June.
    33. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009.pdf is not listed on IDEAS
    34. Glenn W. Harrison, 2011. "Randomisation and Its Discontents," Journal of African Economies, Centre for the Study of African Economies, vol. 20(4), pages 626-652, August.
    35. Hummels, David & Ishii, Jun & Yi, Kei-Mu, 2001. "The nature and growth of vertical specialization in world trade," Journal of International Economics, Elsevier, vol. 54(1), pages 75-96, June.
    36. Abhijit Banerjee & Dean Karlan & Jonathan Zinman, 2015. "Six Randomized Evaluations of Microcredit: Introduction and Further Steps," American Economic Journal: Applied Economics, American Economic Association, vol. 7(1), pages 1-21, January.
    37. Lant Pritchett, Justin Sandefur, 2013. "Context Matters for Size: Why External Validity Claims and Development Practice Don't Mix-Working Paper 336," Working Papers 336, Center for Global Development.
    38. repec:pri:rpdevs:instruments_of_development.pdf is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019. "All that Glitters is not Gold. The Political Economy of Randomized Evaluations in Development," Development and Change, International Institute of Social Studies, vol. 50(3), pages 735-762, May.
    2. Florent BEDECARRATS & Isabelle GUERIN & François ROUBAUD, 2017. "L'étalon-or des évaluations randomisées : économie politique des expérimentations aléatoires dans le domaine du développement," Working Paper 753120cd-506f-4c5f-80ed-7, Agence française de développement.
    3. Joana Silva Afonso, 2020. "Impact evaluation, social performance assessment and standardisation: reflections from microfinance evaluations in Pakistan and Zimbabwe," Working Papers in Economics & Finance 2020-14, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    4. Gloukoviezoff, Georges, 2016. "Evaluating the impact of European microfinance. The foundations," EIF Working Paper Series 2016/33, European Investment Fund (EIF).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2015. "The gold standard for randomised evaluations: from discussion of method to political economics," Working Papers CEB 15-009, ULB -- Universite Libre de Bruxelles.
    2. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019. "All that Glitters is not Gold. The Political Economy of Randomized Evaluations in Development," Development and Change, International Institute of Social Studies, vol. 50(3), pages 735-762, May.
    3. Florent Bedecarrats & Isabelle Guérin & François Roubaud, 2017. "L'étalon-or des évaluations randomisées : du discours de la méthode à l'économie politique," Working Papers ird-01445209, HAL.
    4. Florent BEDECARRATS & Isabelle GUERIN & François ROUBAUD, 2017. "L'étalon-or des évaluations randomisées : économie politique des expérimentations aléatoires dans le domaine du développement," Working Paper 753120cd-506f-4c5f-80ed-7, Agence française de développement.
    5. Joachim Guilhoto & Jean-Marc Siroën & Ayçil Yücer, 2015. "The gravity model, global value chain and the brazilian states," Working Papers DT/2015/02, DIAL (Développement, Institutions et Mondialisation).
    6. Guilhoto, Joaquim José Martins & Siroën, Jean-Marc & Yucer, Ayçil, 2013. "Internal and International Vertical Specialization– Estimations For Brazil and new Approach to Gravity Models," MPRA Paper 46897, University Library of Munich, Germany.
    7. Pol Antràs & Alonso de Gortari, 2020. "On the Geography of Global Value Chains," Econometrica, Econometric Society, vol. 88(4), pages 1553-1598, July.
    8. repec:dau:papers:123456789/11720 is not listed on IDEAS
    9. Inmaculada Martínez-Zarzoso & Martina Vidovic & Anca M. Voicu, 2014. "EU-Accession Effects on Sectoral Trade: A Helpman-Melitz-Rubinstein Approach with Panel Data," CESifo Working Paper Series 4903, CESifo.
    10. Christian Volpe Martincus, 2016. "Out of the Border Labyrinth: An Assessment of Trade Facilitation Initiatives in Latin America and the Caribbean," IDB Publications (Books), Inter-American Development Bank, number 96856, February.
    11. Agnosteva, Delina E. & Anderson, James E. & Yotov, Yoto V., 2019. "Intra-national trade costs: Assaying regional frictions," European Economic Review, Elsevier, vol. 112(C), pages 32-50.
    12. Martínez-Zarzoso, Inmaculada & Voicu, Anca M. & Vidovic, Martina, 2011. "CEECs Integration into Regional Production Networks. Trade Effects of EU-Accession," Proceedings of the German Development Economics Conference, Berlin 2011 55, Verein für Socialpolitik, Research Committee Development Economics.
    13. James E. Anderson & Eric van Wincoop, 2004. "Trade Costs," Journal of Economic Literature, American Economic Association, vol. 42(3), pages 691-751, September.
    14. Franco-Bedoya, Sebastian & Frohm, Erik, 2020. "Global trade in final goods and intermediate inputs: impact of FTAs and reduced “Border Effects”," Working Paper Series 2410, European Central Bank.
    15. Jean-Marc Siroën & Aycil Yucer, 2012. "The impact of MERCOSUR on trade of Brazilian states," Review of World Economics (Weltwirtschaftliches Archiv), Springer;Institut für Weltwirtschaft (Kiel Institute for the World Economy), vol. 148(3), pages 553-582, September.
    16. Jean Baliè & Davide Del Prete & Emiliano Magrini & Pierluigi Montalbano & Silvia Nenci, 2017. "Agriculture and Food Global Value Chains in Sub-Saharan Africa: Does bilateral trade policy impact on backward and forward participation?," Working Papers 03/2017, IMT School for Advanced Studies Lucca, revised Feb 2017.
    17. Willert, Bianca, 2017. "How intermediates trade affects the formation of free trade agreements: A study analyzing pairwise trade flows of 70 countries," Thuenen-Series of Applied Economic Theory 147, University of Rostock, Institute of Economics, revised 2017.
    18. Robert C. Johnson & Guillermo Noguera, 2012. "Fragmentation and Trade in Value Added over Four Decades," NBER Working Papers 18186, National Bureau of Economic Research, Inc.
    19. Franco, Sebastian & Frohm, Erik, 2018. "Reduced "Border Effects", FTAs and International Trade," Working Paper Series 356, Sveriges Riksbank (Central Bank of Sweden).
    20. Athukorala, Prema-chandra, 2014. "How India Fits into Global Production Sharing: Experience, Prospects, and Policy Options," India Policy Forum, National Council of Applied Economic Research, vol. 10(1), pages 57-116.
    21. E. M. Bosker & Bastian Westbrock, 2014. "A theory of trade in a global production network," Working Papers 14-14, Utrecht School of Economics.

    More about this item

    Keywords

    Evaluation d'impact; méthode expérimentale; Essai randomisé; Méthodologie; Economie politique; Développement; Impact evaluation; Randomized control trial; Experimental method; Methodology; Political economy; Development;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D72 - Microeconomics - - Analysis of Collective Decision-Making - - - Political Processes: Rent-seeking, Lobbying, Elections, Legislatures, and Voting Behavior
    • O10 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:wpaper:hal-01686672. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.