IDEAS home Printed from https://ideas.repec.org/a/eee/wdevel/v127y2020ics0305750x19304735.html
   My bibliography  Save this article

Highly prized experiments

Author

Listed:
  • Ravallion, Martin

Abstract

The new Nobel prize winners have expertly popularized randomized controlled trials (RCTs) as the “tool-of-choice” for empirical research. The award is a good opportunity to reflect on the role of RCTs in development-policy evaluation. Unbiasedness is the tool’s main virtue; transparency is another. Practitioners should also be aware of some limitations. First, an RCT assigns the treatment in a different way to most real-world policies, which use purposive selection; given heterogeneous impacts, one is evaluating a different intervention. Second, the tool may only be feasible for non-random subsets of both the relevant populations and the policy options, biasing assessments of overall development effectiveness. Third, given budget-constraints and a bias-variance trade-off, a non-RCT may allow a larger sample size, making its trials often closer to the truth. There is a continuing need for a broad range of research methods for addressing pressing knowledge gaps in fighting poverty.

Suggested Citation

  • Ravallion, Martin, 2020. "Highly prized experiments," World Development, Elsevier, vol. 127(C).
    • Handle: RePEc:eee:wdevel:v:127:y:2020:i:c:s0305750x19304735
    DOI: 10.1016/j.worlddev.2019.104824
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0305750X19304735
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.worlddev.2019.104824?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    2. Martin Ravallion, 2018. "Should the Randomistas (Continue to) Rule?," Working Papers 492, Center for Global Development, revised 17 Jan 2019.
    3. Rachael Meager, 2019. "Understanding the Average Impact of Microcredit Expansions: A Bayesian Hierarchical Analysis of Seven Randomized Experiments," American Economic Journal: Applied Economics, American Economic Association, vol. 11(1), pages 57-91, January.
    4. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    5. Independent Evaluation Group, 2012. "World Bank Group Impact Evaluations : Relevance and Effectiveness," World Bank Publications - Books, The World Bank Group, number 13100.
    6. Meager, Rachael, 2019. "Understanding the average impact of microcredit expansions: a Bayesian hierarchical analysis of seven randomized experiments," LSE Research Online Documents on Economics 88190, London School of Economics and Political Science, LSE Library.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lucas Ronconi & Ana Lúcia Kassouf, 2023. "Demand-Side Obstacles to Publishing Economics Research: A View from the South," Progress in Development Studies, , vol. 23(1), pages 99-105, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kvangraven, Ingrid Harvold, 2020. "Impoverished economics? A critical assessment of the new gold standard," World Development, Elsevier, vol. 127(C).
    2. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    3. Hanushek, Eric A., 2021. "Addressing cross-national generalizability in educational impact evaluation," International Journal of Educational Development, Elsevier, vol. 80(C).
    4. Valente, Christine & Sievertsen, Hans Henrik & Puri, Mahesh C., 2020. "Saving Neonatal Lives for a Quarter," IZA Discussion Papers 13719, Institute of Labor Economics (IZA).
    5. Abramovsky, Laura & Augsburg, Britta & Lührmann, Melanie & Oteiza, Francisco & Rud, Juan Pablo, 2023. "Community matters: Heterogeneous impacts of a sanitation intervention," World Development, Elsevier, vol. 165(C).
    6. Vikram Tyagi & Sophie Webber, 2021. "A rusting gold standard: Failures in an Indonesian RCT, and the implications for poverty reduction," Environment and Planning A, , vol. 53(5), pages 992-1011, August.
    7. Holla,Alaka & Bendini,Maria Magdalena & Dinarte Diaz,Lelys Ileana & Trako,Iva, 2021. "Is Investment in Preprimary Education Too Low ? Lessons from (Quasi) ExperimentalEvidence across Countries," Policy Research Working Paper Series 9723, The World Bank.
    8. Christopher J. Ruhm, 2019. "Shackling the Identification Police?," Southern Economic Journal, John Wiley & Sons, vol. 85(4), pages 1016-1026, April.
    9. Abhijit Banerjee & Emily Breza & Esther Duflo & Cynthia Kinnan, 2019. "Can Microfinance Unlock a Poverty Trap for Some Entrepreneurs?," NBER Working Papers 26346, National Bureau of Economic Research, Inc.
    10. Daniel Bjorkegren & Joshua Blumenstock & Omowunmi Folajimi-Senjobi & Jacqueline Mauro & Suraj R. Nair, 2022. "Instant Loans Can Lift Subjective Well-Being: A Randomized Evaluation of Digital Credit in Nigeria," Papers 2202.13540, arXiv.org.
    11. Stefano DellaVigna & Elizabeth Linos, 2022. "RCTs to Scale: Comprehensive Evidence From Two Nudge Units," Econometrica, Econometric Society, vol. 90(1), pages 81-116, January.
    12. Seth Garz & Xavier Gine & Dean Karlan & Rafe Mazer & Caitlin Sanford & Jonathan Zinman, 2021. "Consumer Protection for Financial Inclusion in Low- and Middle-Income Countries: Bridging Regulator and Academic Perspectives," Annual Review of Financial Economics, Annual Reviews, vol. 13(1), pages 219-246, November.
    13. Clare Balboni & Oriana Bandiera & Robin Burgess & Maitreesh Ghatak & Anton Heil, 2023. "Why Do People Stay Poor?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 137(2), pages 785-844.
    14. A Stefano Caria & Grant Gordon & Maximilian Kasy & Simon Quinn & Soha Osman Shami & Alexander Teytelboym, 2024. "An Adaptive Targeted Field Experiment: Job Search Assistance for Refugees in Jordan," Journal of the European Economic Association, European Economic Association, vol. 22(2), pages 781-836.
    15. Oriana Bandiera & Ahmed Elsayed & Andrea Smurra & Céline Zipfel, 2022. "Young Adults and Labor Markets in Africa," Journal of Economic Perspectives, American Economic Association, vol. 36(1), pages 81-100, Winter.
    16. Davis, C. Austin & Mobarak, Ahmed Mushfiq, 2020. "The challenges of scaling effective interventions: A path forward for research and policy," World Development, Elsevier, vol. 127(C).
    17. Oriana Bandiera & Robin Burgess & Erika Deserranno & Ricardo Morel & Munshi Sulaiman & Imran Rasul, 2023. "Social Incentives, Delivery Agents, and the Effectiveness of Development Interventions," Journal of Political Economy Microeconomics, University of Chicago Press, vol. 1(1), pages 162-224.
    18. Tamara Broderick & Ryan Giordano & Rachael Meager, 2020. "An Automatic Finite-Sample Robustness Metric: When Can Dropping a Little Data Make a Big Difference?," Papers 2011.14999, arXiv.org, revised Jul 2023.
    19. Higney, Anthony & Hanley, Nick & Moro, Mirko, 2022. "The lead-crime hypothesis: A meta-analysis," Regional Science and Urban Economics, Elsevier, vol. 97(C).
    20. Susan Athey & Kristen Grabarz & Michael Luca & Nils Wernerfelt, 2023. "Digital public health interventions at scale: The impact of social media advertising on beliefs and outcomes related to COVID vaccines," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 120(5), pages 2208110120-, January.

    More about this item

    Keywords

    Randomized controlled trials; Bias; Sample selection;
    All these keywords.

    JEL classification:

    • B23 - Schools of Economic Thought and Methodology - - History of Economic Thought since 1925 - - - Econometrics; Quantitative and Mathematical Studies
    • H43 - Public Economics - - Publicly Provided Goods - - - Project Evaluation; Social Discount Rate
    • O22 - Economic Development, Innovation, Technological Change, and Growth - - Development Planning and Policy - - - Project Analysis

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:wdevel:v:127:y:2020:i:c:s0305750x19304735. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/worlddev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.