IDEAS home Printed from https://ideas.repec.org/a/eee/wdevel/v127y2020ics0305750x19304395.html
   My bibliography  Save this article

Randomized interventions and “real” treatment effects: A cautionary tale and an example

Author

Listed:
  • Bulte, Erwin
  • Di Falco, Salvatore
  • Lensink, Robert

Abstract

The experimental approach has revolutionized development economics. Nonetheless, randomization cannot do everything. We discuss challenges to RCTs, paying special attention to internal validity. Randomized interventions in social sciences are not double-blind and do not, in general, hold all relevant covariates constant. Treated and untreated subjects adjust their behavior in response to treatment status. Disentangling the treatment effect into its behavioral component and the direct effect of the intervention is difficult, and implies a return to the toolkit of observational studies. This is illustrated using improved seed distribution in African farming. While standard RCTs found large treatment effects, double-blind RCTs revealed that a large share of this impact is due to farmers allocating extra effort and their best plots to the cultivation of new seeds.

Suggested Citation

  • Bulte, Erwin & Di Falco, Salvatore & Lensink, Robert, 2020. "Randomized interventions and “real” treatment effects: A cautionary tale and an example," World Development, Elsevier, vol. 127(C).
    • Handle: RePEc:eee:wdevel:v:127:y:2020:i:c:s0305750x19304395
    DOI: 10.1016/j.worlddev.2019.104790
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0305750X19304395
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.worlddev.2019.104790?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lori Beaman & Dean Karlan & Bram Thuysbaert & Christopher Udry, 2013. "Profitability of Fertilizer: Experimental Evidence from Female Rice Farmers in Mali," American Economic Review, American Economic Association, vol. 103(3), pages 381-386, May.
    2. Luigi Butera & John List, 2017. "An Economic Approach to Alleviate the Crisis of Confidence in Science: With an Application to the Public Goods Game," Artefactual Field Experiments 00608, The Field Experiments Website.
    3. Sylvain Chassang & Erik Snowberg & Ben Seymour & Cayley Bowles, 2015. "Accounting for Behavior in Treatment Effects: New Applications for Blind Trials," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-13, June.
    4. Beaman, Lori & Karlan, Dean & Thuysbaert, Bram & Udry, Christopher, 2013. "Probability of Fertilizer: Experimental Evidence from Female Rice Farmers in Mali," Working Papers 111, Yale University, Department of Economics.
    5. Christopher B. Barrett & Michael R. Carter, 2010. "The Power and Pitfalls of Experiments in Development Economics: Some Non-random Reflections," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 32(4), pages 515-548.
    6. Erwin Bulte & Gonne Beekman & Salvatore Di Falco & Joseph Hella & Pan Lei, 2014. "Behavioral Responses and the Impact of New Agricultural Technologies: Evidence from a Double-blind Field Experiment in Tanzania," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 96(3), pages 813-830.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Olivia Bertelli, 2020. "Investing in Agriculture when it is worth it. Empirical evidence from rural Uganda," Working Papers hal-02446820, HAL.
    2. Annemie Maertens & Hope Michelson & Vesall Nourani, 2021. "How Do Farmers Learn from Extension Services? Evidence from Malawi," American Journal of Agricultural Economics, John Wiley & Sons, vol. 103(2), pages 569-595, March.
    3. Holden, Stein T. & Westberg, Nina Bruvik, 2016. "Exploring technology use under climate risk and shocks through an experimental lens," African Journal of Agricultural and Resource Economics, African Association of Agricultural Economists, vol. 11(1), pages 1-16, March.
    4. Olivia Bertelli, 2019. "Investing in agriculture when it is worth it. Evidence from rural Uganda," Working Papers DT/2019/05, DIAL (Développement, Institutions et Mondialisation).
    5. Sylvain Chassang & Erik Snowberg & Ben Seymour & Cayley Bowles, 2015. "Accounting for Behavior in Treatment Effects: New Applications for Blind Trials," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-13, June.
    6. Michelson, Hope & Gourlay, Sydney & Lybbert, Travis & Wollburg, Philip, 2023. "Review: Purchased agricultural input quality and small farms," Food Policy, Elsevier, vol. 116(C).
    7. Jayne, T.S. & Mason, Nicole M. & Burke, William J. & Ariga, Joshua, 2016. "Agricultural Input Subsidy Programs In Africa: An Assessment Of Recent Evidence," Feed the Future Innovation Lab for Food Security Policy Research Papers 259509, Michigan State University, Department of Agricultural, Food, and Resource Economics, Feed the Future Innovation Lab for Food Security (FSP).
    8. Matsumoto, Tomoya, 2014. "Disseminating new farming practices among small scale farmers: An experimental intervention in Uganda," Journal of the Japanese and International Economies, Elsevier, vol. 33(C), pages 43-74.
    9. Paola Mallia, 2022. "You reap what (you think) you sow? Evidence on farmers’behavioral adjustments in the case of correct crop varietal identification," Working Papers hal-03597332, HAL.
    10. Federico A. Bugni & Ivan A. Canay & Steve McBride, 2023. "Decomposition and Interpretation of Treatment Effects in Settings with Delayed Outcomes," Papers 2302.11505, arXiv.org, revised Sep 2024.
    11. Jérémie Gignoux & Karen Macours & Daniel Stein & Kelsey Wright, 2021. "Agricultural input subsidies, credit constraints and expectations of future transfers: evidence from Haiti," Working Papers halshs-03131411, HAL.
    12. Nazziwa-Nviiri, Lydia & Van Campenhout, Bjorn & Amwonya, David, 2017. "Stimulating agricultural technology adoption: Lessons from fertilizer use among Ugandan potato farmers," IFPRI discussion papers 1608, International Food Policy Research Institute (IFPRI).
    13. Voors, Maarten & Demont, Matty & Bulte, Erwin, 2016. "New Experiments in Agriculture," African Journal of Agricultural and Resource Economics, African Association of Agricultural Economists, vol. 11(1), pages 1-7, March.
    14. Toba Stephen Olasehinde & Fangbin Qiao & Shiping Mao, 2023. "Impact of Improved Maize Varieties on Production Efficiency in Nigeria: Separating Technology from Managerial Gaps," Agriculture, MDPI, vol. 13(3), pages 1-14, March.
    15. Levine, N. Kendra & Mason, Nicole M. & Morgan, Stephen N., 2016. "Do input subsidies crowd in or crowd out other soil fertility management practices? Panel survey evidence from Zambia," 2016 Fifth International Conference, September 23-26, 2016, Addis Ababa, Ethiopia 246393, African Association of Agricultural Economists (AAAE).
    16. Bloem, Jeffrey R. & Liverpool-Tasie, Saweda & Adjognon, Serge G. & Dillon, Andrew, 2022. "Private Sector Promotion of Climate-Smart Technologies: Experimental Evidence from Nigeria," 2022 Annual Meeting, July 31-August 2, Anaheim, California 322152, Agricultural and Applied Economics Association.
    17. Abhijit Banerjee & Sylvain Chassang & Erik Snowberg, 2016. "Decision Theoretic Approaches to Experiment Design and External Validity," NBER Working Papers 22167, National Bureau of Economic Research, Inc.
    18. Teresa Molina Millán & Karen Macours, 2017. "Attrition in randomized control trials: Using tracking information to correct bias," FEUNL Working Paper Series novaf:wp1702, Universidade Nova de Lisboa, Faculdade de Economia.
    19. Bold, Tessa & Kaizzi, Kayuki C. & Svensson, Jakob & Yanagizawa-Drott, David, 2015. "Low Quality, Low Returns, Low Adoption: Evidence from the Market for Fertilizer and Hybrid Seed in Uganda," Working Paper Series rwp15-033, Harvard University, John F. Kennedy School of Government.
    20. Amy Copley & Alison Decker & Fannie Delavelle & Markus Goldstein & Michael O'Sullivan & Sreelakshmi Papineni, 2020. "COVID-19 Pandemic Through a Gender Lens," World Bank Publications - Reports 34016, The World Bank Group.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:wdevel:v:127:y:2020:i:c:s0305750x19304395. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/worlddev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.