IDEAS home Printed from https://ideas.repec.org/p/hit/hitcei/2013-02.html
   My bibliography  Save this paper

Substitution Bias and External Validity: Why an Innovative Anti-poverty Program Showed no Net Impact

Author

Listed:
  • Morduch, Jonathan
  • Ravi, Shamika
  • Bauchet, Jonathan

Abstract

The net impact of development interventions can depend on the availability of close substitutes to the intervention. We analyze a randomized trial of an innovative anti-poverty program in South India which provides “ultra-poor” households with inputs to create a new, sustainable livelihood. We find no statistically significant evidence of lasting net impact on consumption, income or asset accumulation. Instead, income from the new livelihood substituted for earnings from wage labor. A very similar intervention made a large difference elsewhere in South Asia, however, where wage labor alternatives were less compelling. The analysis highlights the roles of substitution bias and dropout bias in shaping evaluation results and delimiting external validity.

Suggested Citation

  • Morduch, Jonathan & Ravi, Shamika & Bauchet, Jonathan, 2013. "Substitution Bias and External Validity: Why an Innovative Anti-poverty Program Showed no Net Impact," CEI Working Paper Series 2013-02, Center for Economic Institutions, Institute of Economic Research, Hitotsubashi University.
    • Handle: RePEc:hit:hitcei:2013-02
    as

    Download full text from publisher

    File URL: https://hermes-ir.lib.hit-u.ac.jp/hermes/ir/re/28279/wp2013-2.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    2. Joshua D. Angrist & Alan B. Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 69-85, Fall.
    3. Kleibergen, Frank & Paap, Richard, 2006. "Generalized reduced rank tests using the singular value decomposition," Journal of Econometrics, Elsevier, vol. 133(1), pages 97-126, July.
    4. David McKenzie & Christopher Woodruff, 2008. "Experimental Evidence on Returns to Capital and Access to Finance in Mexico," The World Bank Economic Review, World Bank, vol. 22(3), pages 457-482, November.
    5. Matthieu Chemin, 2008. "The Benefits and Costs of Microfinance: Evidence from Bangladesh," Journal of Development Studies, Taylor & Francis Journals, vol. 44(4), pages 463-484, April.
    6. Manuela Angelucci & Dean Karlan & Jonathan Zinman, 2013. "Win Some Lose Some? Evidence from a Randomized Microcredit Program Placement Experiment by Compartamos Banco," Working Papers 1026, Economic Growth Center, Yale University.
    7. M. M. Pitt & S. R. Khandker, 2002. "Credit Programmes for the Poor and Seasonality in Rural Bangladesh," Journal of Development Studies, Taylor & Francis Journals, vol. 39(2), pages 1-24.
    8. B.D. McCullough & Kerry Anne McGeary & Teresa D. Harrison, 2008. "Do economics journal archives promote replicable research?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 41(4), pages 1406-1420, November.
    9. Christopher F Baum & Mark E. Schaffer & Steven Stillman, 2007. "Enhanced routines for instrumental variables/GMM estimation and testing," CERT Discussion Papers 0706, Centre for Economic Reform and Transformation, Heriot Watt University.
    10. Phillips, Peter C.B., 2006. "A Remark On Bimodality And Weak Instrumentation In Structural Equation Estimation," Econometric Theory, Cambridge University Press, vol. 22(5), pages 947-960, October.
    11. repec:pri:rpdevs:deaton_instruments_randomization_learning_all_04april_2010 is not listed on IDEAS
    12. Douglas Staiger & James H. Stock, 1997. "Instrumental Variables Regression with Weak Instruments," Econometrica, Econometric Society, vol. 65(3), pages 557-586, May.
    13. repec:pri:rpdevs:morduch_microfinance_poor is not listed on IDEAS
    14. Jonathan Morduch, 1999. "The Microfinance Promise," Journal of Economic Literature, American Economic Association, vol. 37(4), pages 1569-1614, December.
    15. Pitt, Mark M. & Khandker, Shahidur R., 2012. "Replicating replication : due diligence in Roodman and Morduch's replication of Pitt and Khandker (1998)," Policy Research Working Paper Series 6273, The World Bank.
    16. Abhijit V. Banerjee & Esther Duflo, 2009. "The Experimental Approach to Development Economics," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 151-178, May.
    17. Suresh de Mel & David McKenzie & Christopher Woodruff, 2009. "Returns to Capital in Microenterprises: Evidence from a Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 124(1), pages 423-423.
    18. Mark M. Pitt & Shahidur R. Khandker, 1998. "The Impact of Group-Based Credit Programs on Poor Households in Bangladesh: Does the Gender of Participants Matter?," Journal of Political Economy, University of Chicago Press, vol. 106(5), pages 958-996, October.
    19. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
    20. Orazio Attanasio & Britta Augsburg & Ralph De Haas & Emla Fitzsimons & Heike Harmgart, 2011. "Group lending or individual lending? Evidence from a randomised field experiment in Mongolia," IFS Working Papers W11/20, Institute for Fiscal Studies.
    21. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    22. David Roodman, 2011. "Fitting fully observed recursive mixed-process models with cmp," Stata Journal, StataCorp LP, vol. 11(2), pages 159-206, June.
    23. Manuela Angelucci, Dean Karlan, Jonathan Zinman, 2013. "Win Some Lose Some? Evidence from a Randomized Microcredit Program Placement Experiment by Compartamos Banco-Working Paper 330," Working Papers 330, Center for Global Development.
    24. Britta Augsburg & Ralph De Haas & Heike Harmgart & Costas Meghir, 2012. "Microfinance at the Microfinance at the margin: experimental evidence from Bosnia and Herzegovina vidence from Bosnia and Herzegovina," Working Papers 146, European Bank for Reconstruction and Development, Office of the Chief Economist.
    25. Augsburg, Britta & de Haas, Ralph & Harmgart, Heike & Meghir, Costas, 2014. "Microfinance at the margin: Experimental evidence from Bosnia and Herzegovina," Discussion Papers, Research Unit: Economics of Change SP II 2014-304, WZB Berlin Social Science Center.
    26. Joshua Angrist & Alan Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Working Papers 834, Princeton University, Department of Economics, Industrial Relations Section..
    27. Mark M. Pitt, 2012. "Gunfight at the Not OK Corral: Reply to ‘High Noon for Microfinance’," Journal of Development Studies, Taylor & Francis Journals, vol. 48(12), pages 1886-1891, December.
    28. C. A. Field & A. H. Welsh, 2007. "Bootstrapping clustered data," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(3), pages 369-390, June.
    29. James J. Heckman, 2000. "Causal Parameters and Policy Analysis in Economics: A Twentieth Century Retrospective," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 115(1), pages 45-97.
    30. Jonathan Morduch, 1998. "Does Microfinance Really Help the Poor? New Evidence from Flagship Programs in Bangladesh," Working Papers 198, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    31. Maren Duvendack & Richard Palmer-Jones, 2012. "High Noon for Microfinance Impact Evaluations: Re-investigating the Evidence from Bangladesh," Journal of Development Studies, Taylor & Francis Journals, vol. 48(12), pages 1864-1880, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kazushi Takahashi & Abu Shonchoy & Seiro Ito & Takashi Kurosaki, 2017. "How Does Contract Design Affect the Uptake of Microcredit among the Ultra-poor? Experimental Evidence from the River Islands of Northern Bangladesh," Journal of Development Studies, Taylor & Francis Journals, vol. 53(4), pages 530-547, April.
    2. Esther Gehrke & Michael Grimm, 2018. "Do Cows Have Negative Returns? The Evidence Revisited," Economic Development and Cultural Change, University of Chicago Press, vol. 66(4), pages 673-707.
    3. Emmanuel Owusu Oppong & Yu Baorong & Bruvine Orchidée Mazonga Mfoutou, 2024. "Microinsurance in Ghana: investigating the impact of Outreville's four-factor framework and firm and product characteristics on adoption," The Geneva Papers on Risk and Insurance - Issues and Practice, Palgrave Macmillan;The Geneva Association, vol. 49(3), pages 421-447, July.
    4. McHugh, Neil & Biosca, Olga & Donaldson, Cam, 2015. "Microfinance, health and randomised trials," Health Economics Working Paper Series 201501, Glasgow Caledonian University, Yunus Centre.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Maren Duvendack & Richard Palmer-Jones, 2013. "Replication of quantitative work in development studies: Experiences and suggestions," Progress in Development Studies, , vol. 13(4), pages 307-322, October.
    2. Dean Karlan & Adam Osman & Jonathan Zinman, 2013. "Follow the Money: Methods for Identifying Consumption and Investment Responses to a Liquidity Shock," NBER Working Papers 19696, National Bureau of Economic Research, Inc.
    3. Jaikishan Desai & Kristin Johnson & Alessandro Tarozzi, 2013. "On the Impact of Microcredit: Evidence from a Randomized Intervention in Rural Ethiopia," Working Papers 741, Barcelona School of Economics.
    4. Khandker, Shahidur R. & Samad, Hussain A., 2014. "Dynamic effects of microcredit in Bangladesh," Policy Research Working Paper Series 6821, The World Bank.
    5. David Roodman & Jonathan Morduch, 2014. "The Impact of Microcredit on the Poor in Bangladesh: Revisiting the Evidence," Journal of Development Studies, Taylor & Francis Journals, vol. 50(4), pages 583-604, April.
    6. Manuela Angelucci, Dean Karlan, Jonathan Zinman, 2013. "Win Some Lose Some? Evidence from a Randomized Microcredit Program Placement Experiment by Compartamos Banco-Working Paper 330," Working Papers 330, Center for Global Development.
    7. Manuela Angelucci & Dean Karlan & Jonathan Zinman, 2013. "Win Some Lose Some? Evidence from a Randomized Microcredit Program Placement Experiment by Compartamos Banco," Working Papers 1026, Economic Growth Center, Yale University.
    8. Stimpfle, Alexander & Stadelmann, David, 2015. "The Impact of Fundamental Development Factors on Different Income Groups: International Evidence," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113128, Verein für Socialpolitik / German Economic Association.
    9. Oriana Bandiera & Robin Burgess & Erika Deserranno & Ricardo Morel & Imran Rasul & Munshi Sulaiman & Jack Thiemel, 2022. "Microfinance and Diversification," Economica, London School of Economics and Political Science, vol. 89(S1), pages 239-275, June.
    10. Kadyrbek Sultakeev & Kamalbek Karymshakov & Burulcha Sulaimanova, 2018. "The Impact Of Microfinance On Entrepreneurship In Kyrgyzstan," Acta Oeconomica Pragensia, Prague University of Economics and Business, vol. 2018(2), pages 24-40.
    11. Duvendack, Maren & Palmer-Jones, Richard, 2011. "The microfinance of reproduction and the reproduction of microfinance: understanding the connections between microfinance, empowerment, contraception and fertility in Bangladesh in the 1990s," MPRA Paper 32384, University Library of Munich, Germany.
    12. Erhardt, Eva, 2017. "Microfinance beyond self-employment: Evidence for firms in Bulgaria," MPRA Paper 79294, University Library of Munich, Germany.
    13. Sefa K. Awaworyi, 2014. "The Impact of Microfinance Interventions: A Meta-analysis," Monash Economics Working Papers 03-14, Monash University, Department of Economics.
    14. Maren Duvendack & Richard Palmer-Jones, 2012. "High Noon for Microfinance Impact Evaluations: Re-investigating the Evidence from Bangladesh," Journal of Development Studies, Taylor & Francis Journals, vol. 48(12), pages 1864-1880, December.
    15. Pitt, Mark M. & Khandker, Shahidur R., 2012. "Replicating replication : due diligence in Roodman and Morduch's replication of Pitt and Khandker (1998)," Policy Research Working Paper Series 6273, The World Bank.
    16. Islam, Asadul, 2015. "Heterogeneous effects of microcredit: Evidence from large-scale programs in Bangladesh," Journal of Asian Economics, Elsevier, vol. 37(C), pages 48-58.
    17. Erhardt, Eva Christine, 2017. "Microfinance beyond self-employment: Evidence for firms in Bulgaria," Labour Economics, Elsevier, vol. 47(C), pages 75-95.
    18. Rahimi, Lutfi, 2015. "Does microfinance elevate poverty? Does family size matter in the provision of microcredit? Evidence from a randomised evaluation," MPRA Paper 73906, University Library of Munich, Germany.
    19. Abu S. Shonchoy, 2015. "Seasonal Migration and Microcredit During Agricultural Lean Seasons: Evidence from Northwest Bangladesh," The Developing Economies, Institute of Developing Economies, vol. 53(1), pages 1-26, March.
    20. David Roodman & James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2019. "Fast and wild: Bootstrap inference in Stata using boottest," Stata Journal, StataCorp LP, vol. 19(1), pages 4-60, March.

    More about this item

    JEL classification:

    • O1 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development
    • J2 - Labor and Demographic Economics - - Demand and Supply of Labor
    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General
    • I3 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hit:hitcei:2013-02. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Reiko Suzuki (email available below). General contact details of provider: https://edirc.repec.org/data/cehitjp.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.