IDEAS home Printed from https://ideas.repec.org/p/tin/wpaper/20050017.html
   My bibliography  Save this paper

Measuring Externalities in Program Evaluation

Author

Listed:
  • Wendy Janssens

    (Faculty of Economics and Business Administration, Vrije Universiteit Amsterdam)

Abstract

Impact evaluations of development programmes usually focus on a comparison of participants with a control group. However, if the programme generates externalities for non-participants such an approach will capture only part of the programme’s impact. Based on a unique large-scale quantitative survey we estimate the direct as well as the spillover effects of a women’s empowerment programme in rural India on child immunization and school enrolment. The survey covers both participants and non-participants living in programme villages, as well as respondents in control villages where the programme is not yet active. We account for participation selection bias using instrumental variables. The control villages allow us to test the exclusion restriction and provide us with an effective control group to analyze programme impact. We find both direct effects and significant spillovers on non-participants. The impact of interventions might be substantially underestimated if such external effects were not taken into account.

Suggested Citation

  • Wendy Janssens, 2005. "Measuring Externalities in Program Evaluation," Tinbergen Institute Discussion Papers 05-017/2, Tinbergen Institute, revised 30 Mar 2006.
    • Handle: RePEc:tin:wpaper:20050017
    as

    Download full text from publisher

    File URL: https://papers.tinbergen.nl/05017.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Foster, Andrew D & Rosenzweig, Mark R, 1995. "Learning by Doing and Learning from Others: Human Capital and Technical Change in Agriculture," Journal of Political Economy, University of Chicago Press, vol. 103(6), pages 1176-1209, December.
    2. Kim, Jooseop & Alderman, Harold & Orazem, Peter F, 1999. "Can Private School Subsidies Increase Enrollment for the Poor? The Quetta Urban Fellowship Program," The World Bank Economic Review, World Bank, vol. 13(3), pages 443-465, September.
    3. Charles F. Manski, 2000. "Economic Analysis of Social Interactions," Journal of Economic Perspectives, American Economic Association, vol. 14(3), pages 115-136, Summer.
    4. Sarmistha Pal, 2004. "How Much of the Gender Difference in Child School Enrolment Can Be Explained? Evidence from Rural India," Bulletin of Economic Research, Wiley Blackwell, vol. 56(2), pages 133-158, April.
    5. Esther Duflo & Emmanuel Saez, 2003. "The Role of Information and Social Interactions in Retirement Plan Decisions: Evidence from a Randomized Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 815-842.
    6. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    7. Ghazala Mansuri, 2004. "Community-Based and -Driven Development: A Critical Review," The World Bank Research Observer, World Bank, vol. 19(1), pages 1-39.
    8. Case, A.C. & Katz, L.F., 1991. "The Company You Keep: The Effects Of Family And Neighborhood On Disadvantaged Younths," Harvard Institute of Economic Research Working Papers 1555, Harvard - Institute of Economic Research.
    9. repec:pri:rpdevs:besley_case_diffusion.pdf is not listed on IDEAS
    10. Richard Blundell & Monica Costa Dias, 2000. "Evaluation methods for non-experimental data," Fiscal Studies, Institute for Fiscal Studies, vol. 21(4), pages 427-468, January.
    11. Dreze, Jean & Sen, Amartya, 2002. "India: Development and Participation," OUP Catalogue, Oxford University Press, edition 2, number 9780199257492.
    12. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    13. Munshi, Kaivan, 2004. "Social learning in a heterogeneous population: technology diffusion in the Indian Green Revolution," Journal of Development Economics, Elsevier, vol. 73(1), pages 185-213, February.
    14. Charles F. Manski, 1993. "Identification of Endogenous Social Effects: The Reflection Problem," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 60(3), pages 531-542.
    15. Jalan, Jyotsna & Ravallion, Martin, 2003. "Does piped water reduce diarrhea for children in rural India?," Journal of Econometrics, Elsevier, vol. 112(1), pages 153-173, January.
    16. Timothy Besley & Anne Case, 1994. "Diffusion as a Learning Process: Evidence from HYV Cotton," Working Papers 228, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    17. Roberto Agodini & Mark Dynarski, "undated". "Are Experiments the Only Option? A Look at Dropout Prevention Programs," Mathematica Policy Research Reports 51241adbf9fa4a26add6d54c5, Mathematica Policy Research.
    18. Borooah, Vani K., 2004. "On the incidence of diarrhoea among young Indian children," Economics & Human Biology, Elsevier, vol. 2(1), pages 119-138, March.
    19. James Heckman, 1997. "Instrumental Variables: A Study of Implicit Behavioral Assumptions Used in Making Program Evaluations," Journal of Human Resources, University of Wisconsin Press, vol. 32(3), pages 441-462.
    20. Agarwal, Bina, 2001. "Participatory Exclusions, Community Forestry, and Gender: An Analysis for South Asia and a Conceptual Framework," World Development, Elsevier, vol. 29(10), pages 1623-1648, October.
    21. Rohini Pande, 2003. "Selective gender differences in childhood nutrition and immunization in rural India: The role of siblings," Demography, Springer;Population Association of America (PAA), vol. 40(3), pages 395-418, August.
    22. Deon Filmer & Lant Pritchett, 2001. "Estimating Wealth Effects Without Expenditure Data—Or Tears: An Application To Educational Enrollments In States Of India," Demography, Springer;Population Association of America (PAA), vol. 38(1), pages 115-132, February.
    23. Filmer, Deon & King, Elizabeth M. & Pritchett, Lant, 1998. "Gender disparity in South Asia : comparisons between and within countries," Policy Research Working Paper Series 1867, The World Bank.
    24. Roberto Agodini & Mark Dynarski, 2004. "Are Experiments the Only Option? A Look at Dropout Prevention Programs," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 180-194, February.
    25. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    26. repec:pri:rpdevs:besley_case_diffusion is not listed on IDEAS
    27. Lawrence F. Katz & Jeffrey R. Kling & Jeffrey B. Liebman, 2001. "Moving to Opportunity in Boston: Early Results of a Randomized Mobility Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(2), pages 607-654.
    28. Alderman, Harold & Kim, Jooseop & Orazem, Peter F., 2003. "Design, evaluation, and sustainability of private schools for the poor: the Pakistan urban and rural fellowship school experiments," Economics of Education Review, Elsevier, vol. 22(3), pages 265-274, June.
    29. Janet Currie, 2001. "Early Childhood Education Programs," Journal of Economic Perspectives, American Economic Association, vol. 15(2), pages 213-238, Spring.
    30. Jere R. Behrman & Yingmei Cheng & Petra E. Todd, 2004. "Evaluating Preschool Programs When Length of Exposure to the Program Varies: A Nonparametric Approach," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 108-132, February.
    31. Bruce Sacerdote, 2001. "Peer Effects with Random Assignment: Results for Dartmouth Roommates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(2), pages 681-704.
    32. Jean Drèze & Mamta Murthi, 2001. "Fertility, Education, and Development: Evidence from India," Population and Development Review, The Population Council, Inc., vol. 27(1), pages 33-63, March.
    33. Paul Glewwe, 2002. "Schools and Skills in Developing Countries: Education Policies and Socioeconomic Outcomes," Journal of Economic Literature, American Economic Association, vol. 40(2), pages 436-482, June.
    34. Hoff, Karla & Pandey, Priyanka, 2004. "Belief systems and durable inequalities : an experimental investigation of Indian caste," Policy Research Working Paper Series 3351, The World Bank.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Milla Nyyssölä & Jukka Pirttilä & Susanna Sandström, 2012. "Helping Poor Farmers to Help Themselves: Evidence from a Group-Based Aid Project in Mozambique," WIDER Working Paper Series wp-2012-088, World Institute for Development Economic Research (UNU-WIDER).
    2. Olivia D�Aoust & Olivier Sterck & Philip Verwimp, 2013. "Buying Peace: The Mirage of Demobilizing Rebels," HiCN Working Papers 145, Households in Conflict Network.
    3. Balineau, Gaëlle, 2013. "Disentangling the Effects of Fair Trade on the Quality of Malian Cotton," World Development, Elsevier, vol. 44(C), pages 241-255.
    4. Ruerd Ruben, 2012. "Dimensionner l'aide au développement : ce que nous enseigne l'évaluation. Dimensioning Development Aid: Some Lessons from Evaluation," Revue d’économie du développement, De Boeck Université, vol. 20(4), pages 95-123.
    5. Nyyssölä, Milla & Pirttilä, Jukka & Sandström, Susanna, 2012. "Helping Poor Farmers to Help Themselves: Evidence from a Group-Based Aid Project in Mozambique," WIDER Working Paper Series 088, World Institute for Development Economic Research (UNU-WIDER).
    6. repec:unu:wpaper:wp2012-88 is not listed on IDEAS
    7. Dekker, Marlieen, 2008. "Estimating Wealth effects without expenditure Data: Evidence From rural Ethiopia," Ethiopian Journal of Economics, Ethiopian Economics Association, vol. 15(1), pages 1-85, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Kling, Jeffrey & Liebman, Jeffrey, 2004. "Experimental Analysis of Neighborhood Effects on Youth," Working Paper Series rwp04-034, Harvard University, John F. Kennedy School of Government.
    3. Bonan, Jacopo & Battiston, Pietro & Bleck, Jaimie & LeMay-Boucher, Philippe & Pareglio, Stefano & Sarr, Bassirou & Tavoni, Massimo, 2021. "Social interaction and technology adoption: Experimental evidence from improved cookstoves in Mali," World Development, Elsevier, vol. 144(C).
    4. John Beshears & James J. Choi & David Laibson & Brigitte C. Madrian & Katherine L. Milkman, 2015. "The Effect of Providing Peer Information on Retirement Savings Decisions," Journal of Finance, American Finance Association, vol. 70(3), pages 1161-1201, June.
    5. Glewwe, Paul & Kremer, Michael, 2006. "Schools, Teachers, and Education Outcomes in Developing Countries," Handbook of the Economics of Education, in: Erik Hanushek & F. Welch (ed.), Handbook of the Economics of Education, edition 1, volume 2, chapter 16, pages 945-1017, Elsevier.
    6. Justine Burns & Malcolm Kewsell & Rebecca Thornton, 2009. "Evaluating the Impact of Health Programmes," SALDRU Working Papers 40, Southern Africa Labour and Development Research Unit, University of Cape Town.
    7. Fishman, Arthur & Fishman, Ram & Gneezy, Uri, 2019. "A tale of two food stands: Observational learning in the field," Journal of Economic Behavior & Organization, Elsevier, vol. 159(C), pages 101-108.
    8. Antonia Grohmann & Sahra Sakha, 2015. "The Effect of Peer Observation on Consumption Choices: Experimental Evidence," Discussion Papers of DIW Berlin 1525, DIW Berlin, German Institute for Economic Research.
    9. Patrick Bayer & Randi Hjalmarsson & David Pozen, 2009. "Building Criminal Capital behind Bars: Peer Effects in Juvenile Corrections," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 124(1), pages 105-147.
    10. Kandpal, Eeshani & Baylis, Kathy, 2019. "The social lives of married women: Peer effects in female autonomy and investments in children," Journal of Development Economics, Elsevier, vol. 140(C), pages 26-43.
    11. Grodner, Andrew & Kniesner, Thomas J. & Bishop, John A., 2011. "Social Interactions in the Labor Market," Foundations and Trends(R) in Microeconomics, now publishers, vol. 6(4), pages 265-366, September.
    12. Piero Cipollone & Alfonso Rosolia, 2007. "Social Interactions in High School: Lessons from an Earthquake," American Economic Review, American Economic Association, vol. 97(3), pages 948-965, June.
    13. Leonardo Bursztyn & Florian Ederer & Bruno Ferman & Noam Yuchtman, 2014. "Understanding Mechanisms Underlying Peer Effects: Evidence From a Field Experiment on Financial Decisions," Econometrica, Econometric Society, vol. 82(4), pages 1273-1301, July.
    14. Adriaan R. Soetevent, 2006. "Empirics of the Identification of Social Interactions; An Evaluation of the Approaches and Their Results," Journal of Economic Surveys, Wiley Blackwell, vol. 20(2), pages 193-228, April.
    15. Ravallion, Martin, 2008. "Evaluating Anti-Poverty Programs," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 59, pages 3787-3846, Elsevier.
    16. Feng, Yao, 2011. "Local spillovers and learning from neighbors: Evidence from durable adoptions in rural China," MPRA Paper 33924, University Library of Munich, Germany.
    17. Bet Caeyers, 2014. "Peer effects in development programme awareness of vulnerable groups in rural Tanzania," CSAE Working Paper Series 2014-11, Centre for the Study of African Economies, University of Oxford.
    18. Armin Falk & Urs Fischbacher & Simon G�chter, "undated". "Living in Two Neighborhoods - Social Interactions in the Lab," IEW - Working Papers 150, Institute for Empirical Research in Economics - University of Zurich.
    19. Muzhe Yang & Hsien-Ming Lien & Shin-Yi Chou, 2014. "Is There A Physician Peer Effect? Evidence From New Drug Prescriptions," Economic Inquiry, Western Economic Association International, vol. 52(1), pages 116-137, January.
    20. Orazem, Peter F. & King, Elizabeth M., 2008. "Schooling in Developing Countries: The Roles of Supply, Demand and Government Policy," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 55, pages 3475-3559, Elsevier.

    More about this item

    Keywords

    program evaluation; externalities; education; immunization; India; women's empowerment;
    All these keywords.

    JEL classification:

    • I12 - Health, Education, and Welfare - - Health - - - Health Behavior
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development
    • Z13 - Other Special Topics - - Cultural Economics - - - Economic Sociology; Economic Anthropology; Language; Social and Economic Stratification
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tin:wpaper:20050017. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tinbergen Office +31 (0)10-4088900 (email available below). General contact details of provider: https://edirc.repec.org/data/tinbenl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.