IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/3386.html
   My bibliography  Save this paper

An evaluation of the performance of regression discontinuity design on PROGRESA

Author

Listed:
  • Buddelmeyer, Hielke
  • Skoufias, Emmanuel

Abstract

While providing the most reliable method of evaluating social programs, randomized experiments in developing and developed countries alike are accompanied by political risks and ethical issues that jeopardize the chances of adopting them. In this paper the authors use a unique data set from rural Mexico collected for the purposes of evaluating the impact of the PROGRESA poverty alleviation program to examine the performance of a quasi-experimental estimator, the Regression Discontinuity Design (RDD). Using as a benchmark the impact estimates based on the experimental nature of the sample, we examine how estimates differ when we use the RDD as the estimator for evaluating program impact on two key indicators: child school attendance and child work. Overall the performance of the RDD was remarkably good. The RDD estimates of program impact agreed with the experimental estimates in 10 out of the 12 possible cases. The two cases in which the RDD method failed to reveal any significant program impact on the school attendance of boys and girls were in the first year of the program (round 3). RDD estimates comparable to the experimental estimates were obtained when we used as a comparison group children from non-eligible households in the control localities.

Suggested Citation

  • Buddelmeyer, Hielke & Skoufias, Emmanuel, 2004. "An evaluation of the performance of regression discontinuity design on PROGRESA," Policy Research Working Paper Series 3386, The World Bank.
    • Handle: RePEc:wbk:wbrwps:3386
    as

    Download full text from publisher

    File URL: http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2004/09/09/000009486_20040909125120/Rendered/PDF/wps3386PROGRESA.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Emmanuel Skoufias & Susan Wendy Parker, 2001. "Conditional Cash Transfers and Their Impact on Child Work and Schooling: Evidence from the PROGRESA Program in Mexico," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Fall 2001), pages 45-96, August.
    2. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    3. Petra E. Todd & Jeffrey A. Smith, 2001. "Reconciling Conflicting Evidence on the Performance of Propensity-Score Matching Methods," American Economic Review, American Economic Association, vol. 91(2), pages 112-118, May.
    4. Wilbert van der Klaauw, 2002. "Estimating the Effect of Financial Aid Offers on College Enrollment: A Regression-Discontinuity Approach," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 43(4), pages 1249-1287, November.
    5. Judy L. Baker, 2000. "Evaluating the Impact of Development Projects on Poverty : A Handbook for Practitioners," World Bank Publications - Books, The World Bank Group, number 13949.
    6. Jinyong Hahn & Petra Todd & Wilbert Van der Klaauw, 1999. "Evaluating the Effect of an Antidiscrimination Law Using a Regression-Discontinuity Design," NBER Working Papers 7131, National Bureau of Economic Research, Inc.
    7. Behrman, Jere R. & Deolalikar, Anil B., 1988. "Health and nutrition," Handbook of Development Economics, in: Hollis Chenery & T.N. Srinivasan (ed.), Handbook of Development Economics, edition 1, volume 1, chapter 14, pages 631-711, Elsevier.
    8. Paul Schultz, T., 2004. "School subsidies for the poor: evaluating the Mexican Progresa poverty program," Journal of Development Economics, Elsevier, vol. 74(1), pages 199-250, June.
    9. Schultz, T. Paul, 2001. "School subsidies for the poor," FCND briefs 102, International Food Policy Research Institute (IFPRI).
    10. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    11. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    12. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Ravallion, Martin, 2008. "Evaluating Anti-Poverty Programs," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 59, pages 3787-3846, Elsevier.
    3. Justine Burns & Malcolm Kewsell & Rebecca Thornton, 2009. "Evaluating the Impact of Health Programmes," SALDRU Working Papers 40, Southern Africa Labour and Development Research Unit, University of Cape Town.
    4. Markus Frölich, 2004. "Programme Evaluation with Multiple Treatments," Journal of Economic Surveys, Wiley Blackwell, vol. 18(2), pages 181-224, April.
    5. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    6. Djebbari, Habiba & Smith, Jeffrey, 2008. "Heterogeneous impacts in PROGRESA," Journal of Econometrics, Elsevier, vol. 145(1-2), pages 64-80, July.
    7. de Brauw, Alan & Hoddinott, John, 2011. "Must conditional cash transfer programs be conditioned to be effective? The impact of conditioning transfers on school enrollment in Mexico," Journal of Development Economics, Elsevier, vol. 96(2), pages 359-370, November.
    8. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    9. Mazzutti, Caio Cícero Toledo Piza da Costa, 2016. "Three essays on the causal impacts of child labour laws in Brazil," Economics PhD Theses 0616, Department of Economics, University of Sussex Business School.
    10. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    11. Bryson, Alex & Dorsett, Richard & Purdon, Susan, 2002. "The use of propensity score matching in the evaluation of active labour market policies," LSE Research Online Documents on Economics 4993, London School of Economics and Political Science, LSE Library.
    12. Antonio Trujillo & Jorge Portillo & John Vernon, 2005. "The Impact of Subsidized Health Insurance for the Poor: Evaluating the Colombian Experience Using Propensity Score Matching," International Journal of Health Economics and Management, Springer, vol. 5(3), pages 211-239, September.
    13. James J. Heckman, 2005. "Micro Data, Heterogeneity and the Evaluation of Public Policy Part 2," The American Economist, Sage Publications, vol. 49(1), pages 16-44, March.
    14. Stephan Thomsen, 2009. "Job Search Assistance Programs in Europe: Evaluation Methods and Recent Empirical Findings," FEMM Working Papers 09018, Otto-von-Guericke University Magdeburg, Faculty of Economics and Management.
    15. Raaum, Oddbjørn & Torp, Hege & Zhang, Tao, 2003. "Business cycles and the impact of labour market programmes," Memorandum 14/2002, Oslo University, Department of Economics.
    16. Essama-Nssah, B., 2006. "Propensity score matching and policy impact analysis - a demonstration in EViews," Policy Research Working Paper Series 3877, The World Bank.
    17. Dettmann, E. & Becker, C. & Schmeißer, C., 2011. "Distance functions for matching in small samples," Computational Statistics & Data Analysis, Elsevier, vol. 55(5), pages 1942-1960, May.
    18. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    19. Jeffrey Smith, 2000. "A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 136(III), pages 247-268, September.
    20. Hämäläinen, Kari & Ollikainen, Virve, 2004. "Differential Effects of Active Labour Market Programmes in the Early Stages of Young People's Unemployment," Research Reports 115, VATT Institute for Economic Research.

    More about this item

    Keywords

    Scientific Research&Science Parks; Public Health Promotion; Health Monitoring&Evaluation; Housing&Human Habitats; Anthropology; Scientific Research&Science Parks; Science Education; VN-Acb Mis -- IFC-00535908; Health Monitoring&Evaluation; Housing&Human Habitats;
    All these keywords.

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy
    • I32 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Measurement and Analysis of Poverty
    • J13 - Labor and Demographic Economics - - Demographic Economics - - - Fertility; Family Planning; Child Care; Children; Youth

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:3386. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.