IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp9939.html
   My bibliography  Save this paper

When Can Experimental Evidence Mislead? A Re-Assessment of Canada's Self Sufficiency Project

Author

Listed:
  • Riddell, Chris

    (University of Waterloo)

  • Riddell, W. Craig

    (University of British Columbia, Vancouver)

Abstract

The Self-Sufficiency Project was a well-known welfare-to-work experiment that provided a generous but time-limited financial incentive to leave welfare and enter the workforce. Experimental evidence showed large short-term impacts but no lasting effects. We argue that these conclusions need to be re-assessed. Policy changes implemented during the SSP implied the behavior of the control group did not provide an appropriate counterfactual. We estimate the impacts the financial incentive would have had in a stable policy environment. This re-assessment leads to significant changes in the lessons previously reached. Our study demonstrates that experimental findings need to be interpreted with care.

Suggested Citation

  • Riddell, Chris & Riddell, W. Craig, 2016. "When Can Experimental Evidence Mislead? A Re-Assessment of Canada's Self Sufficiency Project," IZA Discussion Papers 9939, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp9939
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp9939.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Guy Lacroix & Dany Brouillette, 2011. "Assessing the impact of a wage subsidy for single parents on social assistance," Canadian Journal of Economics, Canadian Economics Association, vol. 44(4), pages 1195-1221, November.
    2. Blundell, Richard, 2006. "Earned income tax credit policies: Impact and optimality: The Adam Smith Lecture, 2005," Labour Economics, Elsevier, vol. 13(4), pages 423-443, August.
    3. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    4. Riddell, Chris & Riddell, W. Craig, 2014. "The pitfalls of work requirements in welfare-to-work policies: Experimental evidence on human capital accumulation in the Self-Sufficiency Project," Journal of Public Economics, Elsevier, vol. 117(C), pages 39-49.
    5. David Card & Dean R. Hyslop, 2005. "Estimating the Effects of a Time-Limited Earnings Subsidy for Welfare-Leavers," Econometrica, Econometric Society, vol. 73(6), pages 1723-1770, November.
    6. James Heckman & Neil Hohmann & Jeffrey Smith & Michael Khoo, 2000. "Substitution and Dropout Bias in Social Experiments: A Study of an Influential Social Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 115(2), pages 651-694.
    7. Jeremy Lise & Shannon Seitz & Jeffrey Smith, 2015. "Evaluating search and matching models using experimental data," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-35, December.
    8. Card, David & Hyslop, Dean R., 2009. "The dynamic effects of an earnings subsidy for long-term welfare recipients: Evidence from the self sufficiency project applicant experiment," Journal of Econometrics, Elsevier, vol. 153(1), pages 1-20, November.
    9. Charles Michalopoulos & Philip K. Robins, 2001. "Using financial incentives to encourage welfare recipients to become economically self-sufficient," Economic Policy Review, Federal Reserve Bank of New York, issue Sep, pages 105-123.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tue Gørgens & Dean Robert Hyslop, 2018. "The Specification of Dynamic Discrete-Time Two-State Panel Data Models," Econometrics, MDPI, vol. 7(1), pages 1-16, December.
    2. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    3. Sam Desiere & Bart Cockx, 2021. "How Effective Are Hiring Subsidies to Reduce Long-Term Unemployment among Prime-Aged Jobseekers? Evidence from Belgium," CESifo Working Paper Series 9325, CESifo.
    4. Nicholas-James Clavet & Jean-Yves Duclos & Guy Lacroix, 2013. "Fighting Poverty: Assessing the Effect of Guaranteed Minimum Income Proposals in Quebec," Canadian Public Policy, University of Toronto Press, vol. 39(4), pages 491-516, December.
    5. Rothstein, Jesse & Von Wachter, Till, 2016. "Social Experiments in the Labor Market," Department of Economics, Working Paper Series qt7957p9g6, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    6. Sebastian Galiani & Juan Pantano, 2021. "Structural Models: Inception and Frontier," NBER Working Papers 28698, National Bureau of Economic Research, Inc.
    7. Pablo Ibarrán & David Rosas & Yuri Suarez Dillon Soares, 2006. "Impact Evaluation of a Youth Job Training Program in the Dominican Republic:Ex-Post Evaluation Report of the Labor Training and Modernization Project (DR0134)," OVE Working Papers 0306, Inter-American Development Bank, Office of Evaluation and Oversight (OVE).
    8. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    9. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    10. Alan B. Krueger, 2002. "Inequality, Too Much of a Good Thing," Working Papers 845, Princeton University, Department of Economics, Industrial Relations Section..
    11. Charles Bellemare & Bruce Shearer, 2011. "On The Relevance And Composition Of Gifts Within The Firm: Evidence From Field Experiments," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 52(3), pages 855-882, August.
    12. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    13. Bruno Crépon & Gerard J. van den Berg, 2016. "Active Labor Market Policies," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 521-546, October.
    14. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    15. Sianesi, Barbara, 2017. "Evidence of randomisation bias in a large-scale social experiment: The case of ERA," Journal of Econometrics, Elsevier, vol. 198(1), pages 41-64.
    16. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    17. Kling, Jeffrey R., 2007. "Methodological Frontiers of Public Finance Field Experiments," National Tax Journal, National Tax Association;National Tax Journal, vol. 60(1), pages 109-127, March.
    18. repec:pri:cepsud:87krueger is not listed on IDEAS
    19. Albanese, Andrea & Cockx, Bart & Dejemeppe, Muriel, 2024. "Long-term effects of hiring subsidies for low-educated unemployed youths," Journal of Public Economics, Elsevier, vol. 235(C).
    20. Jeffrey Smith & Jeremy Lise & Shannon N. Seitz, 2003. "Equilibrium Policy Experiments And The Evaluation Of Social Programs," Working Paper 1012, Economics Department, Queen's University.
    21. Pieter Gautier & Paul Muller & Bas van der Klaauw & Michael Rosholm & Michael Svarer, 2018. "Estimating Equilibrium Effects of Job Search Assistance," Journal of Labor Economics, University of Chicago Press, vol. 36(4), pages 1073-1125.

    More about this item

    Keywords

    welfare-to-work policies; social experiments; Self-Sufficiency Project;
    All these keywords.

    JEL classification:

    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • H53 - Public Economics - - National Government Expenditures and Related Policies - - - Government Expenditures and Welfare Programs
    • I38 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Government Programs; Provision and Effects of Welfare Programs

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp9939. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.