IDEAS home Printed from https://ideas.repec.org/p/azt/cemmap/23-02.html
   My bibliography  Save this paper

The role of randomized field trials in social science research: a perspective from evaluations of reforms of social welfare programs

Author

Listed:
  • Robert Moffitt

Abstract

One of the areas of policy research where randomized field trials have been utilized mostintensively is welfare reform. Starting in the late 1960s with experimental tests of a negativeincome tax and continuing through current experimental tests of recent welfare reforms,randomized evaluations have played a strong and increasing role in informing policy. This paper reviews the record of these experiments and assesses the implications of that record for theuse of randomization. The review demonstrates that, while randomized field trials in the areaof welfare reform have been professionally conducted and well-run, and have yielded muchvaluable and credible information, their usefulness has been limited by a number of weaknesses,some of which are inherent in the method and some of which result from constraints imposed bythe political process. The conclusion is that randomized field trials have an important butlimited role to play in future welfare reform evaluations, and that it is essential that they besupplemented by nonexperimental research.

Suggested Citation

  • Robert Moffitt, 2002. "The role of randomized field trials in social science research: a perspective from evaluations of reforms of social welfare programs," CeMMAP working papers 23/02, Institute for Fiscal Studies.
    • Handle: RePEc:azt:cemmap:23/02
    DOI: 10.1920/wp.cem.2002.2302
    as

    Download full text from publisher

    File URL: https://www.cemmap.ac.uk/wp-content/uploads/2020/08/CWP2302.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.1920/wp.cem.2002.2302?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Moffitt, Robert, 1992. "Incentive Effects of the U.S. Welfare System: A Review," Journal of Economic Literature, American Economic Association, vol. 30(1), pages 1-61, March.
    2. David Card & Winston Lin & Philip Robins, 1997. "Would Financial Incentives for Leaving Welfare Lead Some People to Stay on Welfare Longer? An Experimental Evaluation of 'Entry Effects' in the Self-Sufficiency Project," Working Papers 759, Princeton University, Department of Economics, Industrial Relations Section..
    3. Robert A. Moffitt, 1996. "The effect of employment and training programs on entry and exit from the welfare caseload," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 15(1), pages 32-50.
    4. Michael J. Camasso & Radha Jagannathan & Carol Harvey & Mark Killingsworth, 2003. "The use of client surveys to gauge the threat of contamination in welfare reform experiments," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 22(2), pages 207-223.
    5. V. Joseph Hotz & Guido W. Imbens & Jacob A. Klerman, 2000. "The Long-Term Gains from GAIN: A Re-Analysis of the Impacts of the California GAIN Program," NBER Working Papers 8007, National Bureau of Economic Research, Inc.
    6. repec:fth:prinin:380 is not listed on IDEAS
    7. Carol Harvey & Michael J. Camasso & Radha Jagannathan, 2000. "Evaluating Welfare Reform Waivers under Section 1115," Journal of Economic Perspectives, American Economic Association, vol. 14(4), pages 165-188, Fall.
    8. David Card & Philip K. Robins & Winston Lin, 1998. "Would Financial Incentives for Leaving Welfare Lead Some People to Stay on Welfare Longer? An Experimental Evaluation of 'Entry Effects' in the SSP," NBER Working Papers 6449, National Bureau of Economic Research, Inc.
    9. Gary Burtless, 1986. "The work response to a guaranteed income: a survey of experimental evidence," Conference Series ; [Proceedings], Federal Reserve Bank of Boston, vol. 30, pages 22-59.
    10. Alicia H. Munnell, 1987. "Lessons from the income maintenance experiments: an overview," New England Economic Review, Federal Reserve Bank of Boston, issue May, pages 32-45.
    11. David Card & Winston Lin & Philip Robins, 1997. "Would Financial Incentives for Leaving Welfare Lead Some People to Stay on Welfare Longer? An Experimental Evaluation of 'Entry Effects' in the Self-Sufficiency Project," Working Papers 759, Princeton University, Department of Economics, Industrial Relations Section..
    12. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    13. repec:pri:indrel:dsp01rv042t06t is not listed on IDEAS
    14. Rebecca M. Blank, 2002. "Evaluating Welfare Reform in the United States," Journal of Economic Literature, American Economic Association, vol. 40(4), pages 1105-1166, December.
    15. Ashenfelter, Orley & Plant, Mark W, 1990. "Nonparametric Estimates of the Labor-Supply Effects of Negative Income Tax Programs," Journal of Labor Economics, University of Chicago Press, vol. 8(1), pages 396-415, January.
    16. repec:fth:prinin:259 is not listed on IDEAS
    17. V. Joseph Hotz & Guido W. Imbens & Julie H. Mortimer, 1999. "Predicting the Efficacy of Future Training Programs Using Past Experiences," NBER Technical Working Papers 0238, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hugo Benítez-Silva & Richard Disney & Sergi Jiménez-Martín, 2010. "Disability, capacity for work and the business cycle: an international perspective [Has the boom in incapacity benefit claimant numbers passed its peak?]," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 25(63), pages 483-536.
    2. Leventhal, Tama & Dupéré, Véronique, 2011. "Moving to Opportunity: Does long-term exposure to 'low-poverty' neighborhoods make a difference for adolescents?," Social Science & Medicine, Elsevier, vol. 73(5), pages 737-743, September.
    3. World Bank, 2007. "Poverty and Environment : Understanding Linkages at the Household Level," World Bank Publications - Books, The World Bank Group, number 6924.
    4. Kling, Jeffrey R., 2007. "Methodological Frontiers of Public Finance Field Experiments," National Tax Journal, National Tax Association;National Tax Journal, vol. 60(1), pages 109-127, March.
    5. Ravallion, Martin, 2008. "Evaluating Anti-Poverty Programs," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 59, pages 3787-3846, Elsevier.
    6. World Bank, 2007. "Poverty and Environment : Understanding Linkages at the Household Level," World Bank Publications - Reports 7744, The World Bank Group.
    7. Hugo Benitez-Silva & Moshe Buchinsky & John Rust, 2005. "Induced Entry Effects of a $1 for $2 Offset in SSDI Benefits," Department of Economics Working Papers 05-03, Stony Brook University, Department of Economics.
    8. Ted Joyce & Robert Kaestner & Sanders Korenman & Stanley Henshaw, 2004. "Family Cap Provisions and Changes in Births and Abortions," NBER Working Papers 10214, National Bureau of Economic Research, Inc.
    9. Nagy, Gyula & Micklewright, John, 2006. "Az álláskeresés ellenőrzése és a munkanélküliség időtartama. Egy társadalomtudományi kísérlet [Job-search monitoring and unemployment duration: evidence from a randomized control trial]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(7), pages 641-660.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rebecca M. Blank, 2002. "Evaluating Welfare Reform in the United States," Journal of Economic Literature, American Economic Association, vol. 40(4), pages 1105-1166, December.
    2. Fortin, Bernard, 1997. "Dépendance à l’égard de l’aide sociale et réforme de la sécurité du revenu," L'Actualité Economique, Société Canadienne de Science Economique, vol. 73(4), pages 557-573, décembre.
    3. Marianne P. Bitler & Jonah B. Gelbach & Hilary W. Hoynes, 2006. "What Mean Impacts Miss: Distributional Effects of Welfare Reform Experiments," American Economic Review, American Economic Association, vol. 96(4), pages 988-1012, September.
    4. Ted Joyce & Robert Kaestner & Sanders Korenman & Stanley Henshaw, 2004. "Family Cap Provisions and Changes in Births and Abortions," NBER Working Papers 10214, National Bureau of Economic Research, Inc.
    5. John V. Pepper, 2003. "Using Experiments to Evaluate Performance Standards: What Do Welfare-to-Work Demonstrations Reveal to Welfare Reformers?," Journal of Human Resources, University of Wisconsin Press, vol. 38(4).
    6. Lemieux, Thomas & Milligan, Kevin, 2008. "Incentive effects of social assistance: A regression discontinuity approach," Journal of Econometrics, Elsevier, vol. 142(2), pages 807-828, February.
    7. Dehejia, Rajeev H., 2005. "Program evaluation as a decision problem," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 141-173.
    8. Card, David & Robins, Philip K., 2005. "How important are "entry effects" in financial incentive programs for welfare recipients? Experimental evidence from the Self-Sufficiency Project," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 113-139.
    9. Donal O'Neill, 2000. "Evaluating Labour Market Interventions," Economics Department Working Paper Series n990300, Department of Economics, National University of Ireland - Maynooth.
    10. Haider, Steven J. & Klerman, Jacob Alex, 2005. "Dynamic properties of the welfare caseload," Labour Economics, Elsevier, vol. 12(5), pages 629-648, October.
    11. Cobb-Clark, Deborah A. & Dahmann, Sarah C. & Salamanca, Nicolás & Zhu, Anna, 2022. "Intergenerational disadvantage: Learning about equal opportunity from social assistance receipt," Labour Economics, Elsevier, vol. 79(C).
    12. Dave, Dhaval M. & Reichman, Nancy E. & Corman, Hope & Das, Dhiman, 2011. "Effects of welfare reform on vocational education and training," Economics of Education Review, Elsevier, vol. 30(6), pages 1399-1415.
    13. Joshua D. Angrist, 2004. "Treatment effect heterogeneity in theory and practice," Economic Journal, Royal Economic Society, vol. 114(494), pages 52-83, March.
    14. Neumark, David & Powers, Elizabeth T., 2005. "SSI, Labor Supply, and Migration," IZA Discussion Papers 1820, Institute of Labor Economics (IZA).
    15. Gibson, John & Norton, Douglas A. & White, Robert A., 2019. "The backward hustle: An experimental investigation of tax code notches and labor supply," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 432-445.
    16. Dhaval Dave & Hope Corman & Nancy Reichman, 2012. "Effects of Welfare Reform on Education Acquisition of Adult Women," Journal of Labor Research, Springer, vol. 33(2), pages 251-282, June.
    17. L. M. Mead, "undated". "Welfare policy: The administrative frontier," Institute for Research on Poverty Discussion Papers 1093-96, University of Wisconsin Institute for Research on Poverty.
    18. Francesconi, Marco & Rainer, Helmut & van der Klaauw, Wilbert, 2008. "Unintended Consequences of Welfare Reform: The Case of Divorced Parents," IZA Discussion Papers 3891, Institute of Labor Economics (IZA).
    19. Gary King & Emmanuela Gakidou & Nirmala Ravishankar & Ryan T. Moore & Jason Lakin & Manett Vargas & Martha María Téllez-Rojo & Juan Eugenio Hernández Ávila & Mauricio Hernández Ávila & Héctor Hernánde, 2007. "A “politically robust” experimental design for public policy evaluation, with application to the Mexican Universal Health Insurance program," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 26(3), pages 479-506.
    20. Blank, Rebecca M., 2002. "Can equity and efficiency complement each other?," Labour Economics, Elsevier, vol. 9(4), pages 451-468, September.

    More about this item

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • I3 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:azt:cemmap:23/02. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Dermot Watson (email available below). General contact details of provider: https://edirc.repec.org/data/ifsssuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.