IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v27y2003i4p359-394.html
   My bibliography  Save this article

Explaining Variation in the Effects of Welfare-To-Work Programs

Author

Listed:
  • David Greenberg
  • Robert Meyer
  • Charles Michalopoulos
  • Michael Wiseman

Abstract

Evaluations of government-funded training programs often combine results from similar operations in multiple sites. Findings inevitably vary. It is common to relate site-to-site variations in outcomes to variations in program design, participant characteristics, and the local environment. Frequently, such connections are constructed in a narrative synthesis of multisite results. This article uses findings from the evaluations of California's Greater Avenues for Independence (GAIN) program and the National Evaluation of Welfare-to-Work Strategies (NEWWS) to illustrate why it is important to question the legitimacy of such syntheses. The discussion is carried out using a simple multilevel evaluation model that incorporates models of both individual outcomes within sites and variation in program effects across sites. The results indicate that tempting generalizations about GAIN and NEWWS effects are statistically unjustified but that significant progress might be made in identifying the determinants of program effects in future demonstrations with some changes in evaluation strategy.

Suggested Citation

  • David Greenberg & Robert Meyer & Charles Michalopoulos & Michael Wiseman, 2003. "Explaining Variation in the Effects of Welfare-To-Work Programs," Evaluation Review, , vol. 27(4), pages 359-394, August.
    • Handle: RePEc:sae:evarev:v:27:y:2003:i:4:p:359-394
    DOI: 10.1177/0193841X03254347
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X03254347
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X03254347?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Amemiya, Takeshi, 1978. "A Note on a Random Coefficients Model," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 19(3), pages 793-796, October.
    2. Stephen H. Bell & Larry L. Orr, 1994. "Is Subsidized Employment Cost Effective for Welfare Recipients? Experimental Evidence from Seven State Demonstrations," Journal of Human Resources, University of Wisconsin Press, vol. 29(1), pages 42-61.
    3. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    4. David Greenberg & Marvin Mandell & Matthew Onstott, 2000. "The dissemination and utilization of welfare-to-work experiments in state policymaking," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 19(3), pages 367-382.
    5. Gary Burtless & Larry L. Orr, 1986. "Are Classical Experiments Needed for Manpower Policy," Journal of Human Resources, University of Wisconsin Press, vol. 21(4), pages 606-639.
    6. David H. Greenberg & Charles Michalopoulos & Philip K. Robins, 2003. "A Meta-Analysis of Government-Sponsored Training Programs," ILR Review, Cornell University, ILR School, vol. 57(1), pages 31-53, October.
    7. Daniel Friedlander & David H. Greenberg & Philip K. Robins, 1997. "Evaluating Government Training Programs for the Economically Disadvantaged," Journal of Economic Literature, American Economic Association, vol. 35(4), pages 1809-1855, December.
    8. Robert J. LaLonde, 1995. "The Promise of Public Sector-Sponsored Training Programs," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 149-168, Spring.
    9. David Greenberg & Robert H. Meyer & Michael Wiseman, 1994. "Multisite Employment and Training Program Evaluations: A Tale of Three Studies," ILR Review, Cornell University, ILR School, vol. 47(4), pages 679-691, July.
    10. Gary Burtless, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jeounghee Kim, 2010. "Welfare-to-Work Programs and the Dynamics of TANF Use," Journal of Family and Economic Issues, Springer, vol. 31(2), pages 198-211, June.
    2. Piyushimita Thakuriah (Vonu), 2011. "Variations in employment transportation outcomes: Role of site‐level factors," Papers in Regional Science, Wiley Blackwell, vol. 90(4), pages 755-772, November.
    3. Foster, E. Michael & Stephens, Robert & Krivelyova, Anna & Gamfi, Phyllis, 2007. "Can system integration improve mental health outcomes for children and youth?," Children and Youth Services Review, Elsevier, vol. 29(10), pages 1301-1319, October.
    4. David H. Greenberg & Charles Michalopoulos & Philip K. Robin, 2006. "Do experimental and nonexperimental evaluations give different answers about the effectiveness of government-funded training programs?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 25(3), pages 523-552.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael Lechner, 2000. "An Evaluation of Public-Sector-Sponsored Continuous Vocational Training Programs in East Germany," Journal of Human Resources, University of Wisconsin Press, vol. 35(2), pages 347-375.
    2. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    3. Michael Lechner, 2002. "Mikroökonometrische Evaluation arbeitsmarktpolitischer Massnahmen," University of St. Gallen Department of Economics working paper series 2002 2002-20, Department of Economics, University of St. Gallen.
    4. Victor R. Fuchs & Alan B. Krueger & James M. Poterba, 1997. "Why do Economists Disagree About Policy?," NBER Working Papers 6151, National Bureau of Economic Research, Inc.
    5. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    6. Fredrik Andersson & Harry J. Holzer & Julia I. Lane & David Rosenblum & Jeffrey Smith, 2024. "Does Federally Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms," Journal of Human Resources, University of Wisconsin Press, vol. 59(4), pages 1244-1283.
    7. Eichler, Martin & Lechner, Michael, 1996. "Public Sector Sponsored Continuous Vocational Training in East Germany : Institutional Arrangements, Participants, and Results of Empirical Evaluations," Discussion Papers 549, Institut fuer Volkswirtschaftslehre und Statistik, Abteilung fuer Volkswirtschaftslehre.
    8. Smith, Jeffrey, 2000. "Evaluation aktiver Arbeitsmarktpolitik : Erfahrungen aus Nordamerika (Evaluating Avtive Labor Market Policies : Lessons from North America)," Mitteilungen aus der Arbeitsmarkt- und Berufsforschung, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany], vol. 33(3), pages 345-356.
    9. Aakvik, Arild & Heckman, James J. & Vytlacil, Edward J., 2005. "Estimating treatment effects for discrete outcomes when responses to treatment vary: an application to Norwegian vocational rehabilitation programs," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 15-51.
    10. Jeffrey Smith, 2000. "A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 136(III), pages 247-268, September.
    11. Jeffrey A. Smith, 2018. "The usefulness of experiments," IZA World of Labor, Institute of Labor Economics (IZA), pages 436-436, May.
    12. Smith, Jeffrey, 2000. "Evaluation aktiver Arbeitsmarktpolitik : Erfahrungen aus Nordamerika (Evaluating Avtive Labor Market Policies : Lessons from North America)," Mitteilungen aus der Arbeitsmarkt- und Berufsforschung, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany], vol. 33(3), pages 345-356.
    13. Michael Lechner, 1999. "Nonparametric bounds on employment and income effects of continuous vocational training in East Germany," Econometrics Journal, Royal Economic Society, vol. 2(1), pages 1-28.
    14. Arild Aakvik & James J. Heckman & Edward J. Vytlacil, 2000. "Treatment Effects for Discrete Outcomes when Responses to Treatment Vary Among Observationally Identical Persons: An Application to Norwegian ..," NBER Technical Working Papers 0262, National Bureau of Economic Research, Inc.
    15. Hujer, Reinhard & Caliendo, Marco, 2000. "Evaluation of Active Labour Market Policy: Methodological Concepts and Empirical Estimates," IZA Discussion Papers 236, Institute of Labor Economics (IZA).
    16. Donal O'Neill, 2000. "Evaluating Labour Market Interventions," Economics Department Working Paper Series n990300, Department of Economics, National University of Ireland - Maynooth.
    17. Jonathan Morduch, 1999. "The Microfinance Promise," Journal of Economic Literature, American Economic Association, vol. 37(4), pages 1569-1614, December.
    18. Raaum, Oddbjorn & Torp, Hege, 2002. "Labour market training in Norway--effect on earnings," Labour Economics, Elsevier, vol. 9(2), pages 207-247, April.
    19. Pablo Ibarrán & David Rosas & Yuri Suarez Dillon Soares, 2006. "Impact Evaluation of a Youth Job Training Program in the Dominican Republic:Ex-Post Evaluation Report of the Labor Training and Modernization Project (DR0134)," OVE Working Papers 0306, Inter-American Development Bank, Office of Evaluation and Oversight (OVE).
    20. Carol Harvey & Michael J. Camasso & Radha Jagannathan, 2000. "Evaluating Welfare Reform Waivers under Section 1115," Journal of Economic Perspectives, American Economic Association, vol. 14(4), pages 165-188, Fall.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:27:y:2003:i:4:p:359-394. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.