IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v30y2006i4p379-429.html
   My bibliography  Save this article

Evaluating the Effectiveness of the Massachusetts Workforce Development System Using No-Shows as a Nonexperimental Comparison Group

Author

Listed:
  • Steven Raphael

    (Goldman School of Public Policy, University of California, Berkeley)

  • Michael A. Stoll

    (School of Public Policy and Social Research, University of California, Los Angeles)

Abstract

This article examines the effect of the Massachusetts workforce development system on the earnings of disadvantaged adults using nonexperimental data from the late 1990s. The authors construct a comparison sample for program participants using individuals who apply for and are offered services yet do not participate in a training program. They present a series of difference-in-difference estimates that make several alternative efforts to correct for selectivity bias, including econometric models that regression adjust for observable characteristics and fixed-effect models that adjust for time-invariant person effects. They also employ probabilistic matching techniques to more finely align the treatment and comparison samples. On average, program participants experienced 20% increases in annual earnings 1 year postintervention and 25% increases after 2 years. The authors uncover considerable heterogeneity in these effects, suggesting that the most difficult to serve and the most job ready benefit the least.

Suggested Citation

  • Steven Raphael & Michael A. Stoll, 2006. "Evaluating the Effectiveness of the Massachusetts Workforce Development System Using No-Shows as a Nonexperimental Comparison Group," Evaluation Review, , vol. 30(4), pages 379-429, August.
    • Handle: RePEc:sae:evarev:v:30:y:2006:i:4:p:379-429
    DOI: 10.1177/0193841X05282268
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X05282268
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X05282268?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ernst W. Stromsdorfer, 1968. "Determinants of Economic Success in Retraining the Unemployed: The West Virginia Experience," Journal of Human Resources, University of Wisconsin Press, vol. 3(2), pages 139-158.
    2. Stephen H. Bell & Larry L. Orr, 1994. "Is Subsidized Employment Cost Effective for Welfare Recipients? Experimental Evidence from Seven State Demonstrations," Journal of Human Resources, University of Wisconsin Press, vol. 29(1), pages 42-61.
    3. Ellwood, David T., 2000. "The Impact of the Earned Income Tax Credit and Social Policy Reforms on Work, Marriage, and Living Arrangements," National Tax Journal, National Tax Association, vol. 53(n. 4), pages 1063-1106, December.
    4. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    5. Burt S. Barnow, 1987. "The Impact of CETA Programs on Earnings: A Review of the Literature," Journal of Human Resources, University of Wisconsin Press, vol. 22(2), pages 157-193.
    6. Kevin Hollenbeck, 2003. "Net Impact Estimates of the Workforce Development System in Washington State," Upjohn Working Papers 03-92, W.E. Upjohn Institute for Employment Research.
    7. Kornfeld, Robert & Bloom, Howard S, 1999. "Measuring Program Impacts on Earnings and Employment: Do Unemployment Insurance Wage Reports from Employers Agree with Surveys of Individuals?," Journal of Labor Economics, University of Chicago Press, vol. 17(1), pages 168-197, January.
    8. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-660, November.
    9. Daniel Friedlander & David H. Greenberg & Philip K. Robins, 1997. "Evaluating Government Training Programs for the Economically Disadvantaged," Journal of Economic Literature, American Economic Association, vol. 35(4), pages 1809-1855, December.
    10. James J. Heckman & Jeffrey A. Smith, 1999. "The Pre-Program Earnings Dip and the Determinants of Participation in a Social Program: Implications for Simple Program Evaluation Strategies," NBER Working Papers 6983, National Bureau of Economic Research, Inc.
    11. Meyer, Bruce D, 1995. "Natural and Quasi-experiments in Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 151-161, April.
    12. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    13. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    14. Stephen H. Bell & Larry l. Orr & John D. Blomquist & Glen G. Cain, 1995. "Program Applicants as a Comparison Group in Evaluating Training Programs: Theory and a Test," Books from Upjohn Press, W.E. Upjohn Institute for Employment Research, number pacg.
    15. Ellwood, David T., 2000. "The Impact of the Earned Income Tax Credit and Social Policy Reforms on Work, Marriage, and Living Arrangements," National Tax Journal, National Tax Association;National Tax Journal, vol. 53(4), pages 1063-1106, December.
    16. Heckman, James J & Smith, Jeffrey A, 1999. "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies," Economic Journal, Royal Economic Society, vol. 109(457), pages 313-348, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lechner Michael & Miquel Ruth & Wunsch Conny, 2007. "The Curse and Blessing of Training the Unemployed in a Changing Economy: The Case of East Germany After Unification," German Economic Review, De Gruyter, vol. 8(4), pages 468-509, December.
    2. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    3. Michael Lechner, 2000. "An Evaluation of Public-Sector-Sponsored Continuous Vocational Training Programs in East Germany," Journal of Human Resources, University of Wisconsin Press, vol. 35(2), pages 347-375.
    4. Raaum, Oddbjorn & Torp, Hege, 2002. "Labour market training in Norway--effect on earnings," Labour Economics, Elsevier, vol. 9(2), pages 207-247, April.
    5. Regner, Hakan, 2002. "A nonexperimental evaluation of training programs for the unemployed in Sweden," Labour Economics, Elsevier, vol. 9(2), pages 187-206, April.
    6. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    7. Gueorgui Kambourov & Iourii Manovskii & Miana Plesca, 2020. "Occupational mobility and the returns to training," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(1), pages 174-211, February.
    8. Fredrik Andersson & Harry J. Holzer & Julia I. Lane & David Rosenblum & Jeffrey Smith, 2024. "Does Federally Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms," Journal of Human Resources, University of Wisconsin Press, vol. 59(4), pages 1244-1283.
    9. Chen, Yiu Por (Vincent), 2009. "Cream-Skimmer or Underdog? Labor Type Selectivity, Pre-Program Wage, and Rural Labor Training Program Outcome," IZA Discussion Papers 3979, Institute of Labor Economics (IZA).
    10. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    11. Kluve, Jochen & Lehmann, Hartmut & Schmidt, Christoph M., 2008. "Disentangling Treatment Effects of Active Labor Market Policies: The Role of Labor Force Status Sequences," Labour Economics, Elsevier, vol. 15(6), pages 1270-1295, December.
    12. Jochen Kluve & Hartmut Lehmann & Christoph M. Schmidt, 2000. "Disentangling Treatment Effects of Polish Active Labour Market Policies: Evidence from Matched Samples," CERT Discussion Papers 0007, Centre for Economic Reform and Transformation, Heriot Watt University.
    13. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    14. Robert J. LaLonde, 2003. "Employment and Training Programs," NBER Chapters, in: Means-Tested Transfer Programs in the United States, pages 517-586, National Bureau of Economic Research, Inc.
    15. Michael Lechner & Ruth Miquel & Conny Wunsch, 2011. "Long‐Run Effects Of Public Sector Sponsored Training In West Germany," Journal of the European Economic Association, European Economic Association, vol. 9(4), pages 742-784, August.
    16. Stenberg, Anders & Westerlund, Olle, 2016. "Flexibility at a cost – Should governments stimulate tertiary education for adults?," The Journal of the Economics of Ageing, Elsevier, vol. 7(C), pages 69-86.
    17. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    18. Heinrich, Carolyn J. & Mueser, Peter R. & Troske, Kenneth & Jeon, Kyung-Seong & Kahvecioglu, Daver C., 2009. "New Estimates of Public Employment and Training Program Net Impacts: A Nonexperimental Evaluation of the Workforce Investment Act Program," IZA Discussion Papers 4569, Institute of Labor Economics (IZA).
    19. Raphael W. Bostic & Allen C. Prohofsky, 2006. "Enterprise Zones and Individual Welfare: A Case Study of California," Journal of Regional Science, Wiley Blackwell, vol. 46(2), pages 175-203, May.
    20. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:30:y:2006:i:4:p:379-429. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.