IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/19446.html
   My bibliography  Save this paper

Does Federally-Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms

Author

Listed:
  • Fredrik Andersson
  • Harry J. Holzer
  • Julia I. Lane
  • David Rosenblum
  • Jeffrey Smith

Abstract

We study the job training provided under the US Workforce Investment Act (WIA) to adults and dislocated workers in two states. Our substantive contributions center on impacts estimated non-experimentally using administrative data. These impacts compare WIA participants who do and do not receive training. In addition to the usual impacts on earnings and employment, we link our state data to the Longitudinal Employer Household Dynamics (LEHD) data at the U.S. Census Bureau, which allows us to estimate impacts on the characteristics of the firms at which participants find employment. We find moderate positive impacts on employment, earnings and desirable firm characteristics for adults, but not for dislocated workers. Our primary methodological contribution consists of assessing the value of the additional conditioning information provided by the LEHD relative to the data available in state Unemployment Insurance (UI) earnings records. We find that value to be zero.

Suggested Citation

  • Fredrik Andersson & Harry J. Holzer & Julia I. Lane & David Rosenblum & Jeffrey Smith, 2013. "Does Federally-Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms," NBER Working Papers 19446, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:19446
    Note: ED LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w19446.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Markus Frlich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, February.
    2. Matias Busso & John DiNardo & Justin McCrary, 2014. "New Evidence on the Finite Sample Properties of Propensity Score Reweighting and Matching Estimators," The Review of Economics and Statistics, MIT Press, vol. 96(5), pages 885-897, December.
    3. Michael Lechner & Conny Wunsch, 2009. "Are Training Programs More Effective When Unemployment Is High?," Journal of Labor Economics, University of Chicago Press, vol. 27(4), pages 653-692, October.
    4. Kornfeld, Robert & Bloom, Howard S, 1999. "Measuring Program Impacts on Earnings and Employment: Do Unemployment Insurance Wage Reports from Employers Agree with Surveys of Individuals?," Journal of Labor Economics, University of Chicago Press, vol. 17(1), pages 168-197, January.
    5. Pascal Courty & Gerald Marschke, 2004. "An Empirical Investigation of Gaming Responses to Explicit Performance Incentives," Journal of Labor Economics, University of Chicago Press, vol. 22(1), pages 23-56, January.
    6. Chabé-Ferret, Sylvain, 2012. "Matching vs Differencing when Estimating Treatment Effects with Panel Data: the Example of the Effect of Job Training Programs on Earnings," TSE Working Papers 12-356, Toulouse School of Economics (TSE).
    7. Annette Bergemann & Gerard J. Van Den Berg, 2008. "Active Labor Market Policy Effects for Women in Europe - A Survey," Annals of Economics and Statistics, GENES, issue 91-92, pages 385-408.
    8. Chabé-Ferret, Sylvain, 2015. "Analysis of the bias of Matching and Difference-in-Difference under alternative earnings and selection processes," Journal of Econometrics, Elsevier, vol. 185(1), pages 110-123.
    9. Lise, Jeremy & Seitz, Shannon & Smith, Jeffrey A., 2003. "Equilibrium Policy Experiments and the Evaluation of Social Programs," IZA Discussion Papers 758, Institute of Labor Economics (IZA).
    10. Fredriksson, Peter & Johansson, Per, 2002. "Program evaluation and random program starts," Working Paper Series 2003:1, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    11. Dolton, Peter & Smith, Jeffrey A., 2011. "The Impact of the UK New Deal for Lone Parents on Benefit Receipt," IZA Discussion Papers 5491, Institute of Labor Economics (IZA).
    12. Lawrence F. Katz & Jonathan Roth & Richard Hendra & Kelsey Schaberg, 2022. "Why Do Sectoral Employment Programs Work? Lessons from WorkAdvance," Journal of Labor Economics, University of Chicago Press, vol. 40(S1), pages 249-291.
    13. Irma Perez-Johnson & Quinn Moore & Robert Santillano, "undated". "Improving the Effectiveness of Individual Training Accounts: Long-Term Findings from an Experimental Evaluation of Three Service Delivery Models (Appendices)," Mathematica Policy Research Reports 8184db8183ac4abca4b55c4f0, Mathematica Policy Research.
    14. repec:mpr:mprres:5840 is not listed on IDEAS
    15. David H. Greenberg & Charles Michalopoulos & Philip K. Robins, 2004. "What Happens To The Effects Of Government-Funded Training Programs Over Time?," Journal of Human Resources, University of Wisconsin Press, vol. 39(1).
    16. Emily Oster, 2013. "Unobservable Selection and Coefficient Stability: Theory and Validation," NBER Working Papers 19054, National Bureau of Economic Research, Inc.
    17. Daniel Friedlander & David H. Greenberg & Philip K. Robins, 1997. "Evaluating Government Training Programs for the Economically Disadvantaged," Journal of Economic Literature, American Economic Association, vol. 35(4), pages 1809-1855, December.
    18. James Heckman & Salvador Navarro-Lozano, 2004. "Using Matching, Instrumental Variables, and Control Functions to Estimate Economic Choice Models," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 30-57, February.
    19. Robert J. LaLonde, 1995. "The Promise of Public Sector-Sponsored Training Programs," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 149-168, Spring.
    20. Irma Perez-Johnson & Quinn Moore & Robert Santillano, "undated". "Improving the Effectiveness of Individual Training Accounts: Long-Term Findings from an Experimental Evaluation of Three Service Delivery Models," Mathematica Policy Research Reports ddb772ea2aa242a1a74579eb0, Mathematica Policy Research.
    21. Cahuc, Pierre & Le Barbanchon, Thomas, 2010. "Labor market policy evaluation in equilibrium: Some lessons of the job search and matching model," Labour Economics, Elsevier, vol. 17(1), pages 196-205, January.
    22. David H. Greenberg & Charles Michalopoulos & Philip K. Robin, 2006. "Do experimental and nonexperimental evaluations give different answers about the effectiveness of government-funded training programs?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 25(3), pages 523-552.
    23. Martin Biewen & Bernd Fitzenberger & Aderonke Osikominu & Marie Paul, 2014. "The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 837-897.
    24. Alberto Abadie & Guido W. Imbens, 2008. "On the Failure of the Bootstrap for Matching Estimators," Econometrica, Econometric Society, vol. 76(6), pages 1537-1557, November.
    25. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    26. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    27. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    28. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    29. repec:mpr:mprres:6097 is not listed on IDEAS
    30. Black, Dan A. & Smith, J.A.Jeffrey A., 2004. "How robust is the evidence on the effects of college quality? Evidence from matching," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 99-124.
    31. John M. Abowd & Bryce E. Stephens & Lars Vilhuber & Fredrik Andersson & Kevin L. McKinney & Marc Roemer & Simon Woodcock, 2009. "The LEHD Infrastructure Files and the Creation of the Quarterly Workforce Indicators," NBER Chapters, in: Producer Dynamics: New Evidence from Micro Data, pages 149-230, National Bureau of Economic Research, Inc.
    32. repec:mpr:mprres:7217 is not listed on IDEAS
    33. repec:eee:labchp:v:2:y:1986:i:c:p:789-848 is not listed on IDEAS
    34. Heinrich, Carolyn J. & Mueser, Peter R. & Troske, Kenneth & Jeon, Kyung-Seong & Kahvecioglu, Daver C., 2009. "New Estimates of Public Employment and Training Program Net Impacts: A Nonexperimental Evaluation of the Workforce Investment Act Program," IZA Discussion Papers 4569, Institute of Labor Economics (IZA).
    35. John M. Abowd & Paul A. Lengermann & Kevin L. McKinney, 2002. "The Measurement of Human Capital in the U.S. Economy," Longitudinal Employer-Household Dynamics Technical Papers 2002-09, Center for Economic Studies, U.S. Census Bureau, revised Mar 2003.
    36. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    37. James J. Heckman & Carolyn J. Heinrich & Pascal Courty & Gerald Marschke & Jeffrey Smith (ed.), 2011. "The Performance of Performance Standards," Books from Upjohn Press, W.E. Upjohn Institute for Employment Research, number tpps.
    38. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    39. repec:adr:anecst:y:2008:i:91-92:p:17 is not listed on IDEAS
    40. Barbara Sianesi, 2004. "An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 133-155, February.
    41. Peter Z. Schochet & John Burghardt & Sheena McConnell, 2006. "National Job Corps Study and Longer-Term Follow-Up Study: Impact and Benefit-Cost Findings Using Survey and Summary Earnings Records Data," Mathematica Policy Research Reports 8074f4e4499d4e2ab1de13747, Mathematica Policy Research.
    42. Bev Dahlby, 2008. "The Marginal Cost of Public Funds: Theory and Applications," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262042509, April.
    43. Devin G. Pope & Justin R. Sydnor, 2011. "Implementing Anti-discrimination Policies in Statistical Profiling Models," American Economic Journal: Economic Policy, American Economic Association, vol. 3(3), pages 206-231, August.
    44. James J. Heckman & Carolyn Heinrich & Jeffrey Smith, 2002. "The Performance of Performance Standards," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 778-811.
    45. Howard S. Bloom & Larry L. Orr & Stephen H. Bell & George Cave & Fred Doolittle & Winston Lin & Johannes M. Bos, 1997. "The Benefits and Costs of JTPA Title II-A Programs: Key Findings from the National Job Training Partnership Act Study," Journal of Human Resources, University of Wisconsin Press, vol. 32(3), pages 549-576.
    46. Peter Z. Schochet & John Burghardt & Sheena McConnell, 2008. "Does Job Corps Work? Impact Findings from the National Job Corps Study," American Economic Review, American Economic Association, vol. 98(5), pages 1864-1886, December.
    47. James J. Heckman & Jeffrey A. Smith, 1999. "The Pre-Program Earnings Dip and the Determinants of Participation in a Social Program: Implications for Simple Program Evaluation Strategies," NBER Working Papers 6983, National Bureau of Economic Research, Inc.
    48. Heckman, James J & Smith, Jeffrey A, 1999. "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies," Economic Journal, Royal Economic Society, vol. 109(457), pages 313-348, July.
    49. Carolyn Heinrich & Peter Mueser & Kenneth Troske & Kyung-Seong Jeon & Daver Kahvecioglu, 2013. "Do Public Employment and Training Programs Work?," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 2(1), pages 1-23, December.
    50. Busso, Matias & DiNardo, John & McCrary, Justin, 2009. "New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators," IZA Discussion Papers 3998, Institute of Labor Economics (IZA).
    51. Burt S. Barnow & David Greenberg, 2015. "Do Estimated Impacts on Earnings Depend on the Source of the Data Used to Measure Them? Evidence From Previous Social Experiments," Evaluation Review, , vol. 39(2), pages 179-228, April.
    52. David W. Stevens, 2007. "Employment that is not covered by state unemployment insurance Laws," Longitudinal Employer-Household Dynamics Technical Papers 2007-04, Center for Economic Studies, U.S. Census Bureau.
    53. Harry J. Holzer & Julia I. Lane & Lars Vilhuber & Henry Jackson & George Putnam, 2001. "Escaping poverty for low-wage workers The role of employer characteristics and changes," Longitudinal Employer-Household Dynamics Technical Papers 2001-02, Center for Economic Studies, U.S. Census Bureau.
    54. Geoffrey L. Wallace & Robert Haveman, 2007. "The implications of differences between employer and worker employment|earnings reports for policy evaluation," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 26(4), pages 737-754.
    55. Timothy Dunne & J. Bradford Jensen & Mark J. Roberts, 2009. "Producer Dynamics: New Evidence from Micro Data," NBER Books, National Bureau of Economic Research, Inc, number dunn05-1.
    56. Louis Jacobson & Robert J. Lalonde & Daniel Sullivan, 2005. "The Impact of Community College Retraining on Older Displaced Workers: Should We Teach Old Dogs New Tricks?," ILR Review, Cornell University, ILR School, vol. 58(3), pages 398-415, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    2. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    3. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    4. Caliendo, Marco & Mahlstedt, Robert & Mitnik, Oscar A., 2017. "Unobservable, but unimportant? The relevance of usually unobserved variables for the evaluation of labor market policies," Labour Economics, Elsevier, vol. 46(C), pages 14-25.
    5. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    6. David Card & Jochen Kluve & Andrea Weber, 2018. "What Works? A Meta Analysis of Recent Active Labor Market Program Evaluations," Journal of the European Economic Association, European Economic Association, vol. 16(3), pages 894-931.
    7. Martin Biewen & Bernd Fitzenberger & Aderonke Osikominu & Marie Paul, 2014. "The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 837-897.
    8. Raaum, Oddbjorn & Torp, Hege, 2002. "Labour market training in Norway--effect on earnings," Labour Economics, Elsevier, vol. 9(2), pages 207-247, April.
    9. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    10. Annabelle Doerr & Bernd Fitzenberger & Thomas Kruppe & Marie Paul & Anthony Strittmatter, 2017. "Employment and Earnings Effects of Awarding Training Vouchers in Germany," ILR Review, Cornell University, ILR School, vol. 70(3), pages 767-812, May.
    11. Muller, Paul & van der Klaauw, Bas & Heyma, Arjan, 2017. "Comparing Econometric Methods to Empirically Evaluate Job-Search Assistance," IZA Discussion Papers 10531, Institute of Labor Economics (IZA).
    12. Gustavo Canavire-Bacarreza & Luis Castro Peñarrieta & Darwin Ugarte Ontiveros, 2021. "Outliers in Semi-Parametric Estimation of Treatment Effects," Econometrics, MDPI, vol. 9(2), pages 1-32, April.
    13. Hagen, Tobias, 2016. "Econometric Evaluation of a Placement Coaching Program for Recipients of Disability Insurance Benefits in Switzerland," VfS Annual Conference 2016 (Augsburg): Demographic Change 145736, Verein für Socialpolitik / German Economic Association.
    14. Anders Stenberg & Olle Westerlund, 2015. "The long-term earnings consequences of general vs. specific training of the unemployed," IZA Journal of European Labor Studies, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-26, December.
    15. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    16. Hagen, Tobias, 2016. "Econometric evaluation of a placement coaching program for recipients of disability insurance benefits in Switzerland," Working Paper Series 10, Frankfurt University of Applied Sciences, Faculty of Business and Law.
    17. Dolton, Peter & Smith, Jeffrey A., 2011. "The Impact of the UK New Deal for Lone Parents on Benefit Receipt," IZA Discussion Papers 5491, Institute of Labor Economics (IZA).
    18. Regner, Hakan, 2002. "A nonexperimental evaluation of training programs for the unemployed in Sweden," Labour Economics, Elsevier, vol. 9(2), pages 187-206, April.
    19. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    20. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.

    More about this item

    JEL classification:

    • I38 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Government Programs; Provision and Effects of Welfare Programs
    • J08 - Labor and Demographic Economics - - General - - - Labor Economics Policies
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:19446. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.