The Effect of Direct Job Creation and Training Programs on Low-Skilled Workers

This paper summarizes the literature on the impact of employment and training programs and concludes that they have been neither an overwhelming success nor a complete failure in terms of their ability to increase the long-term employment and earnings of disadvantaged workers. Employment and earnings programs capacity to improve the lot of any given participant, and the collective economic well-being of the disad- vantaged, has been modest -- as has been the level of resources devoted to these programs. Two decades of non-experimental program evaluation in the employment and training field have finally taught a less about which there can be little disagreement: Convincing program evaluation is going to require continued use of randomized trials. We wish to emphasize that this is not simply a statement that randomization is a preferable methodological approach regardless of the field of study. Instead, we believe the evidence in the study of employment and training programs overwhelmingly indicates that randomization is essential in program evaluation in the employment and training field. The difficulty seems to be that the earnings determination processes of today's workers and the program selection methods of today's programs interact to make it nearly impossible to produce reliable estimates of how workers ear- nings would have behaved in the absence of a program. The evidence to support this finding comes from both experimental and non-experimental studies. The non-experimental studies indicate that minor changes in methods, for which there is no empirical justification, produce large swings in estimated program effects. The study of experimental findings indicates that perfectly plausible non-experimental methods may lead to dramatic errors in inferences about program effects. This basic finding raises a fundamental question: What is the proper reaction of policy makers? In our view the appropriate reaction is for policymakers to begin the development of a credible research and development effort using randomized clinical trials in a wide variety of study areas.