Active Labour Market Programme Participation for Unemployment Insurance Recipients: A Systematic Review

The objective of this systematic review was to study the effectiveness of Active Labour Market Programmes (ALMP) participation on employment status for unemployment insurance recipients. The primary outcome was measured as exit rate to work in a small time period and as the probability of employment at a given time. The two measures were analysed separately. We also investigated if participation effects differ with the type of ALMP programme and if participation in ALMP was associated with the quality of the job obtained as measured by employment duration and income. A total of 73 studies, consisting of 143 papers, met the inclusion criteria and were critically appraised by the review authors. The final selection comprised 73 studies from 15 different countries. Only 47 studies provided data that permitted the calculation of an effect size for the primary outcome. Of these, six studies could not be used in the data synthesis due to their high risk of bias. An additional two studies could not be used due to overlap of data samples. A total of 39 studies were therefore included in the data synthesis. Overall, ALMP programmes display a limited potential to alter the employment prospects of the individuals they intend to help. The available evidence does suggest that there is an effect of participating in ALMP, but the effect is small and we found no effect of being assigned to ALMP participation at a particular moment. It was not possible to examine a number of other factors which we had reason to expect as impacting on the magnitude of the effect and which may be crucial to policy makers. The results of this review, however, merely suggest that across a number of different programmes there is an overall small effect of ALMP participation on job finding rates, and no evidence of differential effects for different programmes. While additional research is needed, the review does however suggest that there is a small increase in the probability of finding a job after participation in ALMP Executive summary/Abstract BACKGROUND During the 1990s, many countries introduced Active Labour Market Programmes (ALMPs) in an effort to reduce unemployment. The introduction of ALMPs is often motivated by the need to upgrade the skills of especially those suffering long‐term unemployment to improve their productivity and, subsequently, their employability. Other ALMPs are designed to encourage the unemployed to return to work. Typically, compulsory programme participation is required after the individual has received unemployment benefits for a certain period of time. A large variety of different ALMPs exist among countries. They can consist of job search assistance, training, education, subsidized work and similar programmes. Some of the programmes (such as subsidized work, training and education) demand full‐time participation over a long time period (e.g. several months), while other programmes (such as job search assistance and education) are part‐time and have a short duration (e.g. few days/weeks). It is possible to classify these programmes into a set of four core categories: A: (labour market) training, B: Private sector programmes, C: direct employment programmes in the public sector and D: Job search assistance. The categories we use broadly correspond to classifications that have been suggested and used by the OECD and Eurostat (OECD, 2004 and Eurostat, 2005), even though there are differences between OECD and Eurostat in how they define and categorise these programmes. OBJECTIVES The objective of this systematic review was to study the effectiveness of ALMP participation on employment status for unemployment insurance recipients. The primary outcome was measured as exit rate to work in a small time period and as the probability of employment at a given time. The two measures were analysed separately. We also investigated if participation effects differ with the type of ALMP programme and if participation in ALMP was associated with the quality of the job obtained as measured by employment duration and income. SEARCH STRATEGY Relevant studies were identified through electronic searches of bibliographic databases, government policy databanks, internet search engines and hand searching of core journals. We searched to identify both published and unpublished literature. The searches were international in scope. Reference lists of included studies and relevant reviews were also searched. SELECTION CRITERIA All study designs that used a well‐defined control group were eligible for inclusion in this review. Studies that utilized qualitative approaches were not included due to the absence of adequate control group conditions. DATA COLLECTION AND ANALYSIS The total number of potential relevant studies constituted 16,422 hits. A total of 73 studies, consisting of 143 papers, met the inclusion criteria and were critically appraised by the review authors. The final selection comprised 73 studies from 15 different countries. Only 47 studies provided data that permitted the calculation of an effect size for the primary outcome. Of these, six studies could not be used in the data synthesis due to their high risk of bias. An additional two studies could not be used due to overlap of data samples. A total of 39 studies were therefore included in the data synthesis. Only five studies provided data that permitted the calculation of an effect size for secondary outcomes. Random effects models were used to pool data across the studies. We used the point estimate of the hazard ratio (the relative exit rate from unemployment to employment) and the risk difference (the difference in the probability of employment). Pooled estimates were weighted using inverse variance methods, and 95% confidence intervals were estimated. The impact of programme type was examined using meta regression and subgroup analysis. Sensitivity analysis was used to evaluate whether the pooled effect sizes were robust across study design, and to assess the impact of methodological quality and of the quality of data. Funnel plots were used to indicate the probability of publication bias. RESULTS The available evidence suggests that there is a general effect of participating in ALMP. The findings are mixed, however, depending on the approach used to investigate the effect, with no effect found of being assigned to ALMP participation at a particular moment. We found a statistically significant effect of ALMP post participation as measured by hazard ratios and risk difference in separate analyses. The overall impact of ALMP participation obtained using hazard ratios was 1.09, which corresponds to a 52 per cent chance that a treated unemployed person will find a job before a non‐treated unemployed person. The overall impact of ALMP participation was associated with a risk difference of 0.07, which corresponds to a number needed to treat of 15; i.e. for every 15 unemployed people who participate in ALMP, an additional unemployed person will be holding a job approximately one year after participation. The available evidence does not, however, suggest an effect of being assigned to ALMP participation at a particular moment. There was inconclusive evidence that participation in ALMP has an impact on the quality of the job obtained. Sensitivity analyses resulted in no appreciable change in effect size, suggesting that the results are robust. We found no strong indication of the presence of publication bias. The available evidence does not suggest that the effect of ALMP participation differs by type of programme. Other reviews by for example Kluve, 2010 and Card et al., 2010 conclude job search assistance programmes are relatively better, and direct employment programmes in the public sector relatively worse, than other programmes in terms of the likelihood of these different programmes to estimate a significant positive and a significant negative employment outcome. However, it should be kept in mind that the apparently different conclusions concerning relative effectiveness of type of ALMP are obtained based on very different inclusion criteria concerning participants and substantially different approaches and statistical methods. It was not possible to examine whether the participation effect varies with gender, age or educational group, or with labour market condition. AUTHORS' CONCLUSIONS To the best of our knowledge, this is the first systematic review analysing the magnitude (and not merely the statistical significance) of the effect of ALMP participation in unemployed individuals receiving unemployment insurance benefits. Overall, ALMP programmes display a limited potential to alter the employment prospects of the individuals they intend to help. The available evidence does suggest that there is an effect of participating in ALMP, but the effect is small and we found no effect of being assigned to ALMP participation at a particular moment. The four different types of ALMP (labour market training, private sector programmes, direct employment programmes in the public sector and job search assistance) were investigated. The available evidence does not suggest that the ALMP participation effect differs by type of ALMP. It was not possible to examine a number of other factors which we had reason to expect as impacting on the magnitude of the effect and which may be crucial to policy makers. The results of this review, however, merely suggest that across a number of different programmes there is an overall small effect of ALMP participation on job finding rates, and no evidence of differential effects for different programmes. While additional research is needed, the review does however suggest that there is a small increase in the probability of finding a job after participation in ALMP.