Interventions to improve the labour market outcomes of youth: A systematic review of training, entrepreneurship promotion, employment services and subsidized employment interventions

This Campbell systematic review examines the impact of youth employment interventions on the labour market outcomes of young people and business performance. The review summarises findings from 113 reports of 107 interventions in 31 countries. Included studies had to: (1) evaluate an active labour market programme (ALMP) which was designed for ‐ or targeted primarily ‐ young women and men aged between 15 and 35; (2) have an experimental and quasi‐experimental design; and (3) report at least one eligible outcome variable measuring employment, earnings, or business performance. The evidence base covers 107 interventions in 31 countries, including 55 using skills training, 15 with entrepreneurship promotion, ten using employment services and 21 using subsidized employment. Overall, youth employment interventions increase the employment and earnings of those youth who participate in them. But the effect is small with a lot of variation between programmes. There are significant effects for entrepreneurship promotion and skills training, but not for employment services and subsidised employment. Impacts on earnings were also positive but small and highly variable across programmes. Entrepreneurship promotion and skills training were effective in increasing earnings, while effects of employment services and subsidised employment were negligible or statistically insignificant. There is limited evidence of the effects of youth employment programmes on business performance outcomes, and the effect size was not statistically significant. In addition to the variation in impact across different types of programmes, some variation can be explained by country context, intervention design, and profile and characteristics of programme beneficiaries. The impacts of ALMPs are greater in magnitude in low‐ or middle‐income countries than in high‐income countries. Programmes targeting the most disadvantaged youth were associated with bigger programme effects, particularly for earnings outcomes, and effects were slightly larger for women than for men. Plain language summary Active labour market programmes for youth increase employment and earnings. Effects vary between programmes and context The evidence suggests that investing in youth through active labour market measures, such as offering skills training and entrepreneurship promotion, may pay off with higher earnings. The review in brief Youth are disproportionately victims of unemployment and low‐quality jobs. Active labour market programmes increase earnings and employment. But the effects vary greatly between programmes’ type, design and context. What is this review about? Youth unemployment is much greater than the average unemployment rate for adults, in some cases over three times as high. Today, over 73 million young people are unemployed worldwide. Moreover, two out of five young people in the labour force are either working but poor or are unemployed. The youth employment challenge is not only about job creation, but especially about enhancing the quality of jobs for youth. This systematic review assesses the impact of youth employment interventions on the labour market outcomes of young people. The included interventions are training and skills development, entrepreneurship promotion, employment services and subsidized employment. Outcomes of interest include employment, earnings and business performance outcomes. What is the aim of this review? This Campbell systematic review examines the impact of youth employment interventions on the labour market outcomes of young people and business performance. The review summarises findings from 113 reports of 107 interventions in 31 countries. What are the main findings of this review? Included studies had to: (1) evaluate an active labour market programme (ALMP) which was designed for – or targeted primarily – young women and men aged between 15 and 35; (2) have an experimental and quasi‐experimental design; and (3) report at least one eligible outcome variable measuring employment, earnings, or business performance. The evidence base covers 107 interventions in 31 countries, including 55 using skills training, 15 with entrepreneurship promotion, ten using employment services and 21 using subsidized employment. Overall, youth employment interventions increase the employment and earnings of those youth who participate in them. But the effect is small with a lot of variation between programmes. There are significant effects for entrepreneurship promotion and skills training, but not for employment services and subsidised employment. Impacts on earnings were also positive but small and highly variable across programmes. Entrepreneurship promotion and skills training were effective in increasing earnings, while effects of employment services and subsidised employment were negligible or statistically insignificant. There is limited evidence of the effects of youth employment programmes on business performance outcomes, and the effect size was not statistically significant. In addition to the variation in impact across different types of programmes, some variation can be explained by country context, intervention design, and profile and characteristics of programme beneficiaries. The impacts of ALMPs are greater in magnitude in low‐ or middle‐income countries than in high‐income countries. Programmes targeting the most disadvantaged youth were associated with bigger programme effects, particularly for earnings outcomes, and effects were slightly larger for women than for men. What do the findings of this review mean? The evidence suggests that investing in youth through active labour market measures may pay off. Skills training and entrepreneurship promotion interventions appear to yield positive results on average. So, there are potential benefits from combining supply‐ and demand‐side interventions to support youth in the labour market. The evidence indicates the need for careful design of youth employment interventions. The “how” seems to be more important than the “what” and, in this regard, targeting disadvantaged youth may act as a key factor for success. There is a need to strengthen the evidence base with more studies of promising programmes, especially in sub‐Saharan Africa. Further research should investigate intermediate outcomes and soft skills, and should collect cost data. How up‐to‐date is this review? The review authors searched for studies published up to January 2015. This Campbell systematic review was published in November 2017. Abstract Background – Today's labour market is a challenging arena for young people. Over 73 million youth are currently unemployed and many more are affected by vulnerable employment and working poverty. Youth remain highly susceptible to changing patterns in the world of work and experience slow and difficult transitions to stable jobs. What works to support them in the labour market? This is one of the most common and pressing questions posed by policymakers and practitioners today. Methods – This systematic review addresses this question by synthesizing empirical evidence on the labour market outcomes of active labour market programmes (ALMPs) targeting youth worldwide. Eligible interventions comprised skills training such as technical and business skills, entrepreneurship promotion providing access to finance, employment services providing job‐placement and job‐search assistance, and subsized employment providing wage subsidies or public employment. Outcomes of interest included employment, earnings and business performance. Eligible studies included counterfactual‐based impact evaluations conducted in low‐, middle‐ or high‐income countries. A comprehensive systematic search for relevant evidence across more than 70 sources, using search terms in English, French, German, Portuguese and Spanish, identified over 30,000 records that were screened. The search process was completed in January 2015. For the selected studies that met the review's inclusion criteria, data were coded and effect sizes calculated. The analysis explores the interventions’ overall effectiveness and the roles that context, evaluation and programme design and implementation play in moderating impact. Results – A total of 113 eligible impact evaluations were identified, encompassing a unique set of evaluation methods, interventions and geographical coverage. Meta‐analysis methods were employed to synthesize the evidence, based on 2,259 imputed effect sizes. Overall, empirical results indicated positive effects of entrepreneurship promotion and skills training on employment and earnings. Effects of employment services and subsidised employment were generally small and non‐significant. We estimated bigger programme effects in low‐ and middle‐income countries than in high‐income countries, and in programmes targeting disadvantaged youth. Implications – Active measures to support the (re) integration of young women and men into the labour market may succeed in enhancing employment and earnings outcomes and have potential to increase human capital and employment prospects in the long‐term. The evidence suggested that programmes targeting disadvantaged youth are particularly effective. Entrepreneurship promotion and skills training programmes appear to be a particularly promising intervention for improving employment, earnings and business performance, but the evidence base is still relatively small. More rigorous impact evidence is needed for particular employment programmes more generally, including employment services, subsidised employment and entrepreneurship promotion. Executive summary Background The youth of today represent a vast potential for inclusive growth and development. If youth are given the opportunity to build appropriate skills and access decent employment, they can help to accelerate progress on the 2030 Agenda for Sustainable Development and engage in meaningful work that benefits them, their families and society as a whole. Unfortunately, decent jobs are not a feasible prospect for all young women and men. Today, over 73 million young people are unemployed worldwide. Youth unemployment stands at a much higher level than the average unemployment rate for adults, in some cases over three times as high. Moreover, two out of five young people in the labour force are either working but poor or unemployed. The youth employment challenge is therefore not only about job creation, but also – and especially – about enhancing the quality of jobs for youth. Youth's gloomy prospects in the labour market embody a massive waste of potential and a threat to social cohesion. Understanding what works to improve their labour market outcomes is therefore of paramount importance and a development priority for all countries and regions. Objectives The aim of this systematic review was to investigate the impact of youth employment interventions on the labour market outcomes of young people. The interventions under review comprised training and skills development, entrepreneurship promotion, employment services and subsidized employment. Outcomes of interest included employment, earnings and business performance outcomes. Search Methods The review relied on a comprehensive systematic search across more than 70 sources, including literature databases and a large number of websites, which allowed the identification of both published and unpublished studies . The search process included both a primary search (i.e., searching of a wide range of general and specialized databases) and a complementary search (i.e., hand‐searching of relevant websites, searching of dissertations, theses and grey literature databases, citation tracking, screening of reference lists and contacting authors and experts). The in‐depth complementary search allowed the identification of several unpublished studies. The process included search terms in English, French, German, Portuguese and Spanish. The search process was completed in January 2015. Selection Criteria Eligible studies are those that: 1 evaluated an active labour market programme (ALMP) that included at least one of the following categories of interventions: training and skills development (such as technical and non‐technical skills), entrepreneurship promotion (providing access to capital, from financing to entrepreneurial skills that would enhance human capital), employment services (providing job‐placement and job‐search assistance, among other services) and/or subsidized employment (providing wage subsidies or public employment programmes); 2 investigated programmes that were designed for – or targeted primarily – young women and men aged between 15 and 35; 3 reflected completed experimental and quasi‐experimental evaluations measuring impacts on eligible labour market outcomes; and 4 reported at least one eligible outcome variable measuring employment (e.g., probability of employment, hours worked, duration in unemployment), earnings (e.g., reported earnings, wages, consumption) or business performance (e.g., profits, sales). In addition to the above inclusion criteria, the review focused on studies with a publication date between 1990 and 2014. No language restrictions were applied. Data Collection and Analysis A coding tool and manual were developed in order to guide a harmonised data extraction process. Treatment effect estimates were coded across all studies that met the inclusion criteria, along with other parameters and intervention characteristics deemed relevant for the analysis. Additional, non‐reported information was retrieved from authors of the primary studies, supporting the computation of standardized mean differences (SMDs) effect sizes. The SMDs captured the relative magnitude of the treatment effect in a dimensionless way, which was therefore comparable across outcomes and studies. Effect sizes were summarized within and across reports to one effect size per outcome for each study. Random‐effects meta‐analysis methods were employed to synthesize and compare effect sizes reported in the primary studies. Subsequently, multivariate meta‐regression models were estimated and information about intervention‐level, study‐level and country‐level characteristics were included to assess factors associated with the magnitude of reported effect size estimates. Results The primary and complementary searches identified 32,117 records, of which a total of 1,141 records were selected for full text screening. The subsequent selection process led to a sample of 113 reports, which were considered to be of adequate content and methodological rigour for inclusion in the meta‐analysis. The 113 reports represented 107 interventions. The evidence base spanned 31 countries and covered 55 skills training interventions, 15 entrepreneurship promotion interventions, ten employment services interventions and 21 subsidized employment interventions. There were six interventions for which no clear main category of intervention could be established. A large share of the evidence derived from recent publications, with nearly half of the sample produced after 2010. Evaluation designs varied, with 47 per cent of reports relying on experimental designs, 10 per cent on natural experiments and 44 per cent on quasi‐experimental evaluations. Many of the most recent studies were experimental evaluations of interventions implemented in low‐ and middle‐income countries, Not ably from Africa and Latin America and the Caribbean. Intervention characteristics and research designs differed significantly between evaluations implemented in high‐income and low‐ or middle‐income countries. A large proportion of the evidence from high‐income countries derived from quasi‐experimental studies of national programmes, implemented in collaboration with government organizations. In contrast, the evidence from low‐ and middle‐income countries was predominantly based on experimental impact evaluations of rather small‐scale, targeted interventions, which were often implemented by Non‐governmental Organizations (NGOs) or international organizations. The comprehensive systematic search led to the identification and coding of a total of 3,629 treatment effect estimates. These estimates, along with further information reported and/or retrieved from authors of the primary studies and imputation of missing information, allowed the computation of 2,259 SMDs. The following are some of the key results from the meta‐analysis of SMD effect sizes. These findings appear robust to the different study designs employed, as similar results were found for a restricted sample of the most rigorous designs (experimental impact evaluations). However, there was also statistical evidence for small study effects for all outcomes, suggesting the presence of publication bias in the literature. 1 Youth employment interventions may lead to positive outcomes, increasing employment and earnings of participating youth. The positive effect on employment was captured by an overall SMD effect size of 0.04 (95% confidence interval (CI) = 0.03, 0.06) across 105 interventions, demonstrating that young people who were exposed to a youth employment intervention on average had better employment outcomes than those who were not. Effects across studies were, however, heterogeneous (I‐squared=64%). Sub‐group analysis by intervention category indicated that entrepreneurship promotion (SMD=0.16; 95% CI=0.06, 0.26; I‐sq=71%; evidence from seven interventions) and skills training (SMD=0.05, 95% CI=0.02, 0.07; I‐sq=65%; 67 interventions) on average improved employment outcomes. Effects were small and not statistically significant for employment services (SMD=0.01; 95% CI=‐0.02, 0.04; I‐sq=0%; 10 interventions) and subsidised employment (SMD=0.02; 95% CI=‐0.01, 0.06; I‐sq=50%; 16 interventions). The residual inconsistency in subsidized employment estimates suggested further moderator and sub‐group analyses were needed. 2 Impacts on earnings were positive and statistically significant on average, with an effect size of 0.05 SMD (95% CI = 0.03, 0.06) across 92 interventions. Findings exhibit again high heterogeneity, i.e. were highly inconsistent across programmes (I‐squared=82%). Further analysis by intervention categories again indicated that entrepreneurship promotion (SMD=0.09; 95% CI=0.01, 0.18; I‐sq=64%; 12 interventions) and skills training (SMD=0.07; 95% CI=0.05, 0.08; I‐sq=86%; 60 interventions) were effective in raising earnings, while effects of employment services (SMD=0.01; 95% CI=0.00, 0.02; I‐sq=0%; eight interventions) and subsidised employment (SMD=‐0.01; 95% CI=‐0.05, 0.03; I‐sq=61%; nine interventions) were negligible and/or statistically insignificant. The residual inconsistency in subsidized employment estimates again suggested more analysis by moderator variables and sub‐groups. 3 Evidence of youth employment programme effects on business performance outcomes was limited and the effect size was not statistically significant (SMD = 0.03; 95% CI = ‐0.05, 0.12; I‐sq=49%) across 14 interventions. When entrepreneurship promotion interventions were considered in isolation, the impact was larger and significant, at 0.10 SMD (95% CI = 0, 0.19; I‐sq=39%; 10 interventions). Evidence from the small number of evaluations of skills training did not suggest positive or significant effects on business performance outcomes (SMD=‐0.09; 95% CI=‐0.19, 0.01; I‐sq=0; 4 interventions). 4 The high degree of inconsistency across interventions suggested programme impacts concealed major contextual differences. The meta‐analysis showed important differences in the magnitudes of impact across outcomes and interventions. Despite the strong similarities across included studies, the differences in impact were not always driven by chance. Tests for heterogeneity demonstrated substantial variation in the effect size magnitude due to: country context, intervention design, and profile and characteristics of programme beneficiaries. 5 The underlying evidence base varies by country income level. Results suggest impacts of ALMPs are greater in magnitude in low‐ or middle‐income countries than in high‐income countries. In low‐ and middle‐income countries, skills training (SMD=0.06; 95% CI=0.02, 0.10; I‐sq=76%; 38 interventions), entrepreneurship promotion (SMD=0.18; 95% CI=0.06, 0.29; I‐sq=12%; 5 interventions) and subsidised employment (SMD=0.11; 95% CI=0.04, 0.18; I‐sq=11%; 5 interventions) were effective in increasing employment on average. Skills training (SMD=0.12; 95% CI=0.08, 0.16; I‐sq=77%; 39 interventions) and entrepreneurship promotion (SMD=0.14; 95% CI=0.06, 0.22; I‐sq=15%; 10 interventions) also yielded positive results, on average, in terms of income gains. 6 In high‐income countries, the overall effects of ALMPs on employment (SMD=0.04; 95% CI=0.01, 0.07; I‐sq=57%; 52 interventions) and earnings (SMD=0.01; 95% CI=‐0.01, 0.02; I‐sq=70%; 31 interventions) were small. In sub‐group analysis by intervention type, only skills training appeared to effectuate some (albeit small) impact average on employment (SMD=0.04; 95% CI=0.01, 0.07; I‐sq=58%). 7 Programmes targeting the most disadvantaged youth were associated with bigger programme effects, particularly for earnings outcomes. Across measures of targeting, a focus on low‐income youth, those with low levels of education or exhibiting strong disadvantages in the labour market was associated with marginally higher employment (SMD=0.06; 95% CI=0.02, 0.09; I‐sq=66%) and significantly higher earnings gains (SMD=0.13; 95% CI=0.09, 0.18; I‐sq=82%) for youth across all country income levels than employment (SMD=0.03; 95% CI=0.01, 0.06; I‐sq=56%) and earnings (SMD=0.02; 95% CI=0.00, 0.03; I‐sq=73%) for less disadvantaged youth. 8 Looking at differences in effects by gender, the findings suggested that employment and earnings outcomes for women were marginally larger than those for men. 9 The systematic review captured information about the type of skills delivered to young people and found no particular connection between soft skills and better labour market outcomes. Similarly, there was no systematic evidence about the role of public, private or civil entities in the implementation of a youth employment programme. Conclusions The extent and urgency of the youth employment challenge and the level of global attention currently being given to this topic calls for more and better evidence‐based action. Accordingly, this systematic review sought to examine the empirical evidence in order to understand what drives the success (or failure) of youth employment interventions. Investments in youth employment will continue, and even increase, as countries embark on the implementation of the 2030 Agenda for Sustainable Development; therefore, this review focused on identifying “what works” and, as far as possible, “how”. This systematic review builds on a growing base of studies measuring the impact of youth employment interventions and offers a rigorous synthesis and overall balance of empirical evidence taking into account the quality of the underlying research. The review is systematic through a clearly defined and transparent inclusion and exclusion criteria, an objective and extensive search, a punctual data extraction process, a standardized statistical testing and analysis, and a thorough reporting of findings. These elements and underlying methods and tools were laid out and reviewed in the protocol (Kluve et al., 2014). The evidence suggests that investing in youth through active labour market measures may pay off. The evidence also shows a significant impact gap across country income levels. Being unemployed or unskilled in a high‐income country – where labour demand is skill intensive – puts youth at a distinct disadvantage in comparison to a cohort that is, on average, well educated. While ALMPs in high‐income countries can integrate disadvantaged young people into the labour market, they are not able to fully compensate for a lack of skills or other areas where youth failed to gain sufficient benefit from the education system. On the other hand, in lower income countries, with large cohorts of disadvantaged youth, marginal investments in skills and employment opportunities are likely to lead to larger changes in outcomes. Youth‐targeted ALMPs in low‐ and middle‐income countries do lead to impacts on both employment and earnings outcomes. Specifically, skills training and entrepreneurship promotion interventions appear to yield positive results on average. This is an important finding, which points to the potential benefits of combining supply‐ and demand‐side interventions to support youth in the labour market. The evidence also calls for careful design of youth employment interventions. The “how” seems to be more important than the “what” and, in this regard, targeting disadvantaged youth may act as key factors of success. The findings from this review need to be discussed vis‐à‐vis the local and national context and should be complemented by a long‐term and holistic commitment towards youth development. Achieving an understanding of the “how” element is not an easy task. Although the systematic review excluded studies that only reported relative effects, it is also the case that, frequently, impact evaluations do not assess relative effectiveness. Even more often, reports and papers fail to describe the underlying theory of change and observed transmission mechanisms behind an intervention. In some other cases, there is limited information about the characteristics of programme participants in the evaluation sample and their comparison group. Much remains to be done to improve reporting standards and advocate for more and better evidence examining the impact of youth employment interventions. The quality of the primary studies determines the quality of the systematic review and any subsequent synthesis of the evidence. The review supported the identification of important evidence gaps: • It is important to note that despite the large and significant magnitude of effect of entrepreneurship promotion interventions in low‐ and middle‐income countries, the evidence base is still limited and exhibits high variance, calling for more primary studies on this promising intervention type. Similarly, more and better evidence is needed on employment services, wage subsidies and public employment programmes for youth, particularly in low‐ and middle‐income countries. • While the review highlighted a growing evaluation evidence from youth employment programmes implemented in Sub‐Saharan Africa, it also reported very limited information from the Middle East and North Africa, South Asia and East Asia and the Pacific. These are regions were more targeted action to expand the evidence base should be considered. • Similarly, more research is needed on intermediate outcomes in primary studies and evidence synthesis work. This is linked to the importance of improving research‐reporting standards and expanding the scope of outcomes of interest in order to better synthesize evidence about how interventions affect knowledge, skills, attitudes, and behaviours. More and better information on these intermediate outcomes will improve overall understanding about the causality and pathways of change between the intervention and the final outcomes. • Soft skills are highly demanded by employers today. Their role in generating better outcomes is yet to be corroborated and more inquiry is required to understand their role in the causal chain as well as their interaction with more technical skills sets. • Lastly, future primary studies and evidence syntheses should engage with cost information. The applicability of the evidence hinges not only on its internal and external validity but also on its feasibility. More information is needed on programme costs as well as systematic comparisons against programme effects. What may look highly effective may in fact be too expensive to replicate or scale up.