From rules to forests: rule-based versus statistical models for jobseeker profiling

Public employment services (PES) commonly apply profiling systems to target support programs to jobseekers at risk of becoming long-term unemployed. Such systems often codify institutional experiences in a set of decision rules, whose predictive ability, however, is seldomly tested. We systematically evaluate the predictive performance of a rule-based system currently implemented by the PES of Catalonia, Spain, in comparison to the performance of statistical models in predicting future long-term unemployment episodes. Using comprehensive administrative data, we develop linear and machine learning models and evaluate their performance with respect to both discrimination and calibration. Compared to the current rule-based system of Catalonia, our machine learning models achieve greater discrimination ability and remarkable improvements in calibration. Particularly, our random forest model is able to accurately forecast episodes and outperforms the rule-based model by offering robust quantitative predictions that perform well under stress tests. This paper presents the first performance comparison between a complex, currently implemented, rule-based approach and complex statistical profiling models. Our work illustrates the importance of assessing the calibration of profiling models and the potential of statistical tools to assist public employment offices in Spain.