Using machine learning to study effect heterogeneity in large-scale policy interventions: The Dutch decentralisation of the Social Domain

There is lively discussion regarding the potential and pitfalls of artificial intelligence (AI) and machine learning (ML) for public policy. This debate tends to focus on replacing human decision-making with (semi-)automated processes and the unique challenges such applications pose for policymakers and society more generally. As this paper argues, particularly ML could be used in a more direct and less controversial way: to improve policy analysis and inform evidence-based policymaking. ML methods can be used to identify sub-groups in a population that differ in their policy effect in a data-driven way, which might otherwise be missed in standard policy analysis. In doing so, a more complete picture of a policy’s impact on a population can be obtained. I illustrate how ML can complement our understanding of policy interventions by studying the nationwide 2015 decentralisation of the social domain in The Netherlands. This policy intervention delegated responsibilities to administer social care from the national to the municipal level. Using ML methods on entire population data in The Netherlands, I find the policy induced strongly heterogeneous effects that include evidence of local capture and strong urban/rural divides. Findings that are crucial for policymakers to assess whether the policy had the desired outcome.