How the General Benefit Reform Emerged in Finland: A Critical Analysis Using Large Language Models in Policy Research

Utilizing Large Language Models (LLM), this study investigates the evolution of an innovative social security policy idea, the General Benefit concept into a policy reform proposal in Fin-land from 2007 to 2023. Drawing from the ideational analysis we hypothesize that political parties struggled over social security conditionality during the 2010s and that social security simplification was manipulated differently in relation to conditionality. Our primary data is elec-tion manifestos and governmental programs from 2007-2023. We employed LLMs, mainly a customized ChatGPT, for the text analysis of policy documents. Additionally, we conduct a critical human evaluation of the LLMs analysis and publish our model in the GPT store for the open replication of analyses. Findings indicate that the weakening of the tripartite industrial relations system and the break-ing of “status quo of three big parties” allowed new parties to influence social policy in 2010s. The General Benefit emerged as a response to calls for social security simplification and for countering (unconditional) basic income proposals. Adopted in 2023, the General Benefit concept aims to merge Finnish universal / residence-based social insurance benefits for the working-aged while preserving core principles like social risk categories and conditionality. Despite increased nativism from the rising True Finns party, and the adoption of universal / unconditional basic income by several parties, Finnish social policy trends from 2007 to 2023 continued to emphasize employment and public finance sustainability. Our study also contributes to methodological discussions on using LLMs in policy analysis. The “human evaluation”, performed by the authors, confirms that the LLM analysis accurately summarises the main features of the policy evolution. However, we also found that the LLM lacks ability to recognise the nuances of “multidimensional” political language and is not very helpful in cross-sectional evaluation, which leaves the analysis partly shallow. Thus, we con-clude that in qualitative policy analysis, LLMs in their current form are suitable for comple-menting rather than substituting human evaluation.