Fine-Tuning Large Language Models to Simulate German Voting Behaviour (Working Paper)

Surveys are a cornerstone of empirical social science research, providing invaluable insights into the opinions, beliefs, behaviours, and characteristics of people. However, issues such as refusal to participate, skipping questions, sampling bias, and attrition significantly impact the quality and reliability of survey data. Recently, researchers have started investigating the potential of Large Language Models (LLMs) to role-play a pre-defined set of "characters" and simulate their survey responses with little or no additional training data and costs. While previous research on forecasting, imputing, and simulating survey answers with LLMs has focused on zero-shot and few-shot approaches, this study investigates the viability of fine-tuning LLMs to simulate responses of survey participants. We fine-tune Large Language Models (LLMs) on subsets of the data from the German Longitudinal Election Study (GLES) and evaluate their predictive performance on the "vote choice" for a random set of held-out participants. We compare the LLMs' performance against various baseline methods. Our findings show that small, fine-tuned open-source LLMs can outperform zero-shot predictions of larger LLMs. They are able to match the performance of established tabular data classifiers, are more sample efficient, and outperform them in cases with systematic non-responses. This study contributes to the growing body of research on LLMs for simulating survey data by demonstrating the effectiveness of fine-tuning approaches.