Towards a large scale analysis of claims: developing a machine learning method for detecting and classifying politicians’ claims of representation

In recent decades, many theoreticians have argued that we should pay attention to the claims of representation politicians make about groups in society. Nevertheless, despite recent advances on this topic, empirical research on politicians’ claims of representation remains relatively scant and mostly limited to case studies and manual annotation. Therefore, we develop a reusable Natural Language Processing (NLP) system to automatically classify claims by Dutch-speaking Belgian politicians. Following our new operationalization of claims of representation, which includes six constitutive elements, we use a limited amount of manually annotated data to train NLP models to automatically extract and classify these six elements. Our results show that using a combination of transformer learning (such as BERT), classic machine learning algorithms (such as SVMs), and rule-based methods, we can successfully classify each element of claims of representation, with macro F1-scores between 0.61 and 0.91. Taking all elements into account, we are able to correctly classify 74% of all detected claims in Belgian politicians’ Facebook posts between 2010 and 2022. Being the first to automate this process, this study contributes to the literature by offering a tested and validated method for classifying claims in politicians’ communication, thereby allowing large scale, and longitudinal analysis of claims. In the last section of this article, we further demonstrate some of the possibilities of our models by analyzing which groups politicians claimed to represent in the years before and after the start of the COVID-19 pandemic.