Creating evaluative homogeneity: Experience of constructing a national journal ranking

The study investigates the process of creating evaluative homogeneity within a performance-based research funding system. It does so in reference to the experience of creating a national journal ranking in Poland in 2019. It refers to the cases of two disciplines: biology and history. The use of this case is justified by its unique character, that is creating the ranking through means of both bibliometric indicators and expert panels. We therefore asked: What guided the actors who participated in the process of creating the ranking through the use of bibliometric indicators when introducing changes to its initial form? To answer this question, a mixed-methods approach was used. First, we conducted a quantitative analysis of changes made to the ranking during the consecutive steps of its creation. Second, we conducted a qualitative analysis of semi-structured interviews regarding actors’ motivations. By reflecting on the qualitative analysis along with the results of the quantitative part, this study reveals the extent to which actors impacted the form evaluative homogeneity took within the system. It further argues that this is dictated by how actors position themselves in relation to two opposing forces: those favouring the homogenization of research and those maintaining its heterogeneity. As these forces remain imbalanced, the study concludes with a call for further exploration of the interplay between the forces of homogenization and heterogenization and how the tensions between them are mediated within a performance-based research funding system.