A large-scale semi-automated approach for assessing document-type classification errors in bibliometric databases

The accuracy of bibliometric databases in classifying document types (DTs)—such as research articles, conference proceedings, reviews, short notes, letters, book chapters, etc.—is crucial for the academic community, as bibliometric indicators may significantly influence research funding, decision-making, and academic reputation. This study presents a semi-automated methodology to assess the accuracy of DT classification in bibliometric databases, such as Scopus and Web of Science (WoS). The methodology can handle large document volumes and adapt to different DT categories without predefined correspondences. The first phase of the methodology automatically identifies discrepancies in DT classifications between Scopus and WoS, in order to find potentially misclassified documents; the second phase involves manually analyzing these documents to confirm and attribute classification errors. The methodology is applied to a sample of several tens of thousands of papers from the teaching staff of two major universities in Turin (Italy). The results show overall error rates of approximately 2.7% for Scopus and 2.3% for WoS. The paper also analyzes the most common types of errors found in both databases, providing an interpretation of these inaccuracies and some insights for possible improvements in the quality of these databases.