How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature

As the subject of research excellence has received increasing attention (in science policy) over the past few decades, increasing numbers of bibliometric studies have been published dealing with excellent papers. However, many different methods have been used in these studies to identify excellent papers. The present quantitative analysis of the literature has been carried out to acquire an overview of these methods and an indication of an ‘average’ or ‘most frequent’ bibliometric practice. The search in the Web of Science yielded 321 papers dealing with ‘highly cited’, ‘most cited’, ‘top cited’, and ‘most frequently cited’. Of the 321 papers, 16 could not be used in this study. In around 80% of the papers analyzed in this study, a quantitative definition has been provided to identify excellent papers. With definitions that relate to an absolute number, either a certain number of top cited papers (58%) or papers with a minimum number of citations are selected (17%). Approximately 23% worked with percentile rank classes. Over these papers, there is an arithmetic average of the top 7.6% (arithmetic average) or of the top 3% (median). The top 1% is used most frequently in the papers, followed by the top 10%. With the thresholds presented in this study, in future, it will be possible to identify excellent papers based on an ‘average’ or ‘most frequent’ practice among bibliometricians.