Comparison of citation impact between pre-and post-publication peer-selected best papers: the case of Best Paper Awards recipients at computer science conferences

This paper presents a preliminary comparison of citation impact between two types of Best Paper Awards recipients: pre-and post-publication peer-selected best papers. Two hundred and ninety-nine pairs of pre-and post-publication peer-selected best papers from 15 highly prestigious computer science conferences spanning more than two decades were gathered for empirical analysis. Each pair was published in the same proceedings, and their citation information was collected within a 10-year period after their formal publication. A series of pairwise comparisons were performed in order to unbiasedly uncover their differences in citation counts (i.e., a good proxy for citation impact), as well as in citation speed, citation diversity, and citation location distributions (i.e., three characteristic properties of citation impact). The empirical results demonstrate that in computer science, the post-publication peer-selected best papers receive significantly more citations than their pre-publication counterparts, with an estimated probability of up to 0.79 (95% CI: 0.74 ~ 0.83). Meanwhile, they tend to accumulate their citations more quickly in the later (versus earlier) stages—they can scarcely be fast-aging papers, and their accumulated citations tend to cover a broader scope of academic disciplines. Overall, however, the pre-and post-publication peer-selected best papers have very similar citation location distributions. Both of them are most frequently cited in the Background sections, followed by the Methods sections, and finally the Results sections. In addition, the citation impact superiority of the post-publication peer-selected best papers remains consistent across different fields of research and over a long period of time, from 1997 to 2022.