Reproducibility issues with correlating Beall-listed publications and research awards at a small Canadian business school

The issue of “predatory” publishing began to directly affect academics following the creation of two blacklists by a US librarian, Jeffrey Beall. This paper provides a post-publication replication examination of a study that became a high media-profile case. That study involves Derek Pyne, who incorrectly claimed in a 2017 publication in the Journal of Scholarly Publishing that numerous research faculty members at a small business school in Canada were financially remunerated for Beall-blacklisted publications relative to those that did not have such publications. This paper presents new evidence why the claims made in that paper are erroneous. That research claimed to show evidence that Beall’s list of potential predatory journals were positively correlated with internal research awards based on a simple correlation without reporting p-values. We show that the correlation between Beall and ranked journal publications with awards is not statistically significantly different from zero after computing p-values and confidence intervals based on his published results, and by collecting the data from the same public domains as Pyne. We also show that the correlation between Beall and unranked publications with awards depends on only two observations and cannot lead to generalizations.