Rankings, research styles, and publication cultures: a study of American sociology departments

Rankings have become a major form of quality assessment in higher education over the past few decades. Most rankings rely, to varying extent, on bibliometric indicators intended to capture the quantity and quality of the scientific output of institutions. The growing popularity of this practice has raised a number of concerns, one of the most important being whether evaluations of this sort treat different work styles and publication habits in an unbiased manner and, consequently, whether the resulting rankings properly respect the particular modes of research characteristic of various disciplines and subdisciplines. The research reported in this paper looked at this issue, using data on more than one hundred US sociology departments. Our results showed that institutions that are more quantitative in character are more likely to favor journals over books as the dominant form of scientific communication and fare, in general, considerably better on the National Research Council’s assessment than their qualitative equivalents. After controlling for differences in publication practices, the impact of research style declined but remained statistically significant. It thus seems that the greater preference of qualitative departments for books over articles as publication outlets puts them at a disadvantage as far as quality assessments are concerned, although their lagging behind their quantitative counterparts cannot fully be explained by this factor alone.