Contrasting cross-correlation: Meta-analyses of the associations between citations and 13 altmetrics, incorporating moderating variables

Since the Altmetric Manifesto was published in 2010, a large number of studies have quantitatively examined altmetric-citation associations to assess if and how altmetrics differ from citations. These efforts resulted in a diverse array of observations that varies within and between indicators. These studies also differ in the measurement environment, potentially introducing confounding variables. We sought here to clarify this diversity with a meta-analysis and examine if any of several factors moderated the altmetric-citation relationship. We identified relevant literature examining altmetric-citation correlations in the Web of Science and extracted from each study the correlation coefficient, sample size, and potential moderator variables, such as publication year, field, and citation source. We collated 914 correlation coefficients from 111 studies examining 13 altmetric indicators. We fitted multi-level random-effects meta-analysis models for each altmetric indicator individually and examined moderators for Mendeley, Twitter, Altmetric Attention Score, and usage metrics. Mendeley, usage metrics, ResearchGate, Twitter, and peer ratings showed the strongest association with citations, while Facebook, Wikipedia, Google + , blogs, news, and Reddit were only weakly associated with citations. No variable consistently moderated the altmetric-citation relationship, suggesting that, when associations were observed, they were structural and persistent. Altmetrics that did not demonstrate an association evidently differ from citations in the impact construct measured. Hence, our study highlights the diversity among altmetrics channels, as the characteristics of each channel partly define the potential mentions of research in these channels, suggesting a nuanced application of the diverse channels.