IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v8y2014i2p421-430.html
   My bibliography  Save this article

Do Article Influence scores overestimate the citation impact of social science journals in subfields that are related to higher-impact natural science disciplines?

Author

Listed:
  • Walters, William H.

Abstract

Unlike Impact Factors (IF), Article Influence (AI) scores assign greater weight to citations that appear in highly cited journals. The natural sciences tend to have higher citation rates than the social sciences. We might therefore expect that relative to IF, AI overestimates the citation impact of social science journals in subfields that are related to (and presumably cited in) higher-impact natural science disciplines. This study evaluates that assertion through a set of simple and multiple regressions covering seven social science disciplines: anthropology, communication, economics, education, library and information science, psychology, and sociology. Contrary to expectations, AI underestimates 5IF (five-year Impact Factor) for journals in science-related subfields such as scientific communication, science education, scientometrics, biopsychology, and medical sociology. Journals in these subfields have low AI scores relative to their 5IF values. Moreover, the effect of science-related status is considerable—typically 0.60 5IF units or 0.50 SD. This effect is independent of the more general finding that AI scores underestimate 5IF for higher-impact journals. It is also independent of the very modest curvilinearity in the relationship between AI and 5IF.

Suggested Citation

  • Walters, William H., 2014. "Do Article Influence scores overestimate the citation impact of social science journals in subfields that are related to higher-impact natural science disciplines?," Journal of Informetrics, Elsevier, vol. 8(2), pages 421-430.
    • Handle: RePEc:eee:infome:v:8:y:2014:i:2:p:421-430
    DOI: 10.1016/j.joi.2014.02.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157714000273
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2014.02.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Philip M. Davis, 2008. "Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(13), pages 2186-2188, November.
    2. Alexandru T. Balaban, 2012. "Positive and negative aspects of citation indices and journal impact factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 241-247, August.
    3. Chia-Lin Chang & Michael McAleer & Les Oxley, 2011. "What makes a great journal great in the sciences? Which came first, the chicken or the egg?," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 17-40, April.
    4. Benjamin M. Althouse & Jevin D. West & Carl T. Bergstrom & Theodore Bergstrom, 2009. "Differences in impact factor across fields and over time," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(1), pages 27-34, January.
    5. Chia‐Lin Chang & Michael McAleer & Les Oxley, 2011. "What Makes A Great Journal Great In Economics? The Singer Not The Song," Journal of Economic Surveys, Wiley Blackwell, vol. 25(2), pages 326-361, April.
    6. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, vol. 22(1), pages 77-88, March.
    7. Waltman, Ludo & van Eck, Nees Jan & van Leeuwen, Thed N. & Visser, Martijn S., 2013. "Some modifications to the SNIP journal impact indicator," Journal of Informetrics, Elsevier, vol. 7(2), pages 272-285.
    8. Mark R. Elkins & Christopher G. Maher & Robert D. Herbert & Anne M. Moseley & Catherine Sherrington, 2010. "Correlation between the Journal Impact Factor and three other journal citation indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 81-93, October.
    9. Jevin West & Theodore Bergstrom & Carl T. Bergstrom, 2010. "Big Macs and Eigenfactor scores: Don't let correlation coefficients fool you," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(9), pages 1800-1807, September.
    10. Lawrence Smolinsky & Aaron Lercher, 2012. "Citation rates in mathematics: a study of variation by subdiscipline," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 911-924, June.
    11. Ludo Waltman & Nees Jan van Eck, 2010. "The relation between Eigenfactor, audience factor, and influence weight," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(7), pages 1476-1486, July.
    12. Moed, Henk F., 2010. "Measuring contextual citation impact of scientific journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 265-277.
    13. Loet Leydesdorff, 2008. "Caveats for the use of citation indicators in research and journal evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(2), pages 278-287, January.
    14. Judit Bar-Ilan, 2012. "Journal report card," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 249-260, August.
    15. Kristie M. Engemann & Howard J. Wall, 2009. "A journal ranking for the ambitious economist," Review, Federal Reserve Bank of St. Louis, vol. 91(May), pages 127-140.
    16. Jerome K. Vanclay, 2009. "Bias in the journal impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(1), pages 3-12, January.
    17. Franceschet, Massimo, 2010. "Journal influence factors," Journal of Informetrics, Elsevier, vol. 4(3), pages 239-248.
    18. Jonathan M. Levitt & Mike Thelwall, 2008. "Is multidisciplinary research more highly cited? A macrolevel study," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(12), pages 1973-1984, October.
    19. Thed Leeuwen, 2012. "Discussing some basic critique on Journal Impact Factors: revision of earlier comments," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 443-455, August.
    20. Ludo Waltman & Nees Jan van Eck, 2010. "The relation between Eigenfactor, audience factor, and influence weight," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1476-1486, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Dašić Predrag, 2015. "State and Analysis of Scientific Journals in the Field of “Economic Sciences” for the Period 1995-2014," Economic Themes, Sciendo, vol. 53(4), pages 547-581, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Finardi, Ugo, 2013. "Correlation between Journal Impact Factor and Citation Performance: An experimental study," Journal of Informetrics, Elsevier, vol. 7(2), pages 357-370.
    3. Chang, C-L. & McAleer, M.J. & Oxley, L., 2012. "Journal Impact Factor, Eigenfactor, Journal Influence and Article Influence," Econometric Institute Research Papers EI 2012-27, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    4. Chang, Chia-Lin & McAleer, Michael & Oxley, Les, 2013. "Coercive journal self citations, impact factor, Journal Influence and Article Influence," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 93(C), pages 190-197.
    5. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    6. Liwei Cai & Jiahao Tian & Jiaying Liu & Xiaomei Bai & Ivan Lee & Xiangjie Kong & Feng Xia, 2019. "Scholarly impact assessment: a survey of citation weighting solutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 453-478, February.
    7. Pajić, Dejan, 2015. "On the stability of citation-based journal rankings," Journal of Informetrics, Elsevier, vol. 9(4), pages 990-1006.
    8. P. Dorta-González & M. I. Dorta-González, 2013. "Comparing journals from different fields of science and social science through a JCR subject categories normalized impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(2), pages 645-672, May.
    9. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    10. Vanclay, Jerome K., 2012. "Publication patterns of award-winning forest scientists and implications for the Australian ERA journal ranking," Journal of Informetrics, Elsevier, vol. 6(1), pages 19-26.
    11. Waltman, Ludo & van Eck, Nees Jan, 2013. "A systematic empirical comparison of different approaches for normalizing citation impact indicators," Journal of Informetrics, Elsevier, vol. 7(4), pages 833-849.
    12. Chia-Lin Chang & Michael McAleer & Les Oxley, 2010. "Journal Impact Factor Versus Eigenfactor and Article Influence," KIER Working Papers 737, Kyoto University, Institute of Economic Research.
    13. Antonin Mac'e, 2017. "The Limits of Citation Counts," Papers 1711.02695, arXiv.org, revised Sep 2023.
    14. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    15. Damien Besancenot & Abdelghani Maddi, 2019. "Should citations be weighted to assess the influence of an academic article?," Economics Bulletin, AccessEcon, vol. 435(1), pages 435-445.
    16. Peter Jacso, 2012. "Grim tales about the impact factor and the h-index in the Web of Science and the Journal Citation Reports databases: reflections on Vanclay’s criticism," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 325-354, August.
    17. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    18. Loet Leydesdorff, 2012. "Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 355-365, August.
    19. Loet Leydesdorff & Ping Zhou & Lutz Bornmann, 2013. "How can journal impact factors be normalized across fields of science? An assessment in terms of percentile ranks and fractional counts," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 96-107, January.
    20. Gabriel-Alexandru Vîiu & Mihai Păunescu, 2021. "The citation impact of articles from which authors gained monetary rewards based on journal metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 4941-4974, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:8:y:2014:i:2:p:421-430. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.