IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i1d10.1007_s11192-020-03734-9.html
   My bibliography  Save this article

Do open access journal articles experience a citation advantage? Results and methodological reflections of an application of multiple measures to an analysis by WoS subject areas

Author

Listed:
  • Isabel Basson

    (Stellenbosch University)

  • Jaco P. Blanckenberg

    (Stellenbosch University)

  • Heidi Prozesky

    (Stellenbosch University)

Abstract

This study is one of the first that uses the recently introduced open access (OA) labels in the Web of Science (WoS) metadata to investigate whether OA articles published in Directory of Open Access Journals (DOAJ) listed journals experience a citation advantage in comparison to subscription journal articles, specifically those of which no self-archived versions are available. Bibliometric data on all articles and reviews indexed in WoS, and published from 2013 to 2015, were analysed. In addition to normalised citation score (NCS), we used two additional measures of citation advantage: whether an article was cited at all; and whether an article is among the most frequently cited percentile of articles within its respective subject area (pptopX %). For each WoS subject area, the strength of the relationship between access status (whether an article was published in an OA journal) and each of these three measures was calculated. We found that OA journal articles experience a citation advantage in very few subject areas and, in most of these subject areas, the citation advantage was found on only a single measure of citation advantage, namely whether the article was cited at all. Our results lead us to conclude that access status accounts for little of the variability in the number of citations an article accumulates. The methodology and the calculations that were used in this study are described in detail and we believe that the lessons we learnt, and the recommendations we make, will be of much use to future researchers interested in using the WoS OA labels, and to the field of citation advantage in general.

Suggested Citation

  • Isabel Basson & Jaco P. Blanckenberg & Heidi Prozesky, 2021. "Do open access journal articles experience a citation advantage? Results and methodological reflections of an application of multiple measures to an analysis by WoS subject areas," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 459-484, January.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:1:d:10.1007_s11192-020-03734-9
    DOI: 10.1007/s11192-020-03734-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03734-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-020-03734-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Philip M. Davis & Michael J. Fromerth, 2007. "Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles?," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(2), pages 203-215, May.
    2. Shaibu Mohammed & Anthony Morgan & Emmanuel Nyantakyi, 2020. "On the influence of uncited publications on a researcher’s h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1791-1799, March.
    3. Pablo Dorta-González & Sara M. González-Betancor & María Isabel Dorta-González, 2017. "Reconsidering the gold open access citation advantage postulate in a multidisciplinary context: an analysis of the subject categories in the Web of Science database 2009–2014," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 877-901, August.
    4. Thed N. Van Leeuwen & Henk F. Moed & Robert J. W. Tijssen & Martijn S. Visser & Anthony F. J. Van Raan, 2001. "Language biases in the coverage of the Science Citation Index and its consequencesfor international comparisons of national research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 51(1), pages 335-346, April.
    5. Craig, Iain D. & Plume, Andrew M. & McVeigh, Marie E. & Pringle, James & Amin, Mayur, 2007. "Do open access articles have greater citation impact?," Journal of Informetrics, Elsevier, vol. 1(3), pages 239-248.
    6. Quentin L. Burrell, 2002. "Will this paper ever be cited?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 53(3), pages 232-235.
    7. Vahid Garousi & João M. Fernandes, 2017. "Quantity versus impact of software engineering papers: a quantitative study," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 963-1006, August.
    8. Thed N. Van Leeuwen & Martijn S. Visser & Henk F. Moed & Ton J. Nederhof & Anthony F. J. Van Raan, 2003. "The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 57(2), pages 257-280, June.
    9. Hajar Sotudeh & Abbas Horri, 2009. "Countries positioning in open access journals system: An investigation of citation distribution patterns," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(1), pages 7-31, October.
    10. Ludo Waltman & Michael Schreiber, 2013. "On the calculation of percentile-based bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 372-379, February.
    11. Naomi Fukuzawa, 2017. "Characteristics of papers published in journals: an analysis of open access journals, country of publication, and languages used," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 1007-1023, August.
    12. Hajar Sotudeh & Abbas Horri, 2007. "The citation performance of open access journals: A disciplinary investigation of citation distribution models," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(13), pages 2145-2156, November.
    13. Robert J. W. Tijssen & Martijn S. Visser & Thed N. van Leeuwen, 2002. "Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference?," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(3), pages 381-397, July.
    14. Thed N. van Leeuwen & Clifford Tatum & Paul F. Wouters, 2018. "Exploring possibilities to use bibliometric data to monitor gold open access publishing at the national level," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(9), pages 1161-1173, September.
    15. Sandra Miguel & Ely Francina Tannuri de Oliveira & Maria Cláudia Cabrini Grácio, 2016. "Scientific Production on Open Access: A Worldwide Bibliometric Analysis in the Academic and Scientific Context," Publications, MDPI, vol. 4(1), pages 1-15, January.
    16. Ludo Waltman & Michael Schreiber, 2013. "On the calculation of percentile‐based bibliometric indicators," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 372-379, February.
    17. Gunther Eysenbach, 2006. "Citation Advantage of Open Access Articles," Working Papers id:626, eSocialSciences.
    18. Iman Tahamtan & Askar Safipour Afshar & Khadijeh Ahamdzadeh, 2016. "Factors affecting number of citations: a comprehensive review of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1195-1225, June.
    19. Dag W. Aksnes, 2003. "A macro study of self-citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 56(2), pages 235-246, February.
    20. Yassine Gargouri & Chawki Hajjem & Vincent Larivière & Yves Gingras & Les Carr & Tim Brody & Stevan Harnad, 2010. "Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research," PLOS ONE, Public Library of Science, vol. 5(10), pages 1-12, October.
    21. Wolfgang Glänzel & Bart Thijs, 2004. "The influence of author self-citations on bibliometric macro indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 59(3), pages 281-310, March.
    22. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Qianjin Zong & Zhihong Huang & Jiaru Huang, 2023. "Can open access increase LIS research’s policy impact? Using regression analysis and causal inference," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4825-4854, August.
    2. Chompunuch Saravudecha & Duangruthai Na Thungfai & Chananthida Phasom & Sodsri Gunta-in & Aorrakanya Metha & Peangkobfah Punyaphet & Tippawan Sookruay & Wannachai Sakuludomkan & Nut Koonrungsesomboon, 2023. "Hybrid Gold Open Access Citation Advantage in Clinical Medicine: Analysis of Hybrid Journals in the Web of Science," Publications, MDPI, vol. 11(2), pages 1-9, March.
    3. Elizabeth S. Vieira, 2023. "The influence of research collaboration on citation impact: the countries in the European Innovation Scoreboard," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3555-3579, June.
    4. Liwei Zhang & Liang Ma, 2024. "Different open access routes, varying societal impacts: evidence from the Royal Society biological journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(6), pages 3407-3431, June.
    5. Dias, Luis C. & Lev, Benjamin & Anderson, James B., 2023. "Low cited articles in operations research / management science," Omega, Elsevier, vol. 115(C).
    6. Martorell Cunil, Onofre & Otero González, Luis & Durán Santomil, Pablo & Mulet Forteza, Carlos, 2023. "How to accomplish a highly cited paper in the tourism, leisure and hospitality field," Journal of Business Research, Elsevier, vol. 157(C).
    7. Fernando Almeida & José Morais & José Duarte Santos, 2022. "A Bibliometric Analysis of the Scientific Outcomes of European Projects on the Digital Transformation of SMEs," Publications, MDPI, vol. 10(4), pages 1-18, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sergio Copiello, 2019. "The open access citation premium may depend on the openness and inclusiveness of the indexing database, but the relationship is controversial because it is ambiguous where the open access boundary lie," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 995-1018, November.
    2. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    3. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    4. Wang, Zhiqi & Chen, Yue & Glänzel, Wolfgang, 2020. "Preprints as accelerator of scholarly communication: An empirical analysis in Mathematics," Journal of Informetrics, Elsevier, vol. 14(4).
    5. Hamdi A. Al-Jamimi & Galal M. BinMakhashen & Lutz Bornmann & Yousif Ahmed Al Wajih, 2023. "Saudi Arabia research: academic insights and trend analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5595-5627, October.
    6. Iman Tahamtan & Askar Safipour Afshar & Khadijeh Ahamdzadeh, 2016. "Factors affecting number of citations: a comprehensive review of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1195-1225, June.
    7. Mike Thelwall, 2019. "The influence of highly cited papers on field normalised indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 519-537, February.
    8. Hajar Sotudeh & Zohreh Estakhr, 2018. "Sustainability of open access citation advantage: the case of Elsevier’s author-pays hybrid open access journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 563-576, April.
    9. Marcello D’Agostino & Valentino Dardanoni & Roberto Ghiselli Ricci, 2017. "How to standardize (if you must)," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 825-843, November.
    10. Alonso Rodríguez-Navarro & Ricardo Brito, 2019. "Probability and expected frequency of breakthroughs: basis and use of a robust method of research assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 213-235, April.
    11. Elizabeth S. Vieira, 2023. "The influence of research collaboration on citation impact: the countries in the European Innovation Scoreboard," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3555-3579, June.
    12. Chompunuch Saravudecha & Duangruthai Na Thungfai & Chananthida Phasom & Sodsri Gunta-in & Aorrakanya Metha & Peangkobfah Punyaphet & Tippawan Sookruay & Wannachai Sakuludomkan & Nut Koonrungsesomboon, 2023. "Hybrid Gold Open Access Citation Advantage in Clinical Medicine: Analysis of Hybrid Journals in the Web of Science," Publications, MDPI, vol. 11(2), pages 1-9, March.
    13. Waltman, Ludo & van Eck, Nees Jan, 2013. "A systematic empirical comparison of different approaches for normalizing citation impact indicators," Journal of Informetrics, Elsevier, vol. 7(4), pages 833-849.
    14. Aurelia Magdalena Pisoschi & Claudia Gabriela Pisoschi, 2016. "Is open access the solution to increase the impact of scientific journals?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1075-1095, November.
    15. Teja Koler-Povh & Primož Južnič & Goran Turk, 2014. "Impact of open access on citation of scholarly publications in the field of civil engineering," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1033-1045, February.
    16. Rodríguez-Navarro, Alonso & Brito, Ricardo, 2018. "Technological research in the EU is less efficient than the world average. EU research policy risks Europeans’ future," Journal of Informetrics, Elsevier, vol. 12(3), pages 718-731.
    17. Alexander Cuntz & Frank Mueller-Langer & Alessio Muscarnera & Prince C. Oguguo & Marc Scheufen, 2024. "Access to science and innovation in the developing world," WIPO Economic Research Working Papers 78, World Intellectual Property Organization - Economics and Statistics Division.
    18. Hajar Sotudeh & Zahra Ghasempour & Maryam Yaghtin, 2015. "The citation advantage of author-pays model: the case of Springer and Elsevier OA journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(2), pages 581-608, August.
    19. Juliana Loureiro Almeida Campos & André Sobral & Josivan Soares Silva & Thiago Antonio Sousa Araújo & Washington Soares Ferreira-Júnior & Flávia Rosa Santoro & Gilney Charll Santos & Ulysses Paulino A, 2016. "Insularity and citation behavior of scientific articles in young fields: the case of ethnobiology," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1037-1055, November.
    20. Alessio J. G. Brown & Klaus F. Zimmermann, 2017. "Three decades of publishing research in population economics," Journal of Population Economics, Springer;European Society for Population Economics, vol. 30(1), pages 11-27, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:1:d:10.1007_s11192-020-03734-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.