IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v120y2019i3d10.1007_s11192-019-03159-z.html
   My bibliography  Save this article

Evaluation and comparison of influence in international Open Access journals between China and USA

Author

Listed:
  • Rongying Zhao

    (Wuhan University
    Wuhan University
    Wuhan University)

  • Xu Wang

    (Wuhan University
    Wuhan University)

Abstract

This paper makes an evaluation and comparison of influence on international Open Access (OA) between China and USA. First, Chinese and American OA journals are taken as an example. Second, combining traditional citation indicators with altmetrics indicators, a process of influence evaluation and comparison model on Chinese and American OA journals is constructed. Third, an evaluation indicator system of OA journals from two dimensions of academic influence and societal influence is constructed too. Lastly, through normalization, correlation analysis, reliability analysis, validity analysis, factor analysis and two-dimensional analysis, the academic and societal influence of OA journals are evaluated and compared between China and USA. It is found that in terms of academic influence and societal influence, the evaluation scores of international OA journals are quite different between China and USA. The influence of the USA is far greater than that of China. The results show a strong positive correlation among the 7 traditional citation indicators based on JCR, and a high positive correlation among the 8 altmetrics indicators based on Altmetrics Explorer, and the indicators have a significant consistency both overall and internal. There is a highly positive correlation between academic influence and societal influence scores for the Chinese and American OA journals. Overall, the results confirm that the evaluation of societal influence of OA journals is used as supplement to the evaluation of academic influence. Altmetrics is also a powerful tool to measure the societal influence of OA journals, but it cannot replace the traditional citation indicators at present.

Suggested Citation

  • Rongying Zhao & Xu Wang, 2019. "Evaluation and comparison of influence in international Open Access journals between China and USA," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1091-1110, September.
    • Handle: RePEc:spr:scient:v:120:y:2019:i:3:d:10.1007_s11192-019-03159-z
    DOI: 10.1007/s11192-019-03159-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-019-03159-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-019-03159-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Reinhard Werner, 2015. "The focus on bibliometrics makes papers less useful," Nature, Nature, vol. 517(7534), pages 245-245, January.
    2. Houqiang Yu, 2017. "Context of altmetrics data matters: an investigation of count type and user category," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 267-283, April.
    3. Xiaomei Bai & Feng Xia & Ivan Lee & Jun Zhang & Zhaolong Ning, 2016. "Identifying Anomalous Citations for Objective Evaluation of Scholarly Article Impact," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-15, September.
    4. Jingfeng Xia, 2010. "A longitudinal study of scholars attitudes and behaviors toward open‐access journal publishing," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(3), pages 615-624, March.
    5. Stevan Harnad, 2009. "Open access scientometrics and the UK Research Assessment Exercise," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(1), pages 147-156, April.
    6. Jingfeng Xia, 2010. "A longitudinal study of scholars attitudes and behaviors toward open-access journal publishing," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(3), pages 615-624, March.
    7. Aurelia Magdalena Pisoschi & Claudia Gabriela Pisoschi, 2016. "Is open access the solution to increase the impact of scientific journals?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1075-1095, November.
    8. Maja Jokić & Andrea Mervar & Stjepan Mateljan, 2018. "Scientific potential of European fully open access journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1373-1394, March.
    9. Cassidy R. Sugimoto & Sam Work & Vincent Larivière & Stefanie Haustein, 2017. "Scholarly use of social media and altmetrics: A review of the literature," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(9), pages 2037-2062, September.
    10. Kim Holmberg & Han Woo Park, 2018. "An altmetric investigation of the online visibility of South Korea-based scientific journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 603-613, October.
    11. Siluo Yang & Xin Xing & Dietmar Wolfram, 2018. "Difference in the impact of open-access papers published by China and the USA," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1017-1037, May.
    12. Bornmann, Lutz, 2014. "Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics," Journal of Informetrics, Elsevier, vol. 8(4), pages 895-903.
    13. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    14. Henk F. Moed & Gali Halevi, 2015. "Multidimensional assessment of scholarly research impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(10), pages 1988-2002, October.
    15. Lutz Bornmann & Robin Haunschild, 2018. "Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-12, May.
    16. Thelwall, Mike & Nevill, Tamara, 2018. "Could scientists use Altmetric.com scores to predict longer term citation counts?," Journal of Informetrics, Elsevier, vol. 12(1), pages 237-248.
    17. Hajar Sotudeh & Zohreh Estakhr, 2018. "Sustainability of open access citation advantage: the case of Elsevier’s author-pays hybrid open access journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 563-576, April.
    18. Nabeil Maflahi & Mike Thelwall, 2018. "How quickly do publications get read? The evolution of mendeley reader counts for new articles," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(1), pages 158-167, January.
    19. Erjia Yan & Kai Li, 2018. "Which domains do open†access journals do best in? A 5†year longitudinal study," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(6), pages 844-856, June.
    20. Cenyu Shen, 2017. "Open Access Scholarly Journal Publishing in Chinese," Publications, MDPI, vol. 5(4), pages 1-17, September.
    21. Teja Koler-Povh & Primož Južnič & Goran Turk, 2014. "Impact of open access on citation of scholarly publications in the field of civil engineering," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1033-1045, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sumiko Asai, 2020. "The effect of collaboration with large publishers on the internationality and influence of open access journals for research institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 663-677, July.
    2. Wenqi Fu & Jie Xu & Qing Fang & Jingjia Ding & Hanqing Ma, 2023. "Going Open Access: The Attitudes and Actions of Scientific Journal Editors in China," Publications, MDPI, vol. 12(1), pages 1-17, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Siluo Yang & Xin Xing & Dietmar Wolfram, 2018. "Difference in the impact of open-access papers published by China and the USA," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1017-1037, May.
    2. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    3. Yang, Siluo & Zheng, Mengxue & Yu, Yonghao & Wolfram, Dietmar, 2021. "Are Altmetric.com scores effective for research impact evaluation in the social sciences and humanities?," Journal of Informetrics, Elsevier, vol. 15(1).
    4. Lutz Bornmann & Rüdiger Mutz & Robin Haunschild & Felix Moya-Anegon & Mirko Almeida Madeira Clemente & Moritz Stefaner, 2021. "Mapping the impact of papers on various status groups in excellencemapping.net: a new release of the excellence mapping tool based on citation and reader scores," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9305-9331, November.
    5. Hajar Sotudeh & Zeinab Saber & Farzin Ghanbari Aloni & Mahdieh Mirzabeigi & Farshad Khunjush, 2022. "A longitudinal study of the evolution of opinions about open access and its main features: a twitter sentiment analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5587-5611, October.
    6. Xi Zhang & Xianhai Wang & Hongke Zhao & Patricia Ordóñez de Pablos & Yongqiang Sun & Hui Xiong, 2019. "An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1311-1344, June.
    7. Ying Guo & Xiantao Xiao, 2022. "Author-level altmetrics for the evaluation of Chinese scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 973-990, February.
    8. Wang, Zhiqi & Chen, Yue & Glänzel, Wolfgang, 2020. "Preprints as accelerator of scholarly communication: An empirical analysis in Mathematics," Journal of Informetrics, Elsevier, vol. 14(4).
    9. Shahzad, Murtuza & Alhoori, Hamed & Freedman, Reva & Rahman, Shaikh Abdul, 2022. "Quantifying the online long-term interest in research," Journal of Informetrics, Elsevier, vol. 16(2).
    10. Hyejin Park & Han Woo Park, 2018. "Research evaluation of Asian countries using altmetrics: comparing South Korea, Japan, Taiwan, Singapore, and China," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(2), pages 771-788, November.
    11. Li, Huixu & Liu, Lanjian & Wang, Xianwen, 2021. "The open access effect in social media exposure of scholarly articles: A matched-pair analysis," Journal of Informetrics, Elsevier, vol. 15(3).
    12. Lu Liu & Benjamin F. Jones & Brian Uzzi & Dashun Wang, 2023. "Data, measurement and empirical methods in the science of science," Nature Human Behaviour, Nature, vol. 7(7), pages 1046-1058, July.
    13. González-Betancor, Sara M. & Dorta-González, Pablo, 2023. "Does society show differential attention to researchers based on gender and field?," Journal of Informetrics, Elsevier, vol. 17(4).
    14. Sergio Copiello, 2019. "The open access citation premium may depend on the openness and inclusiveness of the indexing database, but the relationship is controversial because it is ambiguous where the open access boundary lie," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 995-1018, November.
    15. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    16. Cristina López-Duarte & Marta M. Vidal-Suárez & Belén González-Díaz, 2019. "Cross-national distance and international business: an analysis of the most influential recent models," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 173-208, October.
    17. Tahamtan, Iman & Bornmann, Lutz, 2018. "Creativity in science and the link to cited references: Is the creative potential of papers reflected in their cited references?," Journal of Informetrics, Elsevier, vol. 12(3), pages 906-930.
    18. Hou, Jianhua & Yang, Xiucai, 2020. "Social media-based sleeping beauties: Defining, identifying and features," Journal of Informetrics, Elsevier, vol. 14(2).
    19. Xiaozan Lyu & Rodrigo Costas, 2020. "How do academic topics shift across altmetric sources? A case study of the research area of Big Data," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 909-943, May.
    20. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:120:y:2019:i:3:d:10.1007_s11192-019-03159-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.