IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v110y2017i2d10.1007_s11192-016-2204-y.html
   My bibliography  Save this article

Impact evaluation of open source software: an Altmetrics perspective

Author

Listed:
  • Rongying Zhao

    (Wuhan University
    Wuhan University)

  • Mingkun Wei

    (Wuhan University
    Wuhan University)

Abstract

With the emergence of Web 2.0, an online platform which encourages online creation of next generation tools, communication has become a nigh-indispensable tool for researchers. Allowing them to acquire, spread, and share research achievements, with a free flow of ideas online. At present, there are a growing number of studies on non-traditional evaluation indicators, but there is much fewer research focused on the software evaluation, especially for open source research software. What this research focuses on is; with the use of the open source project ‘Depsy’, this research evaluated and analyzed data collected from downloads made online through these open source software. Altmetrics cannot be confined to traditional measurable indicators. That the importance of the open source software used, and its position in the online community is itself a strong measure for academic impact and success, which is all too often overlooked in research. The research can also conclude that the multiplexing of software online, through the citation of a citation, ultimately leads to an online peered review system within the community, effectively developing and maintaining through open use software itself. Moreover, the benefits of such a system has only just begun to come to fruition, having a strong impact on academic research, and predicting research impact.

Suggested Citation

  • Rongying Zhao & Mingkun Wei, 2017. "Impact evaluation of open source software: an Altmetrics perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 1017-1033, February.
    • Handle: RePEc:spr:scient:v:110:y:2017:i:2:d:10.1007_s11192-016-2204-y
    DOI: 10.1007/s11192-016-2204-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-016-2204-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-016-2204-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mike Thelwall & Kevan Buckley & Georgios Paltoglou, 2011. "Sentiment in Twitter events," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(2), pages 406-418, February.
    2. Mike Thelwall & Kevan Buckley & Georgios Paltoglou, 2011. "Sentiment in Twitter events," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(2), pages 406-418, February.
    3. Hadas Shema & Judit Bar-Ilan & Mike Thelwall, 2015. "How is research blogged? A content analysis approach," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(6), pages 1136-1149, June.
    4. Kim Holmberg & Mike Thelwall, 2014. "Disciplinary differences in Twitter scholarly communication," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1027-1042, November.
    5. Ehsan Mohammadi & Mike Thelwall, 2014. "Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(8), pages 1627-1638, August.
    6. Isabella Peters & Peter Kraker & Elisabeth Lex & Christian Gumpenberger & Juan Gorraiz, 2016. "Research data explored: an extended analysis of citations and altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 723-744, May.
    7. Björn Hammarfelt, 2014. "Using altmetrics for assessing research impact in the humanities," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1419-1430, November.
    8. William Pearson, 1966. "Estimation of a correlation coefficient from an uncertainty measure," Psychometrika, Springer;The Psychometric Society, vol. 31(3), pages 421-433, September.
    9. Blaise Cronin & Lokman I. Meho, 2008. "The shifting balance of intellectual trade in information studies," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(4), pages 551-564, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yuzhuo Wang & Kai Li, 2024. "How do official software citation formats evolve over time? A longitudinal analysis of R programming language packages," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 3997-4019, July.
    2. Rongying Zhao & Mingkun Wei, 2017. "Academic impact evaluation of Wechat in view of social media perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1777-1791, September.
    3. Wang, Yuzhuo & Zhang, Chengzhi, 2020. "Using the full-text content of academic articles to identify and evaluate algorithm entities in the domain of natural language processing," Journal of Informetrics, Elsevier, vol. 14(4).
    4. Yuzhuo Wang & Chengzhi Zhang & Kai Li, 2022. "A review on method entities in the academic literature: extraction, evaluation, and application," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2479-2520, May.
    5. Li, Kai & Yan, Erjia, 2018. "Co-mention network of R packages: Scientific impact and clustering structure," Journal of Informetrics, Elsevier, vol. 12(1), pages 87-100.
    6. Pan, Xuelian & Yan, Erjia & Cui, Ming & Hua, Weina, 2019. "How important is software to library and information science research? A content analysis of full-text publications," Journal of Informetrics, Elsevier, vol. 13(1), pages 397-406.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    2. Bornmann, Lutz, 2014. "Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics," Journal of Informetrics, Elsevier, vol. 8(4), pages 895-903.
    3. Amalia Mas-Bleda & Mike Thelwall, 2016. "Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2007-2030, December.
    4. Chae, Bongsug (Kevin), 2015. "Insights from hashtag #supplychain and Twitter Analytics: Considering Twitter and Twitter data for supply chain practice and research," International Journal of Production Economics, Elsevier, vol. 165(C), pages 247-259.
    5. Hajar Sotudeh & Zeinab Saber & Farzin Ghanbari Aloni & Mahdieh Mirzabeigi & Farshad Khunjush, 2022. "A longitudinal study of the evolution of opinions about open access and its main features: a twitter sentiment analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5587-5611, October.
    6. Lutz Bornmann, 2015. "Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 1123-1144, June.
    7. Ma, Jie & Tse, Ying Kei & Wang, Xiaojun & Zhang, Minhao, 2019. "Examining customer perception and behaviour through social media research – An empirical study of the United Airlines overbooking crisis," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 127(C), pages 192-205.
    8. Luis-Millán González & José Devís-Devís & Maite Pellicer-Chenoll & Miquel Pans & Alberto Pardo-Ibañez & Xavier García-Massó & Fernanda Peset & Fernanda Garzón-Farinós & Víctor Pérez-Samaniego, 2021. "The Impact of COVID-19 on Sport in Twitter: A Quantitative and Qualitative Content Analysis," IJERPH, MDPI, vol. 18(9), pages 1-20, April.
    9. Diana Maynard & Gerhard Gossen & Adam Funk & Marco Fisichella, 2014. "Should I Care about Your Opinion? Detection of Opinion Interestingness and Dynamics in Social Media," Future Internet, MDPI, vol. 6(3), pages 1-25, August.
    10. Isidro F. Aguillo, 2020. "Altmetrics of the Open Access Institutional Repositories: a webometrics approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(3), pages 1181-1192, June.
    11. Karin Sim Smith & Richard McCreadie & Craig Macdonald & Iadh Ounis, 2018. "Regional Sentiment Bias in Social Media Reporting During Crises," Information Systems Frontiers, Springer, vol. 20(5), pages 1013-1025, October.
    12. Beatriz Barros & Ana Fernández-Zubieta & Raul Fidalgo-Merino & Francisco Triguero, 2018. "Scientific knowledge percolation process and social impact: A case study on the biotechnology and microbiology perceptions on Twitter," Science and Public Policy, Oxford University Press, vol. 45(6), pages 804-814.
    13. Lipizzi, Carlo & Iandoli, Luca & Ramirez Marquez, José Emmanuel, 2015. "Extracting and evaluating conversational patterns in social media: A socio-semantic analysis of customers’ reactions to the launch of new products using Twitter streams," International Journal of Information Management, Elsevier, vol. 35(4), pages 490-503.
    14. Thomas T. Hills & Eugenio Proto & Daniel Sgroi & Chanuki Illushka Seresinhe, 2019. "Historical analysis of national subjective wellbeing using millions of digitized books," Nature Human Behaviour, Nature, vol. 3(12), pages 1271-1275, December.
    15. Ping-Yu Hsu & Hong-Tsuen Lei & Shih-Hsiang Huang & Teng Hao Liao & Yao-Chung Lo & Chin-Chun Lo, 2019. "Effects of sentiment on recommendations in social network," Electronic Markets, Springer;IIM University of St. Gallen, vol. 29(2), pages 253-262, June.
    16. Siluo Yang & Xin Xing & Fan Qi & Maria Cláudia Cabrini Grácio, 2021. "Comparison of academic book impact from a disciplinary perspective: an analysis of citations and altmetric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1101-1123, February.
    17. Mingkun Wei & Abdolreza Noroozi Chakoli, 2020. "Evaluating the relationship between the academic and social impact of open access books based on citation behaviors and social media attention," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2401-2420, December.
    18. Widmar, Nicole Olynk & Bir, Courtney & Clifford, McKenna & Slipchenko, Natalya, 2020. "Social media sentimentas an additional performance measure? Examples from iconic theme park destinations," Journal of Retailing and Consumer Services, Elsevier, vol. 56(C).
    19. Mike Thelwall, 2021. "Measuring Societal Impacts Of Research With Altmetrics? Common Problems And Mistakes," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1302-1314, December.
    20. Stefan Stieglitz & Christian Meske & Björn Ross & Milad Mirbabaie, 2020. "Going Back in Time to Predict the Future - The Complex Role of the Data Collection Period in Social Media Analytics," Information Systems Frontiers, Springer, vol. 22(2), pages 395-409, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:110:y:2017:i:2:d:10.1007_s11192-016-2204-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.