IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v125y2020i2d10.1007_s11192-020-03613-3.html
   My bibliography  Save this article

Can social media usage of scientific literature predict journal indices of AJG, SNIP and JCR? An altmetric study of economics

Author

Listed:
  • Dorte Drongstrup

    (University Library of Southern Denmark)

  • Shafaq Malik

    (Information Technology University)

  • Naif Radi Aljohani

    (King Abdulaziz University)

  • Salem Alelyani

    (King Khalid University
    King Khalid University)

  • Iqra Safder

    (Information Technology University)

  • Saeed-Ul Hassan

    (Information Technology University)

Abstract

Altmetrics are often praised as an alternative or complement to classic bibliometric metrics, especially in the social sciences discipline. However, empirical investigations of altmetrics concerning the social sciences are scarce. This study investigates the extent to which economic research is shared on social media platforms with an emphasis on mentions in policy documents in addition to other mentions such as Twitter or Facebook. Moreover, this study explores machine learning models to predict the likelihood of a research article being classified into the top-quality tier of a journal ranking based on the altmetric mentions. The included journal rankings are the academic journal guide (AJG), source normalized impact per paper (SNIP) and journal citation reports (JCR). The investigated journals have been selected based on the AJG list and extracted from Altmetric.com data. After applying extensive data cleaning on the extracted data, a final set of 55,560 journal article records is obtained. The results indicate that the average number of policy mentions of the publications of economics journals is higher than the other subject areas included in the AJG list. Moreover, the publications in top-ranking economic journals are more likely to have a higher average number of policy mentions. Policy and Twitter mentions are presented as the most significant and informative social media mentions in demonstrating the broader impact and dissemination of Economics discipline followed by Blogs, Facebook, Wikipedia, and News. The results show that Support Vector Machine and Logistic Regression performed best in classifying the journal ranking tiers i.e. SNIP-based with 77% accuracy, JCR-based with 71% accuracy, and AJG-based with 66% accuracy. The models classified the ranking tier AJG18 with lower accuracy than SNIP and JCR. This might be because the AJG18 rankings are based on expert opinion, whereas SNIP and JCR are based on citations.

Suggested Citation

  • Dorte Drongstrup & Shafaq Malik & Naif Radi Aljohani & Salem Alelyani & Iqra Safder & Saeed-Ul Hassan, 2020. "Can social media usage of scientific literature predict journal indices of AJG, SNIP and JCR? An altmetric study of economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1541-1558, November.
    • Handle: RePEc:spr:scient:v:125:y:2020:i:2:d:10.1007_s11192-020-03613-3
    DOI: 10.1007/s11192-020-03613-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03613-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-020-03613-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kim Holmberg & Mike Thelwall, 2014. "Disciplinary differences in Twitter scholarly communication," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1027-1042, November.
    2. Saeed-Ul Hassan & Naif R. Aljohani & Mudassir Shabbir & Umair Ali & Sehrish Iqbal & Raheem Sarwar & Eugenio Martínez-Cámara & Sebastián Ventura & Francisco Herrera, 2020. "Tweet Coupling: a social media methodology for clustering scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 973-991, August.
    3. Anton J. Nederhof, 2006. "Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 81-100, January.
    4. Waltman, Ludo & van Eck, Nees Jan & van Leeuwen, Thed N. & Visser, Martijn S., 2013. "Some modifications to the SNIP journal impact indicator," Journal of Informetrics, Elsevier, vol. 7(2), pages 272-285.
    5. Robin Haunschild & Lutz Bornmann, 2017. "How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1209-1216, March.
    6. Htet Htet Aung & Han Zheng & Mojisola Erdt & Ashley Sara Aw & Sei‐Ching Joanna Sin & Yin‐Leng Theng, 2019. "Investigating familiarity and usage of traditional metrics and altmetrics," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 70(8), pages 872-887, August.
    7. Moed, Henk F., 2010. "Measuring contextual citation impact of scientific journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 265-277.
    8. Nuredini, Kaltrina & Peters, Isabella, 2015. "Economic and Business Studies Journals and Readership Information from Mendeley," EconStor Conference Papers 110467, ZBW - Leibniz Information Centre for Economics.
    9. Saeed-Ul Hassan & Mubashir Imran & Sehrish Iqbal & Naif Radi Aljohani & Raheel Nawaz, 2018. "Deep context of citations using machine-learning models in scholarly full-text articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1645-1662, December.
    10. Saeed-Ul Hassan & Timothy D. Bowman & Mudassir Shabbir & Aqsa Akhtar & Mubashir Imran & Naif Radi Aljohani, 2019. "Influential tweeters in relation to highly cited articles in altmetric big data," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 481-493, April.
    11. Saeed-Ul Hassan & Mubashir Imran & Uzair Gillani & Naif Radi Aljohani & Timothy D. Bowman & Fereshteh Didegah, 2017. "Measuring social media activity of scientific literature: an exhaustive comparison of scopus and novel altmetrics big data," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1037-1057, November.
    12. Stefanie Haustein & Isabella Peters & Cassidy R. Sugimoto & Mike Thelwall & Vincent Larivière, 2014. "Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(4), pages 656-669, April.
    13. Rodrigo Costas & Zohreh Zahedi & Paul Wouters, 2015. "Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(10), pages 2003-2019, October.
    14. Björn Hammarfelt, 2014. "Using altmetrics for assessing research impact in the humanities," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1419-1430, November.
    15. Xin Shuai & Alberto Pepe & Johan Bollen, 2012. "How the Scientific Community Reacts to Newly Submitted Preprints: Article Downloads, Twitter Mentions, and Citations," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-8, November.
    16. Anwar Said & Timothy D. Bowman & Rabeeh Ayaz Abbasi & Naif Radi Aljohani & Saeed-Ul Hassan & Raheel Nawaz, 2019. "Mining network-level properties of Twitter altmetrics data," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 217-235, July.
    17. Nuredini, Kaltrina & Peters, Isabella, 2016. "Enriching the knowledge of altmetrics studies by exploring social media metrics for Economic and Business Studies journals," EconStor Conference Papers 146879, ZBW - Leibniz Information Centre for Economics.
    18. Lutz Bornmann & Robin Haunschild & Werner Marx, 2016. "Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1477-1495, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Croft, William L. & Sack, Jörg-Rüdiger, 2022. "Predicting the citation count and CiteScore of journals one year in advance," Journal of Informetrics, Elsevier, vol. 16(4).
    2. Zhihong Huang & Qianjin Zong & Xuerui Ji, 2022. "The associations between scientific collaborations of LIS research and its policy impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6453-6470, November.
    3. Yaxue Ma & Zhichao Ba & Yuxiang Zhao & Jin Mao & Gang Li, 2021. "Understanding and predicting the dissemination of scientific papers on social media: a two-step simultaneous equation modeling–artificial neural network approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 7051-7085, August.
    4. Ying Guo & Xiantao Xiao, 2022. "Author-level altmetrics for the evaluation of Chinese scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 973-990, February.
    5. Wanjun Xia & Tianrui Li & Chongshou Li, 2023. "A review of scientific impact prediction: tasks, features and methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 543-585, January.
    6. Naif Radi Aljohani & Ayman Fayoumi & Saeed-Ul Hassan, 2021. "An in-text citation classification predictive model for a scholarly search system," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5509-5529, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yaxue Ma & Zhichao Ba & Yuxiang Zhao & Jin Mao & Gang Li, 2021. "Understanding and predicting the dissemination of scientific papers on social media: a two-step simultaneous equation modeling–artificial neural network approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 7051-7085, August.
    2. Amalia Mas-Bleda & Mike Thelwall, 2016. "Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2007-2030, December.
    3. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    4. Zhichao Fang & Rodrigo Costas & Paul Wouters, 2022. "User engagement with scholarly tweets of scientific papers: a large-scale and cross-disciplinary analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4523-4546, August.
    5. Jianhua Hou & Da Ma, 2020. "How the high-impact papers formed? A study using data from social media and citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2597-2615, December.
    6. Siluo Yang & Xin Xing & Fan Qi & Maria Cláudia Cabrini Grácio, 2021. "Comparison of academic book impact from a disciplinary perspective: an analysis of citations and altmetric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1101-1123, February.
    7. Kaltrina Nuredini, 2021. "Investigating Altmetric Information For The Top 1000 Journals From Handelsblatt Ranking In Economic And Business Studies," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1315-1343, December.
    8. Saeed-Ul Hassan & Naif R. Aljohani & Mudassir Shabbir & Umair Ali & Sehrish Iqbal & Raheem Sarwar & Eugenio Martínez-Cámara & Sebastián Ventura & Francisco Herrera, 2020. "Tweet Coupling: a social media methodology for clustering scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 973-991, August.
    9. Zhiqi Wang & Wolfgang Glänzel & Yue Chen, 2020. "The impact of preprints in Library and Information Science: an analysis of citations, usage and social attention indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1403-1423, November.
    10. Mike Thelwall & Kayvan Kousha & Mahshid Abdoli & Emma Stuart & Meiko Makita & Paul Wilson & Jonathan Levitt, 2023. "Do altmetric scores reflect article quality? Evidence from the UK Research Excellence Framework 2021," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(5), pages 582-593, May.
    11. Saeed-Ul Hassan & Sehrish Iqbal & Naif R. Aljohani & Salem Alelyani & Alesia Zuccala, 2020. "Introducing the ‘alt-index’ for measuring the social visibility of scientific research," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(3), pages 1407-1419, June.
    12. José Luis Ortega, 2016. "To be or not to be on Twitter, and its relationship with the tweeting and citation of research papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1353-1364, November.
    13. Banshal, Sumit Kumar & Gupta, Solanki & Lathabai, Hiran H & Singh, Vivek Kumar, 2022. "Power Laws in altmetrics: An empirical analysis," Journal of Informetrics, Elsevier, vol. 16(3).
    14. Lutz Bornmann, 2015. "Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 1123-1144, June.
    15. Walters, William H., 2017. "Do subjective journal ratings represent whole journals or typical articles? Unweighted or weighted citation impact?," Journal of Informetrics, Elsevier, vol. 11(3), pages 730-744.
    16. Houqiang Yu, 2017. "Context of altmetrics data matters: an investigation of count type and user category," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 267-283, April.
    17. Mohammadamin Erfanmanesh & A. Noorhidawati & A. Abrizah, 2019. "What can Bookmetrix tell us about the impact of Springer Nature’s books," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 521-536, October.
    18. Mike Thelwall, 2021. "Measuring Societal Impacts Of Research With Altmetrics? Common Problems And Mistakes," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1302-1314, December.
    19. Bornmann, Lutz, 2014. "Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics," Journal of Informetrics, Elsevier, vol. 8(4), pages 895-903.
    20. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:125:y:2020:i:2:d:10.1007_s11192-020-03613-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.