IDEAS home Printed from https://ideas.repec.org/a/gam/jpubli/v11y2023i2p31-d1161937.html
   My bibliography  Save this article

Measuring the Impact and Influence of Scientific Activity in the Humanities and Social Sciences

Author

Listed:
  • Daniela De Filippo

    (Institute of Philosophy (IFS), Spanish National Research Council (CSIC), 26-28 Albasanz Street, 28037 Madrid, Spain)

  • Fernanda Morillo

    (Institute of Philosophy (IFS), Spanish National Research Council (CSIC), 26-28 Albasanz Street, 28037 Madrid, Spain)

  • Borja González-Albo

    (Centre for Human and Social Sciences (CCHS), Spanish National Research Council (CSIC), 26-28 Albasanz Street, 28037 Madrid, Spain)

Abstract

Scientific activity in the Humanities and Social Sciences (HSS) presents special characteristics that require the use of various sources and methodologies to adequately assess its impact and influence on both academic and non-academic audiences. This study aims to explore the validity of traditional and alternative information sources for the analysis of the characteristics of HSS research and its academic impact and influence (considering social, media, informative and political influence). It is also intended to highlight the differences between Humanities (H) and Social Sciences (SS) and analyse the variables that determine the different types of impact and influence of research in each of them. The following sources of information are used: Web of Science, conCIENCIA (institutional database), Google Scholar, Unpaywall, Altmetric.com and Overton, focused on the study of the Spanish National Research Council (CSIC). The results obtained show that institutional sources make local research visible, which has high percentages of open access. The usefulness of alternative sources to measure social, media, informative and political influence is verified, since HSS publications have an important number of mentions. Significant differences are observed between H and SS in terms of publication coverage (higher in H in the institutional database), language (more Spanish in H), open access (higher percentages in SS) and impact measured through conCIENCIA (the greatest number of documents with a high impact is found in H). In addition, the influence on non-academic audiences is increased by the international orientation of research, the greater academic impact, the participation of SS centres and the immediacy of publications. This study is a starting point for future research, as it explores several tools and data sources to analyse the influence of HSS research on different audiences. A comprehensive analysis will also facilitate the proposal of new metrics applied to the HSS assessment, highlighting its importance for society as a whole.

Suggested Citation

  • Daniela De Filippo & Fernanda Morillo & Borja González-Albo, 2023. "Measuring the Impact and Influence of Scientific Activity in the Humanities and Social Sciences," Publications, MDPI, vol. 11(2), pages 1-17, June.
    • Handle: RePEc:gam:jpubli:v:11:y:2023:i:2:p:31-:d:1161937
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2304-6775/11/2/31/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2304-6775/11/2/31/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Emanuel Kulczycki & Tim C. E. Engels & Janne Pölönen & Kasper Bruun & Marta Dušková & Raf Guns & Robert Nowotniak & Michal Petr & Gunnar Sivertsen & Andreja Istenič Starčič & Alesia Zuccala, 2018. "Publication patterns in the social sciences and humanities: evidence from eight European countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 463-486, July.
    2. Anton J. Nederhof, 2006. "Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 81-100, January.
    3. Kuang-hua Chen & Muh-chyun Tang & Chun-mei Wang & Jieh Hsiang, 2015. "Exploring alternative metrics of scholarly performance in the social sciences and humanities in Taiwan," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 97-112, January.
    4. Antonio Eleazar Serrano-López & Peter Ingwersen & Elias Sanz-Casado, 2017. "Wind power research in Wikipedia: Does Wikipedia demonstrate direct influence of research publications and can it be used as adequate source in research evaluation?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1471-1488, September.
    5. Christian Schloegl & Juan Gorraiz, 2010. "Comparison of citation and usage indicators: the case of oncology journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 567-580, March.
    6. Cameron Neylon & Shirley Wu, 2009. "Article-Level Metrics and the Evolution of Scientific Impact," PLOS Biology, Public Library of Science, vol. 7(11), pages 1-6, November.
    7. Christian Gumpenberger & Wolfgang Glänzel & Juan Gorraiz, 2016. "The ecstasy and the agony of the altmetric score," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(2), pages 977-982, August.
    8. Ortega, José Luis, 2015. "Relationship between altmetric and bibliometric indicators across academic social sites: The case of CSIC's members," Journal of Informetrics, Elsevier, vol. 9(1), pages 39-49.
    9. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    10. Yang, Siluo & Zheng, Mengxue & Yu, Yonghao & Wolfram, Dietmar, 2021. "Are Altmetric.com scores effective for research impact evaluation in the social sciences and humanities?," Journal of Informetrics, Elsevier, vol. 15(1).
    11. Michael Taylor, 2020. "An altmetric attention advantage for open access books in the humanities and social sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2523-2543, December.
    12. Ortega, José Luis, 2018. "The life cycle of altmetric impact: A longitudinal study of six metrics from PlumX," Journal of Informetrics, Elsevier, vol. 12(3), pages 579-589.
    13. Fernanda Morillo & Ignacio Santabárbara & Javier Aparicio, 2013. "The automatic normalisation challenge: detailed addresses identification," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 953-966, June.
    14. Thelwall, Mike & Nevill, Tamara, 2018. "Could scientists use Altmetric.com scores to predict longer term citation counts?," Journal of Informetrics, Elsevier, vol. 12(1), pages 237-248.
    15. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    16. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    17. Zhichao Fang & Rodrigo Costas & Wencan Tian & Xianwen Wang & Paul Wouters, 2020. "An extensive analysis of the presence of altmetric data for Web of Science publications across subject fields and research topics," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2519-2549, September.
    18. Truyken L. B. Ossenblok & Tim C. E. Engels & Gunnar Sivertsen, 2012. "The representation of the social sciences and humanities in the Web of Science--a comparison of publication patterns and incentive structures in Flanders and Norway (2005--9)," Research Evaluation, Oxford University Press, vol. 21(4), pages 280-290, September.
    19. Ehsan Mohammadi & Mike Thelwall, 2014. "Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(8), pages 1627-1638, August.
    20. Hans Jonker & Florian Vanlee & Walter Ysebaert, 2022. "Societal impact of university research in the written press: media attention in the context of SIUR and the open science agenda among social scientists in Flanders, Belgium," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7289-7306, December.
    21. Richard Van Noorden, 2014. "Online collaboration: Scientists and the social network," Nature, Nature, vol. 512(7513), pages 126-129, August.
    22. Rodrigo Costas & Zohreh Zahedi & Paul Wouters, 2015. "Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(10), pages 2003-2019, October.
    23. Andrea Bonaccorsi & Cinzia Daraio & Stefano Fantoni & Viola Folli & Marco Leonetti & Giancarlo Ruocco, 2017. "Do social sciences and humanities behave like life and hard sciences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 607-653, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Siluo & Zheng, Mengxue & Yu, Yonghao & Wolfram, Dietmar, 2021. "Are Altmetric.com scores effective for research impact evaluation in the social sciences and humanities?," Journal of Informetrics, Elsevier, vol. 15(1).
    2. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    3. Ying Guo & Xiantao Xiao, 2022. "Author-level altmetrics for the evaluation of Chinese scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 973-990, February.
    4. Sergio Copiello, 2019. "Research Interest: another undisclosed (and redundant) algorithm by ResearchGate," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 351-360, July.
    5. Maryam Moshtagh & Tahereh Jowkar & Maryam Yaghtin & Hajar Sotudeh, 2023. "The moderating effect of altmetrics on the correlations between single and multi-faceted university ranking systems: the case of THE and QS vs. Nature Index and Leiden," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 761-781, January.
    6. Núria Bautista-Puig & Daniela De Filippo & Elba Mauleón & Elías Sanz-Casado, 2019. "Scientific Landscape of Citizen Science Publications: Dynamics, Content and Presence in Social Media," Publications, MDPI, vol. 7(1), pages 1-22, February.
    7. Łukasz Wiechetek & Zbigniew Pastuszak, 2022. "Academic social networks metrics: an effective indicator for university performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(3), pages 1381-1401, March.
    8. Daniela Filippo & Pablo Sastrón-Toledo, 2023. "Influence of research on open science in the public policy sphere," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1995-2017, March.
    9. Jianhua Hou & Da Ma, 2020. "How the high-impact papers formed? A study using data from social media and citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2597-2615, December.
    10. Siluo Yang & Xin Xing & Fan Qi & Maria Cláudia Cabrini Grácio, 2021. "Comparison of academic book impact from a disciplinary perspective: an analysis of citations and altmetric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1101-1123, February.
    11. Martín-Martín, Alberto & Orduna-Malea, Enrique & Delgado López-Cózar, Emilio, 2018. "Author-level metrics in the new academic profile platforms: The online behaviour of the Bibliometrics community," Journal of Informetrics, Elsevier, vol. 12(2), pages 494-509.
    12. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    13. Sergio Copiello & Pietro Bonifaci, 2019. "ResearchGate Score, full-text research items, and full-text reads: a follow-up study," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1255-1262, May.
    14. Kaltrina Nuredini, 2021. "Investigating Altmetric Information For The Top 1000 Journals From Handelsblatt Ranking In Economic And Business Studies," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1315-1343, December.
    15. Latefa Ali Dardas & Malik Sallam & Amanda Woodward & Nadia Sweis & Narjes Sweis & Faleh A. Sawair, 2023. "Evaluating Research Impact Based on Semantic Scholar Highly Influential Citations, Total Citations, and Altmetric Attention Scores: The Quest for Refined Measures Remains Illusive," Publications, MDPI, vol. 11(1), pages 1-16, January.
    16. Amalia Mas-Bleda & Mike Thelwall, 2016. "Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2007-2030, December.
    17. Sergio Copiello & Pietro Bonifaci, 2018. "A few remarks on ResearchGate score and academic reputation," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 301-306, January.
    18. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    19. Zhou, Qingqing & Zhang, Chengzhi, 2021. "Breaking community boundary: Comparing academic and social communication preferences regarding global pandemics," Journal of Informetrics, Elsevier, vol. 15(3).
    20. González-Betancor, Sara M. & Dorta-González, Pablo, 2023. "Does society show differential attention to researchers based on gender and field?," Journal of Informetrics, Elsevier, vol. 17(4).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jpubli:v:11:y:2023:i:2:p:31-:d:1161937. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.