IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i7d10.1007_s11192-024-05091-3.html
   My bibliography  Save this article

Policy citations of scientometric articles: an altmetric study

Author

Listed:
  • Hashem Atapour

    (University of Tabriz)

  • Robabeh Maddahi

    (University of Tabriz)

  • Rasoul Zavaraqi

    (University of Tabriz)

Abstract

Policy citations are considered as one of the important indicators of the societal impact of research. Scientometrics is a field that, among other goals, focus on contributing to science policy, so the presence of scientometric researches in policy documents become important. Accordingly, this study aims to measure the impact of scientometric researches on policy by examining the mentions of scientometric articles in policy documents. The dataset used in this study includes 5525 scientometric articles indexed in Web of Science between 2013 and 2022. The Overton database were used to collect policy citations. The results showed that out of 5525 scientometric articles, 921 articles (16.67%) were cited at least once in policy documents. Additionally, older articles were cited more frequently than recent ones in policy documents. Scientometrics Journal ranked first in terms of the number of articles being cited in policy documents, while Research Policy and Research Evaluation Journals ranked first and second, respectively, in terms of coverage, density, and intensity. Subject analysis of the cited articles in policy documents showed that articles on national/international scholar collaborations, scholar productivity/scholar performance, and funding and sponsorship were cited more frequently in policy documents. Finally, Open Access articles were cited more frequently than non-open access articles in policy documents. However, there was not significant difference between policy citations of multi-authored and sing-authored articles. Overall, policy citations of scientometric articles were fair compared to other fields, and for greater impact of this field on policy, publishing open access, and greater attention to the topics identified in this study can be helpful.

Suggested Citation

  • Hashem Atapour & Robabeh Maddahi & Rasoul Zavaraqi, 2024. "Policy citations of scientometric articles: an altmetric study," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4423-4436, July.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:7:d:10.1007_s11192-024-05091-3
    DOI: 10.1007/s11192-024-05091-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05091-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-024-05091-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Paul Nightingale & Alister Scott, 2007. "Peer review and the relevance gap: Ten suggestions for policy-makers," Science and Public Policy, Oxford University Press, vol. 34(8), pages 543-553, October.
    2. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    3. Joly, P.B. & Gaunand, A. & Colinet, L. & Larédo, P. & Lemarié, S. & Matt, M., 2015. "ASIRPA: a comprehensive theory-based approach to assessing the societal impacts of a research organization," Working Papers 2015-04, Grenoble Applied Economics Laboratory (GAEL).
    4. Qianjin Zong & Zhihong Huang & Jiaru Huang, 2023. "Can open access increase LIS research’s policy impact? Using regression analysis and causal inference," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4825-4854, August.
    5. O. Mryglod & S. Nazarovets & S. Kozmenko, 2021. "Universal and specific features of Ukrainian economic research: publication analysis based on Crossref data," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 8187-8203, September.
    6. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    7. Zhihong Huang & Qianjin Zong & Xuerui Ji, 2022. "The associations between scientific collaborations of LIS research and its policy impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6453-6470, November.
    8. William W. Hood & Concepción S. Wilson, 2001. "The Literature of Bibliometrics, Scientometrics, and Informetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 52(2), pages 291-314, October.
    9. Isidro F. Aguillo, 2020. "Altmetrics of the Open Access Institutional Repositories: a webometrics approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(3), pages 1181-1192, June.
    10. Michael Taylor, 2020. "An altmetric attention advantage for open access books in the humanities and social sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2523-2543, December.
    11. Dorte Drongstrup & Shafaq Malik & Naif Radi Aljohani & Salem Alelyani & Iqra Safder & Saeed-Ul Hassan, 2020. "Can social media usage of scientific literature predict journal indices of AJG, SNIP and JCR? An altmetric study of economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1541-1558, November.
    12. Robin Haunschild & Lutz Bornmann, 2017. "How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1209-1216, March.
    13. Ã ine Regan & Maeve Henchion, 2019. "Making sense of altmetrics: The perceived threats and opportunities for academic identity," Science and Public Policy, Oxford University Press, vol. 46(4), pages 479-489.
    14. Bornmann, Lutz, 2014. "Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime," Journal of Informetrics, Elsevier, vol. 8(4), pages 935-950.
    15. Hans Pohl, 2021. "Internationalisation, innovation, and academic–corporate co-publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1329-1358, February.
    16. Lutz Bornmann & Robin Haunschild & Werner Marx, 2016. "Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1477-1495, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhihong Huang & Qianjin Zong & Xuerui Ji, 2022. "The associations between scientific collaborations of LIS research and its policy impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6453-6470, November.
    2. Ying Guo & Xiantao Xiao, 2022. "Author-level altmetrics for the evaluation of Chinese scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 973-990, February.
    3. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    4. Bornmann, Lutz & Haunschild, Robin, 2018. "Normalization of zero-inflated data: An empirical analysis of a new indicator family and its use with altmetrics data," Journal of Informetrics, Elsevier, vol. 12(3), pages 998-1011.
    5. Liwei Zhang & Liang Ma, 2024. "Different open access routes, varying societal impacts: evidence from the Royal Society biological journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(6), pages 3407-3431, June.
    6. Caitlin Drummond Otten & Baruch Fischhoff, 2022. "Assessing broader impacts of funded research: the US National Science Foundation v. Lamar Smith [What is Societal Impact of Research and How Can it Be Assessed? A Literature Survey]," Science and Public Policy, Oxford University Press, vol. 49(2), pages 313-323.
    7. Andrea Bonaccorsi & Filippo Chiarello & Gualtiero Fantoni, 2021. "Impact for whom? Mapping the users of public research with lexicon-based text mining," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1745-1774, February.
    8. Qianjin Zong & Zhihong Huang & Jiaru Huang, 2023. "Can open access increase LIS research’s policy impact? Using regression analysis and causal inference," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4825-4854, August.
    9. Yaxue Ma & Zhichao Ba & Yuxiang Zhao & Jin Mao & Gang Li, 2021. "Understanding and predicting the dissemination of scientific papers on social media: a two-step simultaneous equation modeling–artificial neural network approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 7051-7085, August.
    10. Nathalie Taverdet-Popiolek, 2022. "Economic Footprint of a Large French Research and Technology Organisation in Europe: Deciphering a Simplified Model and Appraising the Results," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(1), pages 44-69, March.
    11. Lutz Bornmann, 2020. "Bibliometrics-based decision tree (BBDT) for deciding whether two universities in the Leiden ranking differ substantially in their performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1255-1258, February.
    12. Ruth Zárate-Rueda & Yolima Ivonne Beltrán-Villamizar & Daniella Murallas-Sánchez, 2021. "Social representations of socioenvironmental dynamics in extractive ecosystems and conservation practices with sustainable development: a bibliometric analysis," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 23(11), pages 16428-16453, November.
    13. Marco Gatti, 2018. "The Impact of Management Accounting Research: An Analysis of the Past and a Look at the Future," International Journal of Business and Management, Canadian Center of Science and Education, vol. 13(5), pages 1-47, March.
    14. José Antonio Moreira de Rezende & Reginaldo Gonçalves Leão Junior & Otávio de Souza Martins Gomes, 2024. "Scientometric Analysis of Publications on Household Electricity Theft and Energy Consumption Load Profiling in a Smart Grid Context," Sustainability, MDPI, vol. 16(22), pages 1-19, November.
    15. Lutz Bornmann & Robin Haunschild & Werner Marx, 2016. "Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1477-1495, December.
    16. Houqiang Yu & Xueting Cao & Tingting Xiao & Zhenyi Yang, 2020. "How accurate are policy document mentions? A first look at the role of altmetrics database," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1517-1540, November.
    17. Knut Blind & Alex Fenton, 2022. "Standard-relevant publications: evidence, processes and influencing factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 577-602, January.
    18. Muhammad N. Mahmood & Subas P. Dhakal, 2023. "Ageing population and society: a scientometric analysis," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3133-3150, August.
    19. D’Este, Pablo & Robinson-García, Nicolás, 2023. "Interdisciplinary research and the societal visibility of science: The advantages of spanning multiple and distant scientific fields," Research Policy, Elsevier, vol. 52(2).
    20. Oscar LLOPIS & Joaquin AZAGRA-CARO, 2015. "Who do you care about? Scientists’ personality traits and perceived beneficiary impact," Cahiers du GREThA (2007-2019) 2015-29, Groupe de Recherche en Economie Théorique et Appliquée (GREThA).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:7:d:10.1007_s11192-024-05091-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.