IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v98y2014i2d10.1007_s11192-013-1068-7.html
   My bibliography  Save this article

Substance without citation: evaluating the online impact of grey literature

Author

Listed:
  • David Wilkinson

    (School of Technology, University of Wolverhampton)

  • Pardeep Sud

    (School of Technology, University of Wolverhampton)

  • Mike Thelwall

    (School of Technology, University of Wolverhampton)

Abstract

Individuals and organisations producing information or knowledge for others sometimes need to be able to provide evidence of the value of their work in the same way that scientists may use journal impact factors and citations to indicate the value of their papers. There are many cases, however, when organisations are charged with producing reports but have no real way of measuring their impact, including when they are distributed free, do not attract academic citations and their sales cannot be tracked. Here, the web impact report (WIRe) is proposed as a novel solution for this problem. A WIRe consists of a range of web-derived statistics about the frequency and geographic location of online mentions of an organisation’s reports. WIRe data is typically derived from commercial search engines. This article defines the component parts of a WIRe and describes how to collect and analyse the necessary data. The process is illustrated with a comparison of the web impact of the reports of a large UK organisation. Although a formal evaluation was not conducted, the results suggest that WIRes can indicate different levels of web impact between reports and can reveal the type of online impact that the reports have.

Suggested Citation

  • David Wilkinson & Pardeep Sud & Mike Thelwall, 2014. "Substance without citation: evaluating the online impact of grey literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 797-806, February.
    • Handle: RePEc:spr:scient:v:98:y:2014:i:2:d:10.1007_s11192-013-1068-7
    DOI: 10.1007/s11192-013-1068-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-013-1068-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-013-1068-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Blaise Cronin & Herbert W. Snyder & Howard Rosenbaum & Anna Martinson & Ewa Callahan, 1998. "Invoked on the Web," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 49(14), pages 1319-1328.
    2. Butler, Linda, 2003. "Explaining Australia's increased share of ISI publications--the effects of a funding formula based on publication counts," Research Policy, Elsevier, vol. 32(1), pages 143-155, January.
    3. Mike Thelwall, 2008. "Extracting accurate and complete results from search engines: Case study windows live," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(1), pages 38-50, January.
    4. Anne Gentil-Beccot & Salvatore Mele & Travis C. Brooks, 2010. "Citing and reading behaviours in high-energy physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(2), pages 345-355, August.
    5. Liwen Vaughan & Debora Shaw, 2003. "Bibliographic and Web citations: What is the difference?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 54(14), pages 1313-1322, December.
    6. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    7. Blaise Cronin & Debora Shaw, 2002. "Banking (on) different forms of symbolic capital," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 53(14), pages 1267-1270, December.
    8. Mike Thelwall & Katie Vann & Ruth Fairclough, 2006. "Web issue analysis: An integrated water resource management case study," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(10), pages 1303-1314, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mike Thelwall, 2021. "Measuring Societal Impacts Of Research With Altmetrics? Common Problems And Mistakes," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1302-1314, December.
    2. Matthew S. Bickley & Kayvan Kousha & Michael Thelwall, 2020. "Can the impact of grey literature be assessed? An investigation of UK government publications cited by articles and books," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1425-1444, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    2. Fairclough, Ruth & Thelwall, Mike, 2015. "National research impact indicators from Mendeley readers," Journal of Informetrics, Elsevier, vol. 9(4), pages 845-859.
    3. Pardeep Sud & Mike Thelwall, 2014. "Linked title mentions: a new automated link search candidate," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1831-1849, December.
    4. Gobinda Chowdhury & Kushwanth Koya & Pete Philipson, 2016. "Measuring the Impact of Research: Lessons from the UK’s Research Excellence Framework 2014," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-15, June.
    5. Xuemei Li & Mike Thelwall & Dean Giustini, 2012. "Validating online reference managers for scholarly impact measurement," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 461-471, May.
    6. Jonathan P. Doh & Lorraine Eden & Anne S. Tsui & Srilata Zaheer, 2023. "Developing international business scholarship for global societal impact," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 54(5), pages 757-767, July.
    7. Dangzhi Zhao & Elisabeth Logan, 2002. "Citation analysis using scientific publications on the Web as data source: A case study in the XML research area," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(3), pages 449-472, July.
    8. Alexander Kalgin & Olga Kalgina & Anna Lebedeva, 2019. "Publication Metrics as a Tool for Measuring Research Productivity and Their Relation to Motivation," Voprosy obrazovaniya / Educational Studies Moscow, National Research University Higher School of Economics, issue 1, pages 44-86.
    9. Thelwall, Mike & Sud, Pardeep, 2012. "Webometric research with the Bing Search API 2.0," Journal of Informetrics, Elsevier, vol. 6(1), pages 44-52.
    10. Jianhua Hou, 2017. "Exploration into the evolution and historical roots of citation analysis by referenced publication year spectroscopy," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1437-1452, March.
    11. Jianhua Hou & Xiucai Yang & Yang Zhang, 2023. "The effect of social media knowledge cascade: an analysis of scientific papers diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5169-5195, September.
    12. Fernandez Martinez, Roberto & Lostado Lorza, Ruben & Santos Delgado, Ana Alexandra & Piedra, Nelson, 2021. "Use of classification trees and rule-based models to optimize the funding assignment to research projects: A case study of UTPL," Journal of Informetrics, Elsevier, vol. 15(1).
    13. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    14. Isidro F. Aguillo, 2020. "Altmetrics of the Open Access Institutional Repositories: a webometrics approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(3), pages 1181-1192, June.
    15. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 9447, Victoria University of Wellington, Chair in Public Finance.
    16. Nathalie Taverdet-Popiolek, 2022. "Economic Footprint of a Large French Research and Technology Organisation in Europe: Deciphering a Simplified Model and Appraising the Results," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(1), pages 44-69, March.
    17. Pablo D’Este & Irene Ramos-Vielba & Richard Woolley & Nabil Amara, 2018. "How do researchers generate scientific and societal impacts? Toward an analytical and operational framework," Science and Public Policy, Oxford University Press, vol. 45(6), pages 752-763.
    18. Juha-Pekka Lauronen, 2022. "Tension in Interpretations of the Social Impact of the Social Sciences: Walking a Tightrope Between Divergent Conceptualizations of Research Utilization," SAGE Open, , vol. 12(2), pages 21582440221, April.
    19. Qingjun Zhao & Jiancheng Guan, 2012. "Modeling the dynamic relation between science and technology in nanotechnology," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 561-579, February.
    20. repec:dau:papers:123456789/8741 is not listed on IDEAS
    21. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "National-scale research performance assessment at the individual level," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(2), pages 347-364, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:98:y:2014:i:2:d:10.1007_s11192-013-1068-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.