IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v94y2013i3d10.1007_s11192-012-0777-7.html
   My bibliography  Save this article

A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners

Author

Listed:
  • Anne-Wil Harzing

    (University of Melbourne)

Abstract

Most governmental research assessment exercises do not use citation data for the Social Sciences and Humanities as Web of Science or Scopus coverage in these disciplines is considered to be insufficient. We therefore assess to what extent Google Scholar can be used as an alternative source of citation data. In order to provide a credible alternative, Google Scholar needs to be stable over time, display comprehensive coverage, and provide non-biased comparisons across disciplines. This article assesses these conditions through a longitudinal study of 20 Nobel Prize winners in Chemistry, Economics, Medicine and Physics. Our results indicate that Google Scholar displays considerable stability over time. However, coverage for disciplines that have traditionally been poorly represented in Google Scholar (Chemistry and Physics) is increasing rapidly. Google Scholar’s coverage is also comprehensive; all of the 800 most cited publications by our Nobelists can be located in Google Scholar, although in four cases there are some problems with the results. Finally, we argue that Google Scholar might provide a less biased comparison across disciplines than the Web of Science. The use of Google Scholar might therefore redress the traditionally disadvantaged position of the Social Sciences in citation analysis.

Suggested Citation

  • Anne-Wil Harzing, 2013. "A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 1057-1075, March.
  • Handle: RePEc:spr:scient:v:94:y:2013:i:3:d:10.1007_s11192-012-0777-7
    DOI: 10.1007/s11192-012-0777-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-012-0777-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-012-0777-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bornmann, Lutz & Marx, Werner & Schier, Hermann & Rahm, Erhard & Thor, Andreas & Daniel, Hans-Dieter, 2009. "Convergent validity of bibliometric Google Scholar data in the field of chemistry—Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published els," Journal of Informetrics, Elsevier, vol. 3(1), pages 27-35.
    2. Judit Bar-Ilan, 2010. "Citations to the “Introduction to informetrics” indexed by WOS, Scopus and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 495-506, March.
    3. John Mingers & Evangelia A. E. C. G. Lipitakis, 2010. "Counting the citations: a comparison of Web of Science and Google Scholar in the field of business and management," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(2), pages 613-625, November.
    4. Norris, Michael & Oppenheim, Charles, 2007. "Comparing alternatives to the Web of Science for coverage of the social sciences’ literature," Journal of Informetrics, Elsevier, vol. 1(2), pages 161-169.
    5. Anton J. Nederhof, 2006. "Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 81-100, January.
    6. Mu‐hsuan Huang & Yu‐wei Chang, 2008. "Characteristics of research output in social sciences and humanities: From a research evaluation perspective," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1819-1828, September.
    7. Massimo Franceschet, 2010. "A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 243-258, April.
    8. Judit Bar-Ilan, 2008. "Which h-index? — A comparison of WoS, Scopus and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 257-271, February.
    9. Bar-Ilan, Judit & Levene, Mark & Lin, Ayelet, 2007. "Some measures for comparing citation databases," Journal of Informetrics, Elsevier, vol. 1(1), pages 26-34.
    10. Kayvan Kousha & Mike Thelwall & Somayeh Rezaie, 2011. "Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(11), pages 2147-2164, November.
    11. Clare V. Thornley & Andrea C. Johnson & Alan F. Smeaton & Hyowon Lee, 2011. "The scholarly impact of TRECVid (2003–2009)," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(4), pages 613-627, April.
    12. Lutz Bornmann & Hans-Dieter Daniel, 2005. "Does the h-index for ranking of scientists really work?," Scientometrics, Springer;Akadémiai Kiadó, vol. 65(3), pages 391-392, December.
    13. Kayvan Kousha & Mike Thelwall & Somayeh Rezaie, 2011. "Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(11), pages 2147-2164, November.
    14. Kayvan Kousha & Mike Thelwall, 2008. "Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 273-294, February.
    15. Anne-Wil Harzing, 2005. "Australian Research Output in Economics and Business: High Volume, Low Impact?," Australian Journal of Management, Australian School of Business, vol. 30(2), pages 183-200, December.
    16. Liwen Vaughan & Debora Shaw, 2008. "A new look at evidence of scholarly citation in citation indexes and from web sources," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 317-330, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Halevi, Gali & Moed, Henk & Bar-Ilan, Judit, 2017. "Suitability of Google Scholar as a source of scientific information and as a source of data for scientific evaluation—Review of the Literature," Journal of Informetrics, Elsevier, vol. 11(3), pages 823-834.
    2. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    3. Martin-Martin, Alberto & Orduna-Malea, Enrique & Harzing, Anne-Wil & Delgado López-Cózar, Emilio, 2017. "Can we use Google Scholar to identify highly-cited documents?," Journal of Informetrics, Elsevier, vol. 11(1), pages 152-163.
    4. Joost C. F. Winter & Amir A. Zadpoor & Dimitra Dodou, 2014. "The expansion of Google Scholar versus Web of Science: a longitudinal study," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1547-1565, February.
    5. Massimo Franceschet, 2010. "A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 243-258, April.
    6. J. W. Fedderke, 2013. "The objectivity of national research foundation peer review in South Africa assessed against bibliometric indexes," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 177-206, November.
    7. Alberto Martín-Martín & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2018. "Coverage of highly-cited documents in Google Scholar, Web of Science, and Scopus: a multidisciplinary comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2175-2188, September.
    8. Kousha, Kayvan & Thelwall, Mike, 2018. "Can Microsoft Academic help to assess the citation impact of academic books?," Journal of Informetrics, Elsevier, vol. 12(3), pages 972-984.
    9. Kousha, Kayvan & Thelwall, Mike & Rezaie, Somayeh, 2010. "Using the Web for research evaluation: The Integrated Online Impact indicator," Journal of Informetrics, Elsevier, vol. 4(1), pages 124-135.
    10. Daniel Torres-Salinas & Nicolás Robinson-Garcia & Juan Gorraiz, 2017. "Filling the citation gap: measuring the multidimensional impact of the academic book at institutional level with PlumX," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1371-1384, December.
    11. Boyack, Kevin W. & Klavans, Richard, 2014. "Including cited non-source items in a large-scale map of science: What difference does it make?," Journal of Informetrics, Elsevier, vol. 8(3), pages 569-580.
    12. Antonio Cavacini, 2015. "What is the best database for computer science journal articles?," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2059-2071, March.
    13. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    14. Elizabeth S. Vieira & José A. N. F. Gomes, 2009. "A comparison of Scopus and Web of Science for a typical university," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(2), pages 587-600, November.
    15. Yongjun Zhu & Erjia Yan & Silvio Peroni & Chao Che, 2020. "Nine million book items and eleven million citations: a study of book-based scholarly communication using OpenCitations," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1097-1112, February.
    16. Müller, Harry & Dilger, Alexander, 2013. "Der Einfluss des Forschungsschwerpunkts auf den Zitationserfolg: Eine empirische Untersuchung anhand der Gesamtpublikationen deutschsprachiger Hochschullehrer für BWL," Discussion Papers of the Institute for Organisational Economics 1/2013, University of Münster, Institute for Organisational Economics.
    17. Fedderke, J.W. & Goldschmidt, M., 2015. "Does massive funding support of researchers work?: Evaluating the impact of the South African research chair funding initiative," Research Policy, Elsevier, vol. 44(2), pages 467-482.
    18. Moed, Henk F. & Bar-Ilan, Judit & Halevi, Gali, 2016. "A new methodology for comparing Google Scholar and Scopus," Journal of Informetrics, Elsevier, vol. 10(2), pages 533-551.
    19. Teja Koler-Povh & Primož Južnič & Goran Turk, 2014. "Impact of open access on citation of scholarly publications in the field of civil engineering," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1033-1045, February.
    20. Mike Thelwall, 2017. "Judit Bar-Ilan: information scientist, computer scientist, scientometrician," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1235-1244, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:94:y:2013:i:3:d:10.1007_s11192-012-0777-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.