IDEAS home Printed from https://ideas.repec.org/a/wly/mgtdec/v42y2021i8p1933-1944.html
   My bibliography  Save this article

The unbearable lightness of scientometric indices

Author

Listed:
  • Matteo Migheli
  • Giovanni B. Ramello

Abstract

This paper advances the debate on scholarly publishing and the role of bibliometric indices in evaluating authors and their research, through a theoretical discussion and an empirical case study focused on economics and the impact factor. The rationale of the current bibliometric system is that reputation, assessed by citation figures, can be converted into an objective measure. We instead argue that it provides questionable results, because it fails to properly consider the meaning of indicators built for different purposes, as well as the psychological bias generated by the wrong interpretation of those indicators. However, the potential for abating these distortions exists.

Suggested Citation

  • Matteo Migheli & Giovanni B. Ramello, 2021. "The unbearable lightness of scientometric indices," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 42(8), pages 1933-1944, December.
    • Handle: RePEc:wly:mgtdec:v:42:y:2021:i:8:p:1933-1944
    DOI: 10.1002/mde.3486
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/mde.3486
    Download Restriction: no
    File URL: https://libkey.io/10.1002/mde.3486?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Hendrik P. Dalen, 2021. "How the publish-or-perish principle divides a science: the case of economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1675-1694, February.
    2. Thomas Eger & Marc Scheufen, 2018. "The Economics of Open Access," Books, Edward Elgar Publishing, number 17006.
    3. Manuel Trajtenberg, 1990. "A Penny for Your Quotes: Patent Citations and the Value of Innovations," RAND Journal of Economics, The RAND Corporation, vol. 21(1), pages 172-187, Spring.
    4. James J. Heckman & Sidharth Moktan, 2020. "Publishing and promotion in economics - The tyranny of the Top Five," Vox eBook Chapters, in: Sebastian Galliani & Ugo Panizza (ed.), Publishing and Measuring Success in Economics, edition 1, volume 1, chapter 1, pages 23-32, Centre for Economic Policy Research.
    5. Bronwyn H. Hall & Adam Jaffe & Manuel Trajtenberg, 2005. "Market Value and Patent Citations," RAND Journal of Economics, The RAND Corporation, vol. 36(1), pages 16-38, Spring.
    6. Migheli, Matteo & Zotti, Roberto, 2020. "The strange case of the Matthew effect and beauty contests: Research evaluation and specialisation in Italian universities," Socio-Economic Planning Sciences, Elsevier, vol. 71(C).
    7. repec:fth:harver:1473 is not listed on IDEAS
    8. Lynne M. Pepall, 2002. "The Simple Economics of Brand Stretching," The Journal of Business, University of Chicago Press, vol. 75(3), pages 535-552, July.
    9. Alain Marciano & Antonio Nicita & Giovanni Battista Ramello, 2020. "Big data and big techs: understanding the value of information in platform capitalism," European Journal of Law and Economics, Springer, vol. 50(3), pages 345-358, December.
    10. Matteo Migheli & Giovanni Battista Ramello, 2018. "The market of academic attention," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 113-133, January.
    11. Zvi Griliches, 1998. "Patent Statistics as Economic Indicators: A Survey," NBER Chapters, in: R&D and Productivity: The Econometric Evidence, pages 287-343, National Bureau of Economic Research, Inc.
    12. Matteo Migheli & Giovanni B. Ramello, 2013. "Open Access, Social Norms & Publication Choice," ICER Working Papers 03-2013, ICER - International Centre for Economic Research.
    13. Mario Biagioli, 2016. "Watch out for cheats in citation game," Nature, Nature, vol. 535(7611), pages 201-201, July.
    14. Sandra Rousseau & Ronald Rousseau, 2021. "Bibliometric Techniques And Their Use In Business And Economics Research," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1428-1451, December.
    15. Johan Bollen & Herbert Van de Sompel & Aric Hagberg & Ryan Chute, 2009. "A Principal Component Analysis of 39 Scientific Impact Measures," PLOS ONE, Public Library of Science, vol. 4(6), pages 1-11, June.
    16. Matteo Migheli & Giovanni Ramello, 2013. "Open access, social norms and publication choice," European Journal of Law and Economics, Springer, vol. 35(2), pages 149-167, April.
    17. van Dalen, Hendrik Peter, 2021. "How the publish-or-perish principle divides a science: The case of economists," Other publications TiSEM a6a5a855-bb5a-4d52-a841-3, Tilburg University, School of Economics and Management.
    18. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    19. Dietmar Harhoff & Francis Narin & F. M. Scherer & Katrin Vopel, 1999. "Citation Frequency And The Value Of Patented Inventions," The Review of Economics and Statistics, MIT Press, vol. 81(3), pages 511-515, August.
    20. Osterloh, Margit & Frey, Bruno S., 2020. "How to avoid borrowed plumes in academia," Research Policy, Elsevier, vol. 49(1).
    21. David A. Pendlebury & Jonathan Adams, 2012. "Comments on a critique of the Thomson Reuters journal impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 395-401, August.
    22. Roberto Ippoliti & Giovanni B. Ramello & Frederic M. Scherer, 2021. "Partnership and innovation in the pharmaceutical industry: the case of clinical research," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 30(3), pages 317-334, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Prem Kumar Singh, 2022. "t-index: entropy based random document and citation analysis using average h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 637-660, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guan-Can Yang & Gang Li & Chun-Ya Li & Yun-Hua Zhao & Jing Zhang & Tong Liu & Dar-Zen Chen & Mu-Hsuan Huang, 2015. "Using the comprehensive patent citation network (CPC) to evaluate patent value," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1319-1346, December.
    2. Emanuele Bacchiocchi & Fabio Montobbio, 2010. "International Knowledge Diffusion and Home‐bias Effect: Do USPTO and EPO Patent Citations Tell the Same Story?," Scandinavian Journal of Economics, Wiley Blackwell, vol. 112(3), pages 441-470, September.
    3. Hans Georg Helmstädter & Anna Gehlke & Lukasz Hill & Bernd Klöver & Laura Wallor & Christoph Badelt & Matthias Firgo & Oliver Fritz & Kathrin Hofmann & Mark Horridge & Jürgen Janger & Peter Mayerhofer, 2020. "Perspektiven der wissenschaftlichen Metropolregion Hamburg – Eine vergleichende Analyse. Anhänge," WIFO Studies, WIFO, number 62513.
    4. Yi Deng, 2005. "The Value of Knowledge Flows: Evidence from Patent Citations Data," Computing in Economics and Finance 2005 374, Society for Computational Economics.
    5. Altuntas, Serkan & Dereli, Turkay & Kusiak, Andrew, 2015. "Analysis of patent documents with weighted association rules," Technological Forecasting and Social Change, Elsevier, vol. 92(C), pages 249-262.
    6. Wang, Xun, 2022. "Capital account liberalization, financial dependence and technological innovation: Cross-country evidence," Journal of Banking & Finance, Elsevier, vol. 145(C).
    7. Toivanen, Otto & Väänänen, Lotta, 2010. "Returns to Inventors," CEPR Discussion Papers 7744, C.E.P.R. Discussion Papers.
    8. Gerald A. Carlino & Robert M. Hunt, 2009. "What explains the quantity and quality of local inventive activity?," Working Papers 09-12, Federal Reserve Bank of Philadelphia.
    9. Altwies, Joy E. & Nemet, Gregory F., 2013. "Innovation in the U.S. building sector: An assessment of patent citations in building energy control technology," Energy Policy, Elsevier, vol. 52(C), pages 819-831.
    10. Jaimin Goh & Jaehong Lee & Wonchang Hur & Yunchang Ju, 2019. "Do Analysts Fully Reflect Information in Patents about Future Earnings?," Sustainability, MDPI, vol. 11(10), pages 1-17, May.
    11. Pierre-Alexandre Balland & David L. Rigby, 2015. "The geography and evolution of complex knowledge," Papers in Evolutionary Economic Geography (PEEG) 1502, Utrecht University, Department of Human Geography and Spatial Planning, Group Economic Geography, revised Jan 2015.
    12. Luisito Bertinelli & Arnaud Bourgain & Florian Léon, 2020. "Corruption and tax compliance: evidence from small retailers in Bamako, Mali," Applied Economics Letters, Taylor & Francis Journals, vol. 27(5), pages 366-370, March.
    13. Justus Haucap & Nima Moshgbar & W. Benedikt Schmal, 2021. "The impact of the German 'DEAL' on competition in the academic publishing market," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 42(8), pages 2027-2049, December.
    14. Roger Svensson, 2013. "Publicly-funded R&D programs and survival of patents," Applied Economics, Taylor & Francis Journals, vol. 45(10), pages 1343-1358, April.
    15. Antonio Messeni Petruzzelli & Daniele Rotolo & Vito Albino, 2014. "Determinants of Patent Citations in Biotechnology: An Analysis of Patent Influence Across the Industrial and Organizational Boundaries," SPRU Working Paper Series 2014-05, SPRU - Science Policy Research Unit, University of Sussex Business School.
    16. Pfister, Curdin & Koomen, Miriam & Harhoff, Dietmar & Backes-Gellner, Uschi, 2021. "Regional innovation effects of applied research institutions," Research Policy, Elsevier, vol. 50(4).
    17. Carlo Giglio & Roberto Sbragia & Roberto Musmanno & Roberto Palmieri, 2021. "Cross-country learning from patents: an analysis of citations flows in innovation trajectories," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7917-7936, September.
    18. Hussinger, Katrin & Pacher, Sebastian, 2019. "Information ambiguity, patents and the market value of innovative assets," Research Policy, Elsevier, vol. 48(3), pages 665-675.
    19. Adam B. Jaffe & Gaétan de Rassenfosse, 2017. "Patent citation data in social science research: Overview and best practices," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(6), pages 1360-1374, June.
    20. Yi Deng, 2005. "The value of knowledge spillovers," Working Paper Series 2005-14, Federal Reserve Bank of San Francisco.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:mgtdec:v:42:y:2021:i:8:p:1933-1944. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/7976 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.