IDEAS home Printed from https://ideas.repec.org/a/wly/mgtdec/v42y2021i8p1933-1944.html
   My bibliography  Save this article

The unbearable lightness of scientometric indices

Author

Listed:
  • Matteo Migheli
  • Giovanni B. Ramello

Abstract

This paper advances the debate on scholarly publishing and the role of bibliometric indices in evaluating authors and their research, through a theoretical discussion and an empirical case study focused on economics and the impact factor. The rationale of the current bibliometric system is that reputation, assessed by citation figures, can be converted into an objective measure. We instead argue that it provides questionable results, because it fails to properly consider the meaning of indicators built for different purposes, as well as the psychological bias generated by the wrong interpretation of those indicators. However, the potential for abating these distortions exists.

Suggested Citation

  • Matteo Migheli & Giovanni B. Ramello, 2021. "The unbearable lightness of scientometric indices," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 42(8), pages 1933-1944, December.
    • Handle: RePEc:wly:mgtdec:v:42:y:2021:i:8:p:1933-1944
    DOI: 10.1002/mde.3486
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/mde.3486
    Download Restriction: no
    File URL: https://libkey.io/10.1002/mde.3486?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Hendrik P. Dalen, 2021. "How the publish-or-perish principle divides a science: the case of economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1675-1694, February.
    2. Manuel Trajtenberg, 1990. "A Penny for Your Quotes: Patent Citations and the Value of Innovations," RAND Journal of Economics, The RAND Corporation, vol. 21(1), pages 172-187, Spring.
    3. James J. Heckman & Sidharth Moktan, 2020. "Publishing and promotion in economics - The tyranny of the Top Five," Vox eBook Chapters, in: Sebastian Galliani & Ugo Panizza (ed.), Publishing and Measuring Success in Economics, edition 1, volume 1, chapter 1, pages 23-32, Centre for Economic Policy Research.
    4. Bronwyn H. Hall & Adam Jaffe & Manuel Trajtenberg, 2005. "Market Value and Patent Citations," RAND Journal of Economics, The RAND Corporation, vol. 36(1), pages 16-38, Spring.
    5. repec:fth:harver:1473 is not listed on IDEAS
    6. Lynne M. Pepall, 2002. "The Simple Economics of Brand Stretching," The Journal of Business, University of Chicago Press, vol. 75(3), pages 535-552, July.
    7. Alain Marciano & Antonio Nicita & Giovanni Battista Ramello, 2020. "Big data and big techs: understanding the value of information in platform capitalism," European Journal of Law and Economics, Springer, vol. 50(3), pages 345-358, December.
    8. Mario Biagioli, 2016. "Watch out for cheats in citation game," Nature, Nature, vol. 535(7611), pages 201-201, July.
    9. Johan Bollen & Herbert Van de Sompel & Aric Hagberg & Ryan Chute, 2009. "A Principal Component Analysis of 39 Scientific Impact Measures," PLOS ONE, Public Library of Science, vol. 4(6), pages 1-11, June.
    10. Matteo Migheli & Giovanni Ramello, 2013. "Open access, social norms and publication choice," European Journal of Law and Economics, Springer, vol. 35(2), pages 149-167, April.
    11. van Dalen, Hendrik Peter, 2021. "How the publish-or-perish principle divides a science: The case of economists," Other publications TiSEM a6a5a855-bb5a-4d52-a841-3, Tilburg University, School of Economics and Management.
    12. Dietmar Harhoff & Francis Narin & F. M. Scherer & Katrin Vopel, 1999. "Citation Frequency And The Value Of Patented Inventions," The Review of Economics and Statistics, MIT Press, vol. 81(3), pages 511-515, August.
    13. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    14. Migheli Matteo & Ramello, Giovanni B., 2013. "Open Access, Social Norms & Publication Choice," IEL Working Papers 13, Institute of Public Policy and Public Choice - POLIS.
    15. David A. Pendlebury & Jonathan Adams, 2012. "Comments on a critique of the Thomson Reuters journal impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 395-401, August.
    16. Roberto Ippoliti & Giovanni B. Ramello & Frederic M. Scherer, 2021. "Partnership and innovation in the pharmaceutical industry: the case of clinical research," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 30(3), pages 317-334, April.
    17. Thomas Eger & Marc Scheufen, 2018. "The Economics of Open Access," Books, Edward Elgar Publishing, number 17006.
    18. Migheli, Matteo & Zotti, Roberto, 2020. "The strange case of the Matthew effect and beauty contests: Research evaluation and specialisation in Italian universities," Socio-Economic Planning Sciences, Elsevier, vol. 71(C).
    19. Matteo Migheli & Giovanni Battista Ramello, 2018. "The market of academic attention," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 113-133, January.
    20. Zvi Griliches, 1998. "Patent Statistics as Economic Indicators: A Survey," NBER Chapters, in: R&D and Productivity: The Econometric Evidence, pages 287-343, National Bureau of Economic Research, Inc.
    21. Sandra Rousseau & Ronald Rousseau, 2021. "Bibliometric Techniques And Their Use In Business And Economics Research," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1428-1451, December.
    22. Osterloh, Margit & Frey, Bruno S., 2020. "How to avoid borrowed plumes in academia," Research Policy, Elsevier, vol. 49(1).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Prem Kumar Singh, 2022. "t-index: entropy based random document and citation analysis using average h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 637-660, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guan-Can Yang & Gang Li & Chun-Ya Li & Yun-Hua Zhao & Jing Zhang & Tong Liu & Dar-Zen Chen & Mu-Hsuan Huang, 2015. "Using the comprehensive patent citation network (CPC) to evaluate patent value," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1319-1346, December.
    2. Emanuele Bacchiocchi & Fabio Montobbio, 2010. "International Knowledge Diffusion and Home‐bias Effect: Do USPTO and EPO Patent Citations Tell the Same Story?," Scandinavian Journal of Economics, Wiley Blackwell, vol. 112(3), pages 441-470, September.
    3. Altuntas, Serkan & Dereli, Turkay & Kusiak, Andrew, 2015. "Analysis of patent documents with weighted association rules," Technological Forecasting and Social Change, Elsevier, vol. 92(C), pages 249-262.
    4. Wang, Xun, 2022. "Capital account liberalization, financial dependence and technological innovation: Cross-country evidence," Journal of Banking & Finance, Elsevier, vol. 145(C).
    5. Toivanen, Otto & Väänänen, Lotta, 2010. "Returns to Inventors," CEPR Discussion Papers 7744, C.E.P.R. Discussion Papers.
    6. Altwies, Joy E. & Nemet, Gregory F., 2013. "Innovation in the U.S. building sector: An assessment of patent citations in building energy control technology," Energy Policy, Elsevier, vol. 52(C), pages 819-831.
    7. Jaimin Goh & Jaehong Lee & Wonchang Hur & Yunchang Ju, 2019. "Do Analysts Fully Reflect Information in Patents about Future Earnings?," Sustainability, MDPI, vol. 11(10), pages 1-17, May.
    8. Luisito Bertinelli & Arnaud Bourgain & Florian Léon, 2020. "Corruption and tax compliance: evidence from small retailers in Bamako, Mali," Applied Economics Letters, Taylor & Francis Journals, vol. 27(5), pages 366-370, March.
    9. Justus Haucap & Nima Moshgbar & W. Benedikt Schmal, 2021. "The impact of the German 'DEAL' on competition in the academic publishing market," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 42(8), pages 2027-2049, December.
    10. Pfister, Curdin & Koomen, Miriam & Harhoff, Dietmar & Backes-Gellner, Uschi, 2021. "Regional innovation effects of applied research institutions," Research Policy, Elsevier, vol. 50(4).
    11. Carlo Giglio & Roberto Sbragia & Roberto Musmanno & Roberto Palmieri, 2021. "Cross-country learning from patents: an analysis of citations flows in innovation trajectories," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7917-7936, September.
    12. Adam B. Jaffe & Gaétan de Rassenfosse, 2017. "Patent citation data in social science research: Overview and best practices," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(6), pages 1360-1374, June.
    13. Scott D. Bass & Lukasz A. Kurgan, 2010. "Discovery of factors influencing patent value based on machine learning in patents in the field of nanotechnology," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 217-241, February.
    14. Rong, Zhao & Wu, Xiaokai & Boeing, Philipp, 2017. "The effect of institutional ownership on firm innovation: Evidence from Chinese listed firms," Research Policy, Elsevier, vol. 46(9), pages 1533-1551.
    15. N. N., 2020. "WIFO-Monatsberichte, Heft 9/2020," WIFO Monatsberichte (monthly reports), WIFO, vol. 93(9), September.
    16. Albino, Vito & Ardito, Lorenzo & Dangelico, Rosa Maria & Messeni Petruzzelli, Antonio, 2014. "Understanding the development trends of low-carbon energy technologies: A patent analysis," Applied Energy, Elsevier, vol. 135(C), pages 836-854.
    17. Michele Cincera & Ela Ince, 2019. "Types of Innovation and Firm performance," Working Papers TIMES² 2019-032, ULB -- Universite Libre de Bruxelles.
    18. Paola Giuri & Myriam Mariani & Stefano Brusoni & Gustavo Crespi & Dominique Francoz & Alfonso Gambardella & Walter Garcia-Fontes & Aldo Geuna & Raul Gonzales & Dietmar Harhoff & Karin Hoisl & Christia, 2005. "Everything you Always Wanted to Know about Inventors (but Never Asked): Evidence from the PatVal-EU Survey," LEM Papers Series 2005/20, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    19. Antoine Dechezleprêtre & Yann Ménière & Myra Mohnen, 2017. "International patent families: from application strategies to statistical indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(2), pages 793-828, May.
    20. Petra Moser & Joerg Ohmstedt & Paul W. Rhode, 2015. "Patent Citations and the Size of the Inventive Step - Evidence from Hybrid Corn," NBER Working Papers 21443, National Bureau of Economic Research, Inc.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:mgtdec:v:42:y:2021:i:8:p:1933-1944. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/7976 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.