IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v1y2007i3p185-192.html
   My bibliography  Save this article

Scoring research output using statistical quantile plotting

Author

Listed:
  • Beirlant, Jan
  • Glänzel, Wolfgang
  • Carbonez, An
  • Leemans, Herlinde

Abstract

In this paper, we propose two methods for scoring scientific output based on statistical quantile plotting. First, a rescaling of journal impact factors for scoring scientific output on a macro level is proposed. It is based on normal quantile plotting which allows to transform impact data over several subject categories to a standardized distribution. This can be used in comparing scientific output of larger entities such as departments working in quite different areas of research. Next, as an alternative to the Hirsch index [Hirsch, J.E. (2005). An index to quantify an individuals scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572], the extreme value index is proposed as an indicator for assessment of the research performance of individual scientists. In case of Lotkaian–Zipf–Pareto behaviour of citation counts of an individual, the extreme value index can be interpreted as the slope in a Pareto–Zipf quantile plot. This index, in contrast to the Hirsch index, is not influenced by the number of publications but stresses the decay of the statistical tail of citation counts. It appears to be much less sensitive to the science field than the Hirsch index.

Suggested Citation

  • Beirlant, Jan & Glänzel, Wolfgang & Carbonez, An & Leemans, Herlinde, 2007. "Scoring research output using statistical quantile plotting," Journal of Informetrics, Elsevier, vol. 1(3), pages 185-192.
    • Handle: RePEc:eee:infome:v:1:y:2007:i:3:p:185-192
    DOI: 10.1016/j.joi.2007.04.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157707000454
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2007.04.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Igor Podlubny, 2005. "Comparison of scientific impact expressed by the number of citations in different fields of science," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(1), pages 95-99, July.
    2. Wolfgang Glänzel & Henk F. Moed, 2002. "Journal impact measures in bibliometric research," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 171-193, February.
    3. Leo Egghe & Ronald Rousseau, 2006. "An informetric model for the Hirsch-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 121-129, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Waltman, L. & van Eck, N.J.P., 2009. "Some Comments on Egghe’s Derivation of the Impact Factor Distribution," ERIM Report Series Research in Management ERS-2009-016-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    2. Stegehuis, Clara & Litvak, Nelly & Waltman, Ludo, 2015. "Predicting the long-term citation impact of recent publications," Journal of Informetrics, Elsevier, vol. 9(3), pages 642-657.
    3. Wolfgang Glänzel & Bart Thijs & Koenraad Debackere, 2014. "The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 939-952, November.
    4. Waltman, Ludo & van Eck, Nees Jan, 2009. "Some comments on Egghe's derivation of the impact factor distribution," Journal of Informetrics, Elsevier, vol. 3(4), pages 363-366.
    5. J. Martin Zyl, 2013. "The generalized Pareto distribution fitted to research outputs of countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 1099-1109, March.
    6. Wolfgang Glänzel, 2013. "High-end performance or outlier? Evaluating the tail of scientometric distributions," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(1), pages 13-23, October.
    7. Loet Leydesdorff & Ping Zhou & Lutz Bornmann, 2013. "How can journal impact factors be normalized across fields of science? An assessment in terms of percentile ranks and fractional counts," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 96-107, January.
    8. Tang, Li, 2013. "Does “birds of a feather flock together” matter—Evidence from a longitudinal study on US–China scientific collaboration," Journal of Informetrics, Elsevier, vol. 7(2), pages 330-344.
    9. Richard S.J. Tol, 2013. "Measuring catch-up growth in malnourished populations," Working Paper Series 6013, Department of Economics, University of Sussex Business School.
    10. Tol, Richard S.J., 2013. "Identifying excellent researchers: A new approach," Journal of Informetrics, Elsevier, vol. 7(4), pages 803-810.
    11. Mutz, Rüdiger & Daniel, Hans-Dieter, 2012. "Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor," Journal of Informetrics, Elsevier, vol. 6(2), pages 169-176.
    12. Wolfgang Glänzel & Henk F. Moed, 2013. "Opinion paper: thoughts and facts on bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 381-394, July.
    13. Wolfgang Glänzel & András Schubert & Bart Thijs & Koenraad Debackere, 2011. "A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 415-424, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    2. Bárbara S. Lancho-Barrantes & Vicente P. Guerrero-Bote & Félix Moya-Anegón, 2010. "The iceberg hypothesis revisited," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(2), pages 443-461, November.
    3. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    4. John Panaretos & Chrisovaladis Malesios, 2009. "Assessing scientific research performance and impact with single indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 635-670, December.
    5. Bertoli-Barsotti, Lucio & Lando, Tommaso, 2019. "How mean rank and mean size may determine the generalised Lorenz curve: With application to citation analysis," Journal of Informetrics, Elsevier, vol. 13(1), pages 387-396.
    6. Jiancheng Guan & Yan Yan & Jingjing Zhang, 2015. "How do collaborative features affect scientific output? Evidences from wind power field," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 333-355, January.
    7. Juan E. Iglesias & Carlos Pecharromán, 2007. "Scaling the h-index for different scientific ISI fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 73(3), pages 303-320, December.
    8. Jonathan Adams & Karen Gurney & Louise Jackson, 2008. "Calibrating the zoom — a test of Zitt’s hypothesis," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(1), pages 81-95, April.
    9. Kwame Boamah‐Addo & Tomasz J. Kozubowski & Anna K. Panorska, 2023. "A discrete truncated Zipf distribution," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 77(2), pages 156-187, May.
    10. Fiorenzo Franceschini & Maurizio Galetto & Domenico Maisano & Luca Mastrogiacomo, 2012. "The success-index: an alternative approach to the h-index for evaluating an individual’s research output," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 621-641, September.
    11. Jo Royle & Louisa Coles & Dorothy Williams & Paul Evans, 2007. "Publishing in international journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(1), pages 59-86, April.
    12. Lathabai, Hiran H., 2020. "ψ-index: A new overall productivity index for actors of science and technology," Journal of Informetrics, Elsevier, vol. 14(4).
    13. Jerome K. Vanclay, 2012. "Impact factor: outdated artefact or stepping-stone to journal certification?," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 211-238, August.
    14. Rey-Martí, Andrea & Ribeiro-Soriano, Domingo & Palacios-Marqués, Daniel, 2016. "A bibliometric analysis of social entrepreneurship," Journal of Business Research, Elsevier, vol. 69(5), pages 1651-1655.
    15. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Costa, 2023. "Correlating article citedness and journal impact: an empirical investigation by field on a large-scale dataset," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1877-1894, March.
    16. Artemis Chaleplioglou, 2024. "Papers in and Papers out of the Spotlight: Comparative Bibliometric and Altmetrics Analysis of Biomedical Reports with and without News Media Stories," Publications, MDPI, vol. 12(4), pages 1-14, September.
    17. Eduardo A. Haddad & Jesus P. Mena-Chalco, Otávio J.G. Sidone, 2016. "Produção Científica e Redes de Colaboração dos Docentes Vinculados aos Programas de Pós-graduação em Economia no Brasil," Working Papers, Department of Economics 2016_10, University of São Paulo (FEA-USP).
    18. Mutz, Rüdiger & Daniel, Hans-Dieter, 2012. "Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor," Journal of Informetrics, Elsevier, vol. 6(2), pages 169-176.
    19. Wolfgang Glänzel & Henk F. Moed, 2013. "Opinion paper: thoughts and facts on bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 381-394, July.
    20. Cristiano Varin & Manuela Cattelan & David Firth, 2016. "Statistical modelling of citation exchange between statistics journals," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(1), pages 1-63, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:1:y:2007:i:3:p:185-192. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.