IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v6y2012i4p566-579.html
   My bibliography  Save this article

Aggregating different paper quality measures with a generalized h-index

Author

Listed:
  • Gagolewski, Marek
  • Mesiar, Radko

Abstract

The process of assessing individual authors should rely upon a proper aggregation of reliable and valid papers’ quality metrics. Citations are merely one possible way to measure appreciation of publications. In this study we propose some new, SJR- and SNIP-based indicators, which not only take into account the broadly conceived popularity of a paper (manifested by the number of citations), but also other factors like its potential, or the quality of papers that cite a given publication. We explore the relation and correlation between different metrics and study how they affect the values of a real-valued generalized h-index calculated for 11 prominent scientometricians. We note that the h-index is a very unstable impact function, highly sensitive for applying input elements’ scaling. Our analysis is not only of theoretical significance: data scaling is often performed to normalize citations across disciplines. Uncontrolled application of this operation may lead to unfair and biased (toward some groups) decisions. This puts the validity of authors assessment and ranking using the h-index into question. Obviously, a good impact function to be used in practice should not be as much sensitive to changing input data as the analyzed one.

Suggested Citation

  • Gagolewski, Marek & Mesiar, Radko, 2012. "Aggregating different paper quality measures with a generalized h-index," Journal of Informetrics, Elsevier, vol. 6(4), pages 566-579.
    • Handle: RePEc:eee:infome:v:6:y:2012:i:4:p:566-579
    DOI: 10.1016/j.joi.2012.05.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157712000417
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2012.05.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Igor Podlubny, 2005. "Comparison of scientific impact expressed by the number of citations in different fields of science," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(1), pages 95-99, July.
    2. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    3. Woeginger, Gerhard J., 2008. "An axiomatic analysis of Egghe’s g-index," Journal of Informetrics, Elsevier, vol. 2(4), pages 364-368.
    4. Quesada, Antonio, 2009. "Monotonicity and the Hirsch index," Journal of Informetrics, Elsevier, vol. 3(2), pages 158-160.
    5. Loet Leydesdorff & Tobias Opthof, 2010. "Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(11), pages 2365-2369, November.
    6. Guns, Raf & Rousseau, Ronald, 2009. "Real and rational variants of the h-index and the g-index," Journal of Informetrics, Elsevier, vol. 3(1), pages 64-71.
    7. Marek Gągolewski & Przemysław Grzegorzewski, 2009. "A geometric approach to the construction of scientific impact indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 617-634, December.
    8. van Eck, Nees Jan & Waltman, Ludo, 2008. "Generalizing the h- and g-indices," Journal of Informetrics, Elsevier, vol. 2(4), pages 263-271.
    9. Woeginger, Gerhard J., 2008. "An axiomatic characterization of the Hirsch-index," Mathematical Social Sciences, Elsevier, vol. 56(2), pages 224-232, September.
    10. Lokman I. Meho & Cassidy R. Sugimoto, 2009. "Assessing the scholarly impact of information studies: A tale of two citation databases—Scopus and Web of Science," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(12), pages 2499-2508, December.
    11. Moed, Henk F., 2010. "Measuring contextual citation impact of scientific journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 265-277.
    12. Michel Grabisch & Jean-Luc Marichal & Radko Mesiar & Endre Pap, 2009. "Aggregation functions," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-00445120, HAL.
    13. Nair, Gopalan M. & Turlach, Berwin A., 2012. "The stochastic h-index," Journal of Informetrics, Elsevier, vol. 6(1), pages 80-87.
    14. González-Pereira, Borja & Guerrero-Bote, Vicente P. & Moya-Anegón, Félix, 2010. "A new approach to the metric of journals’ scientific prestige: The SJR indicator," Journal of Informetrics, Elsevier, vol. 4(3), pages 379-391.
    15. van Eck, N.J.P. & Waltman, L., 2008. "Generalizing the h- and g-indices," ERIM Report Series Research in Management ERS-2008-049-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    16. Antonio Quesada, 2010. "More axiomatics for the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 413-418, February.
    17. Woeginger, Gerhard J., 2008. "A symmetry axiom for scientific impact indices," Journal of Informetrics, Elsevier, vol. 2(4), pages 298-303.
    18. Gagolewski, Marek, 2011. "Bibliometric impact assessment with R and the CITAN package," Journal of Informetrics, Elsevier, vol. 5(4), pages 678-692.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cena, Anna & Gagolewski, Marek & Mesiar, Radko, 2015. "Problems and challenges of information resources producers’ clustering," Journal of Informetrics, Elsevier, vol. 9(2), pages 273-284.
    2. Gagolewski, Marek, 2013. "Scientific impact assessment cannot be fair," Journal of Informetrics, Elsevier, vol. 7(4), pages 792-802.
    3. Boczek, Michał & Hovana, Anton & Hutník, Ondrej & Kaluszka, Marek, 2021. "New monotone measure-based integrals inspired by scientific impact problem," European Journal of Operational Research, Elsevier, vol. 290(1), pages 346-357.
    4. Dašić Predrag, 2015. "State and Analysis of Scientific Journals in the Field of “Economic Sciences” for the Period 1995-2014," Economic Themes, Sciendo, vol. 53(4), pages 547-581, December.
    5. Fabio Zagonari, 2019. "Scientific Production and Productivity for Characterizing an Author’s Publication History: Simple and Nested Gini’s and Hirsch’s Indexes Combined," Publications, MDPI, vol. 7(2), pages 1-30, May.
    6. James C. Ryan, 2016. "A validation of the individual annual h-index (hIa): application of the hIa to a qualitatively and quantitatively different sample," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(1), pages 577-590, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gagolewski, Marek, 2011. "Bibliometric impact assessment with R and the CITAN package," Journal of Informetrics, Elsevier, vol. 5(4), pages 678-692.
    2. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    3. Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
    4. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    5. Stefano Vannucci, 2010. "Dominance dimension: a common parametric formulation for integer-valued scientific impact indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 43-48, July.
    6. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    7. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    8. Gagolewski, Marek, 2013. "Scientific impact assessment cannot be fair," Journal of Informetrics, Elsevier, vol. 7(4), pages 792-802.
    9. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    10. Antonio Quesada, 2011. "Further characterizations of the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 107-114, April.
    11. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    12. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    13. Quesada, Antonio, 2011. "Axiomatics for the Hirsch index and the Egghe index," Journal of Informetrics, Elsevier, vol. 5(3), pages 476-480.
    14. Adachi, Tsuyoshi & Kongo, Takumi, 2015. "Further axiomatizations of Egghe's g-index," Journal of Informetrics, Elsevier, vol. 9(4), pages 839-844.
    15. Franceschini, Fiorenzo & Maisano, Domenico A., 2010. "Analysis of the Hirsch index's operational properties," European Journal of Operational Research, Elsevier, vol. 203(2), pages 494-504, June.
    16. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    17. Kongo, Takumi, 2014. "An alternative axiomatization of the Hirsch index," Journal of Informetrics, Elsevier, vol. 8(1), pages 252-258.
    18. Ash Mohammad Abbas, 2011. "Weighted indices for evaluating the quality of research with multiple authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 107-131, July.
    19. Serge Galam, 2011. "Tailor based allocations for multiple authorship: a fractional gh-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 365-379, October.
    20. repec:cte:werepe:11672 is not listed on IDEAS
    21. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:6:y:2012:i:4:p:566-579. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.