IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v8y2014i3p738-748.html
   My bibliography  Save this article

Examples for counterintuitive behavior of the new citation-rank indicator P100 for bibliometric evaluations

Author

Listed:
  • Schreiber, Michael

Abstract

A new percentile-based rating scale P100 has recently been proposed to describe the citation impact in terms of the distribution of the unique citation values. Here I investigate P100 for 5 example datasets, two simple fictitious models and three larger empirical samples. Counterintuitive behavior is demonstrated in the model datasets, pointing to difficulties when the evolution with time of the indicator is analyzed or when different fields or publication years are compared. It is shown that similar problems can occur for the three larger datasets of empirical citation values. Further, it is observed that the performance evaluation result in terms of percentiles can be influenced by selecting different journals for publication of a manuscript.

Suggested Citation

  • Schreiber, Michael, 2014. "Examples for counterintuitive behavior of the new citation-rank indicator P100 for bibliometric evaluations," Journal of Informetrics, Elsevier, vol. 8(3), pages 738-748.
    • Handle: RePEc:eee:infome:v:8:y:2014:i:3:p:738-748
    DOI: 10.1016/j.joi.2014.06.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157714000601
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2014.06.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2013. "Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P1," Journal of Informetrics, Elsevier, vol. 7(4), pages 933-944.
    2. Lutz Bornmann & Rüdiger Mutz, 2014. "From P100 to P100': A new citation-rank approach," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(9), pages 1939-1943, September.
    3. Bornmann, Lutz & Leydesdorff, Loet & Mutz, Rüdiger, 2013. "The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits," Journal of Informetrics, Elsevier, vol. 7(1), pages 158-165.
    4. Michael Schreiber, 2013. "Uncertainties and ambiguities in percentiles and how to avoid them," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(3), pages 640-643, March.
    5. Michael Schreiber, 2013. "How much do different ways of calculating percentiles influence the derived performance indicators? A case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 821-829, December.
    6. Michael Schreiber, 2013. "Empirical evidence for the relevance of fractional scoring in the calculation of percentile rank scores," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(4), pages 861-867, April.
    7. Michael Schreiber, 2013. "Uncertainties and ambiguities in percentiles and how to avoid them," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(3), pages 640-643, March.
    8. Michael Schreiber, 2013. "Empirical evidence for the relevance of fractional scoring in the calculation of percentile rank scores," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(4), pages 861-867, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Schreiber, Michael, 2014. "How to improve the outcome of performance evaluations in terms of percentiles for citation frequencies of my papers," Journal of Informetrics, Elsevier, vol. 8(4), pages 873-879.
    2. Schreiber, Michael, 2014. "Is the new citation-rank approach P100′ in bibliometrics really new?," Journal of Informetrics, Elsevier, vol. 8(4), pages 997-1004.
    3. Fabio Zagonari, 2019. "Scientific Production and Productivity for Characterizing an Author’s Publication History: Simple and Nested Gini’s and Hirsch’s Indexes Combined," Publications, MDPI, vol. 7(2), pages 1-30, May.
    4. Bornmann, Lutz & Marx, Werner, 2015. "Methods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgements of experts?," Journal of Informetrics, Elsevier, vol. 9(2), pages 408-418.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Schreiber, Michael, 2014. "How to improve the outcome of performance evaluations in terms of percentiles for citation frequencies of my papers," Journal of Informetrics, Elsevier, vol. 8(4), pages 873-879.
    2. Schreiber, Michael, 2014. "Is the new citation-rank approach P100′ in bibliometrics really new?," Journal of Informetrics, Elsevier, vol. 8(4), pages 997-1004.
    3. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
    4. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2013. "Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P1," Journal of Informetrics, Elsevier, vol. 7(4), pages 933-944.
    5. Lutz Bornmann & Werner Marx & Andreas Barth, 2013. "The Normalization of Citation Counts Based on Classification Systems," Publications, MDPI, vol. 1(2), pages 1-9, August.
    6. Bornmann, Lutz & Marx, Werner, 2015. "Methods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgements of experts?," Journal of Informetrics, Elsevier, vol. 9(2), pages 408-418.
    7. Lutz Bornmann & Richard Williams, 2020. "An evaluation of percentile measures of citation impact, and a proposal for making them better," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1457-1478, August.
    8. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2014. "How to improve the prediction based on citation impact percentiles for years shortly after the publication date?," Journal of Informetrics, Elsevier, vol. 8(1), pages 175-180.
    9. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    10. Bornmann, Lutz & Haunschild, Robin, 2016. "Citation score normalized by cited references (CSNCR): The introduction of a new citation impact indicator," Journal of Informetrics, Elsevier, vol. 10(3), pages 875-887.
    11. Gerson Pech & Catarina Delgado, 2020. "Assessing the publication impact using citation data from both Scopus and WoS databases: an approach validated in 15 research fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 909-924, November.
    12. Thelwall, Mike, 2016. "The precision of the arithmetic mean, geometric mean and percentiles for citation data: An experimental simulation modelling approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 110-123.
    13. Chen, Shiji & Arsenault, Clément & Larivière, Vincent, 2015. "Are top-cited papers more interdisciplinary?," Journal of Informetrics, Elsevier, vol. 9(4), pages 1034-1046.
    14. Bornmann, Lutz & Stefaner, Moritz & de Moya Anegón, Felix & Mutz, Rüdiger, 2014. "What is the effect of country-specific characteristics on the research performance of scientific institutions? Using multi-level statistical models to rank and map universities and research-focused in," Journal of Informetrics, Elsevier, vol. 8(3), pages 581-593.
    15. Ashraf Uddin & Jaideep Bhoosreddy & Marisha Tiwari & Vivek Kumar Singh, 2016. "A Sciento-text framework to characterize research strength of institutions at fine-grained thematic area level," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1135-1150, March.
    16. Vinkler, Péter, 2014. "The use of the Percentage Rank Position index for comparative evaluation of journals," Journal of Informetrics, Elsevier, vol. 8(2), pages 340-348.
    17. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    18. Thor, Andreas & Marx, Werner & Leydesdorff, Loet & Bornmann, Lutz, 2016. "Introducing CitedReferencesExplorer (CRExplorer): A program for reference publication year spectroscopy with cited references standardization," Journal of Informetrics, Elsevier, vol. 10(2), pages 503-515.
    19. Rodríguez-Navarro, Alonso & Brito, Ricardo, 2018. "Technological research in the EU is less efficient than the world average. EU research policy risks Europeans’ future," Journal of Informetrics, Elsevier, vol. 12(3), pages 718-731.
    20. Cinzia Daraio & Simone Di Leo & Loet Leydesdorff, 2022. "Using the Leiden Rankings as a Heuristics: Evidence from Italian universities in the European landscape," LEM Papers Series 2022/08, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:8:y:2014:i:3:p:738-748. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.