Research assessment by percentile-based double rank analysis
Author
Abstract
Suggested Citation
DOI: 10.1016/j.joi.2018.01.011
Download full text from publisher
As the access to this document is restricted, you may want to search for a different version of it.
References listed on IDEAS
- Michal Brzezinski, 2015.
"Power laws in citation distributions: evidence from Scopus,"
Scientometrics, Springer;Akadémiai Kiadó, vol. 103(1), pages 213-228, April.
- Michał Brzeziński, 2014. "Power laws in citation distributions: Evidence from Scopus," Working Papers 2014-05, Faculty of Economic Sciences, University of Warsaw.
- T. S. Evans & N. Hopkins & B. S. Kaube, 2012. "Universality of performance indicators based on citation and reference counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 473-495, November.
- Thelwall, Mike & Wilson, Paul, 2014. "Regression for citation data: An evaluation of different methods," Journal of Informetrics, Elsevier, vol. 8(4), pages 963-971.
- Jonathan Adams & Karen Gurney & Stuart Marshall, 2007. "Profiling citation impact: A new methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(2), pages 325-344, August.
- Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2016. "A farewell to the MNCS and like size-independent indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 646-651.
- Pedro Albarrán & Juan A. Crespo & Ignacio Ortuño & Javier Ruiz-Castillo, 2011.
"The skewness of science in 219 sub-fields and a number of aggregates,"
Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 385-397, August.
- Crespo, Juan A. & Albarrán, Pedro & Ortuño, Ignacio, 2010. "The skewness of science in 219 sub-fields and a number of aggregates," UC3M Working papers. Economics we1038, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Albarrán, Pedro & Crespo, Juan A. & Ortuño, Ignacio, 2011. "The skewness of science in 219 sub-fields and a number of aggregates," UC3M Working papers. Economics we1109, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Ruiz-Castillo, Javier & Ortuño-Ortin, Ignacio & Albarran, Pedro & Crespo, Juan A., 2010. "The Skewness of Science in 219 Sub-Fields and a Number of Aggregates," CEPR Discussion Papers 8126, C.E.P.R. Discussion Papers.
- J Sylvan Katz, 2016. "What Is a Complex Innovation System?," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-24, June.
- Michael Schreiber, 2013. "How much do different ways of calculating percentiles influence the derived performance indicators? A case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 821-829, December.
- Loet Leydesdorff & Lutz Bornmann & Rüdiger Mutz & Tobias Opthof, 2011.
"Turning the tables on citation analysis one more time: Principles for comparing sets of documents,"
Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(7), pages 1370-1381, July.
- Loet Leydesdorff & Lutz Bornmann & Rüdiger Mutz & Tobias Opthof, 2011. "Turning the tables on citation analysis one more time: Principles for comparing sets of documents," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(7), pages 1370-1381, July.
- Rodríguez-Navarro, Alonso & Brito, Ricardo, 2018. "Double rank analysis for research assessment," Journal of Informetrics, Elsevier, vol. 12(1), pages 31-41.
- Albarrán, Pedro & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2011.
"High- and low-impact citation measures: Empirical applications,"
Journal of Informetrics, Elsevier, vol. 5(1), pages 122-145.
- Albarrán, Pedro & Ortuño, Ignacio, 2010. "High - and low-impact citation measures: empirical applications," UC3M Working papers. Economics we10_09, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Ruiz-Castillo, Javier & Ortuño-Ortin, Ignacio & Albarran, Pedro, 2010. "High- and Low-Impact Citation Measures: Empirical Applications," CEPR Discussion Papers 7886, C.E.P.R. Discussion Papers.
- Lutz Bornmann, 2014. "How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature," Research Evaluation, Oxford University Press, vol. 23(2), pages 166-173.
- Ruiz-Castillo, Javier & Waltman, Ludo, 2015.
"Field-normalized citation impact indicators using algorithmically constructed classification systems of science,"
Journal of Informetrics, Elsevier, vol. 9(1), pages 102-117.
- Waltman, Ludo, 2014. "Field-normalized citation impact indicators using algorithmically constructed classification systems of science," UC3M Working papers. Economics we1403, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2013. "Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P1," Journal of Informetrics, Elsevier, vol. 7(4), pages 933-944.
- Salter, Ammon J. & Martin, Ben R., 2001. "The economic benefits of publicly funded basic research: a critical review," Research Policy, Elsevier, vol. 30(3), pages 509-532, March.
- Lutz Bornmann & Rüdiger Mutz, 2014. "From P100 to P100': A new citation-rank approach," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(9), pages 1939-1943, September.
- Jesper W. Schneider & Rodrigo Costas, 2017. "Identifying potential “breakthrough” publications using refined citation analyses: Three related explorative approaches," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(3), pages 709-723, March.
- Ronald Rousseau, 2012. "Basic properties of both percentile rank scores and the I3 indicator," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(2), pages 416-420, February.
- Loet Leydesdorff & Lutz Bornmann, 2011.
"Integrated impact indicators compared with impact factors: An alternative research design with policy implications,"
Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(11), pages 2133-2146, November.
- Loet Leydesdorff & Lutz Bornmann, 2011. "Integrated impact indicators compared with impact factors: An alternative research design with policy implications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(11), pages 2133-2146, November.
- Schreiber, Michael, 2013. "A case study of the arbitrariness of the h-index and the highly-cited-publications indicator," Journal of Informetrics, Elsevier, vol. 7(2), pages 379-387.
- Ronald Rousseau, 2012. "Basic properties of both percentile rank scores and the I3 indicator," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(2), pages 416-420, February.
- Martin, Ben R. & Irvine, John, 1993.
"Assessing basic research : Some partial indicators of scientific progress in radio astronomy,"
Research Policy, Elsevier, vol. 22(2), pages 106-106, April.
- Martin, Ben R. & Irvine, John, 1983. "Assessing basic research : Some partial indicators of scientific progress in radio astronomy," Research Policy, Elsevier, vol. 12(2), pages 61-90, April.
- Pedro Albarrán & Ignacio Ortuño & Javier Ruiz-Castillo, 2011.
"Average-based versus high- and low-impact indicators for the evaluation of scientific distributions,"
Research Evaluation, Oxford University Press, vol. 20(4), pages 325-339, October.
- Albarrán, Pedro & Ortuño, Ignacio, 2010. "Average-based versus high-and low-impact indicators for the evaluation of scientific distributions," UC3M Working papers. Economics we1040, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Ruiz-Castillo, Javier & Ortuño-Ortin, Ignacio & Albarran, Pedro, 2010. "Average-based versus High- and Low-Impact Indicators for the Evaluation of Scientific Distributions," CEPR Discussion Papers 7887, C.E.P.R. Discussion Papers.
- Lutz Bornmann, 2013.
"How to analyze percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes, and top-cited papers,"
Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(3), pages 587-595, March.
- Lutz Bornmann, 2013. "How to analyze percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes, and top‐cited papers," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(3), pages 587-595, March.
- Ludo Waltman & Michael Schreiber, 2013. "On the calculation of percentile-based bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 372-379, February.
- Rinia, E. J. & van Leeuwen, Th. N. & van Vuren, H. G. & van Raan, A. F. J., 1998. "Comparative analysis of a set of bibliometric indicators and central peer review criteria: Evaluation of condensed matter physics in the Netherlands," Research Policy, Elsevier, vol. 27(1), pages 95-107, May.
- Michael J. Stringer & Marta Sales-Pardo & Luís A. Nunes Amaral, 2010. "Statistical validation of a global model for the distribution of the ultimate number of citations accrued by papers published in a scientific journal," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1377-1385, July.
- Bornmann, Lutz & Haunschild, Robin, 2016. "Citation score normalized by cited references (CSNCR): The introduction of a new citation impact indicator," Journal of Informetrics, Elsevier, vol. 10(3), pages 875-887.
- Andrea Bonaccorsi, 2007. "Explaining poor performance of European science: Institutions versus policies," Science and Public Policy, Oxford University Press, vol. 34(5), pages 303-316, June.
- Lutz Bornmann & Caroline Wagner & Loet Leydesdorff, 2015. "BRICS countries and scientific excellence: A bibliometric analysis of most frequently cited papers," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(7), pages 1507-1513, July.
- Albarrán, Pedro & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2011.
"The measurement of low- and high-impact in citation distributions: Technical results,"
Journal of Informetrics, Elsevier, vol. 5(1), pages 48-63.
- Albarrán, Pedro & Ortuño, Ignacio, 2009. "The measurement of low- and high-impact in citation distributions : technical results," UC3M Working papers. Economics we095735, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Li, Yunrong & Radicchi, Filippo & Castellano, Claudio & Ruiz-Castillo, Javier, 2013.
"Quantitative evaluation of alternative field normalization procedures,"
Journal of Informetrics, Elsevier, vol. 7(3), pages 746-755.
- Radicchi, Filippo & Castellano, Claudio, 2013. "Quantitative evaluation of alternative field normalization procedures," UC3M Working papers. Economics we1305, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Robert J. W. Tijssen & Martijn S. Visser & Thed N. van Leeuwen, 2002. "Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference?," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(3), pages 381-397, July.
- Bornmann, Lutz & Leydesdorff, Loet & Mutz, Rüdiger, 2013. "The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits," Journal of Informetrics, Elsevier, vol. 7(1), pages 158-165.
- Giovanni Abramo & Ciriaco Andrea D’Angelo, 2014. "How do you define and measure research productivity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1129-1144, November.
- Javier Ruiz-Castillo, 2012.
"The evaluation of citation distributions,"
SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 3(1), pages 291-310, March.
- Ruiz-Castillo, Javier, 2011. "The Evaluation of Citation Distributions," CEPR Discussion Papers 8681, C.E.P.R. Discussion Papers.
- Pedro Albarrán & Antonio Perianes-Rodríguez & Javier Ruiz-Castillo, 2015.
"Differences in citation impact across countries,"
Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(3), pages 512-525, March.
- Albarrán, Pedro & Perianes-Rodríguez, Antonio, 2013. "Differences in citation impact across countries," UC3M Working papers. Economics we1229, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
- Wolfgang Glänzel & Bart Thijs & Koenraad Debackere, 2014. "The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 939-952, November.
- Waltman, Ludo & van Eck, Nees Jan & Wouters, Paul, 2013. "Counting publications and citations: Is more always better?," Journal of Informetrics, Elsevier, vol. 7(3), pages 635-641.
- Stevan Harnad, 2009. "Open access scientometrics and the UK Research Assessment Exercise," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(1), pages 147-156, April.
- Dag W Aksnes & Randi Elisabeth Taxt, 2004. "Peer reviews and bibliometric indicators: a comparative study at a Norwegian university," Research Evaluation, Oxford University Press, vol. 13(1), pages 33-41, April.
- Anthony F. J. van Raan, 2005. "Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 133-143, January.
- Dag W. Aksnes & Gunnar Sivertsen, 2004. "The effect of highly cited papers on national citation indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 59(2), pages 213-224, February.
- Rüdiger Mutz & Hans-Dieter Daniel, 2015. "What is behind the curtain of the Leiden Ranking?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(9), pages 1950-1953, September.
- Thelwall, Mike & Wilson, Paul, 2014. "Distributions for cited articles from individual subjects and years," Journal of Informetrics, Elsevier, vol. 8(4), pages 824-839.
- David A. King, 2004. "The scientific impact of nations," Nature, Nature, vol. 430(6997), pages 311-316, July.
- Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Rodríguez-Navarro, Alonso & Brito, Ricardo, 2024. "Rank analysis of most cited publications, a new approach for research assessments," Journal of Informetrics, Elsevier, vol. 18(2).
- Melika Mosleh & Saeed Roshani & Mario Coccia, 2022. "Scientific laws of research funding to support citations and diffusion of knowledge in life science," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1931-1951, April.
- Rodríguez-Navarro, Alonso & Brito, Ricardo, 2018. "Technological research in the EU is less efficient than the world average. EU research policy risks Europeans’ future," Journal of Informetrics, Elsevier, vol. 12(3), pages 718-731.
- Gerson Pech & Catarina Delgado, 2020. "Percentile and stochastic-based approach to the comparison of the number of citations of articles indexed in different bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 223-252, April.
- Alonso Rodríguez-Navarro & Ricardo Brito, 2022. "The link between countries’ economic and scientific wealth has a complex dependence on technological activity and research policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2871-2896, May.
- Alonso Rodríguez-Navarro & Ricardo Brito, 2019. "Probability and expected frequency of breakthroughs: basis and use of a robust method of research assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 213-235, April.
- Brito, Ricardo & Navarro, Alonso Rodríguez, 2021. "The inconsistency of h-index: A mathematical analysis," Journal of Informetrics, Elsevier, vol. 15(1).
- Gerson Pech & Catarina Delgado, 2020. "Assessing the publication impact using citation data from both Scopus and WoS databases: an approach validated in 15 research fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 909-924, November.
- Pech, Gerson & Delgado, Catarina, 2021. "Screening the most highly cited papers in longitudinal bibliometric studies and systematic literature reviews of a research field or journal: Widespread used metrics vs a percentile citation-based app," Journal of Informetrics, Elsevier, vol. 15(3).
- Gabriel-Alexandru Vȋiu & Mihai Păunescu, 2021. "The lack of meaningful boundary differences between journal impact factor quartiles undermines their independent use in research evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1495-1525, February.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Alonso Rodríguez-Navarro & Ricardo Brito, 2019. "Probability and expected frequency of breakthroughs: basis and use of a robust method of research assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 213-235, April.
- Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
- Rodríguez-Navarro, Alonso & Brito, Ricardo, 2018. "Double rank analysis for research assessment," Journal of Informetrics, Elsevier, vol. 12(1), pages 31-41.
- Albarrán, Pedro & Herrero, Carmen & Ruiz-Castillo, Javier & Villar, Antonio, 2017.
"The Herrero-Villar approach to citation impact,"
Journal of Informetrics, Elsevier, vol. 11(2), pages 625-640.
- Albarrán, Pedro & Herrero, Carmen & Villar, Antonio, 2016. "The Herrero-Villar approach to citation impact," UC3M Working papers. Economics 23969, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Rodríguez-Navarro, Alonso & Brito, Ricardo, 2018. "Technological research in the EU is less efficient than the world average. EU research policy risks Europeans’ future," Journal of Informetrics, Elsevier, vol. 12(3), pages 718-731.
- Lutz Bornmann & Alexander Tekles & Loet Leydesdorff, 2019. "How well does I3 perform for impact measurement compared to other bibliometric indicators? The convergent validity of several (field-normalized) indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1187-1205, May.
- Brito, Ricardo & Navarro, Alonso Rodríguez, 2021. "The inconsistency of h-index: A mathematical analysis," Journal of Informetrics, Elsevier, vol. 15(1).
- Andrea Bonaccorsi & Tindaro Cicero & Peter Haddawy & Saeed-UL Hassan, 2017. "Explaining the transatlantic gap in research excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 217-241, January.
- Vîiu, Gabriel-Alexandru, 2018. "The lognormal distribution explains the remarkable pattern documented by characteristic scores and scales in scientometrics," Journal of Informetrics, Elsevier, vol. 12(2), pages 401-415.
- Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2013. "Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P1," Journal of Informetrics, Elsevier, vol. 7(4), pages 933-944.
- Thelwall, Mike, 2016. "The precision of the arithmetic mean, geometric mean and percentiles for citation data: An experimental simulation modelling approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 110-123.
- Schreiber, Michael, 2014. "How to improve the outcome of performance evaluations in terms of percentiles for citation frequencies of my papers," Journal of Informetrics, Elsevier, vol. 8(4), pages 873-879.
- Bonaccorsi, Andrea & Haddawy, Peter & Cicero, Tindaro & Hassan, Saeed-Ul, 2017. "The solitude of stars. An analysis of the distributed excellence model of European universities," Journal of Informetrics, Elsevier, vol. 11(2), pages 435-454.
- Loet Leydesdorff & Paul Wouters & Lutz Bornmann, 2016. "Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators—a state-of-the-art report," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2129-2150, December.
- Lutz Bornmann & Richard Williams, 2020. "An evaluation of percentile measures of citation impact, and a proposal for making them better," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1457-1478, August.
- Neus Herranz & Javier Ruiz-Castillo, 2013.
"The end of the “European Paradox”,"
Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 453-464, April.
- Herranz, Neus, 2011. "The end of the European Paradox," UC3M Working papers. Economics we1127, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Ruiz-Castillo, Javier & Herranz, Neus, 2011. "The end of the European Paradox," CEPR Discussion Papers 8674, C.E.P.R. Discussion Papers.
- Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
- Herranz, Neus & Ruiz-Castillo, Javier, 2012.
"Sub-field normalization in the multiplicative case: Average-based citation indicators,"
Journal of Informetrics, Elsevier, vol. 6(4), pages 543-556.
- Ruiz-Castillo, Javier & Herranz, Neus, 2011. "Sub-field normalization in the multiplicative case: Average-based citation indicators," CEPR Discussion Papers 8715, C.E.P.R. Discussion Papers.
- Herranz, Neus, 2012. "Sub-field normalization in the multiplicative case : average-based citation indicators," UC3M Working papers. Economics we1130, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Schreiber, Michael, 2014. "Is the new citation-rank approach P100′ in bibliometrics really new?," Journal of Informetrics, Elsevier, vol. 8(4), pages 997-1004.
- Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
More about this item
Keywords
Research assessment; Bibliometric analysis; Citation distribution; Double-rank analysis; Percentile distribution;All these keywords.
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:12:y:2018:i:1:p:315-329. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.