IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v13y2019i4s1751157719301397.html
   My bibliography  Save this article

Journal ranking should depend on the level of aggregation

Author

Listed:
  • Csató, László

Abstract

Journal ranking is becoming more important in assessing the quality of academic research. Several indices have been suggested for this purpose, typically on the basis of a citation graph between the journals. We follow an axiomatic approach and find an impossibility theorem: any self-consistent ranking method, which satisfies a natural monotonicity property, should depend on the level of aggregation. Our result presents a trade-off between two axiomatic properties and reveals a dilemma of aggregation.

Suggested Citation

  • Csató, László, 2019. "Journal ranking should depend on the level of aggregation," Journal of Informetrics, Elsevier, vol. 13(4).
    • Handle: RePEc:eee:infome:v:13:y:2019:i:4:s1751157719301397
    DOI: 10.1016/j.joi.2019.100975
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157719301397
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2019.100975?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. ,, 2014. "A ranking method based on handicaps," Theoretical Economics, Econometric Society, vol. 9(3), September.
    2. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    3. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    4. Wolfgang Glänzel & Henk F. Moed, 2013. "Opinion paper: thoughts and facts on bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 381-394, July.
    5. Woeginger, Gerhard J., 2008. "An axiomatic analysis of Egghe’s g-index," Journal of Informetrics, Elsevier, vol. 2(4), pages 364-368.
    6. László Csató, 2019. "Some Impossibilities of Ranking in Generalized Tournaments," International Game Theory Review (IGTR), World Scientific Publishing Co. Pte. Ltd., vol. 21(01), pages 1-15, March.
    7. repec:hal:pseose:halshs-01109087 is not listed on IDEAS
    8. Ignacio Palacios-Huerta & Oscar Volij, 2004. "The Measurement of Intellectual Influence," Econometrica, Econometric Society, vol. 72(3), pages 963-977, May.
    9. Miroiu, Adrian, 2013. "Axiomatizing the Hirsch index: Quantity and quality disjoined," Journal of Informetrics, Elsevier, vol. 7(1), pages 10-15.
    10. Lasso de la Vega, Casilda & Volij, Oscar, 2018. "Ranking scholars: A measure representation," Journal of Informetrics, Elsevier, vol. 12(2), pages 510-517.
    11. Kongo, Takumi, 2014. "An alternative axiomatization of the Hirsch index," Journal of Informetrics, Elsevier, vol. 8(1), pages 252-258.
    12. László Csató, 2015. "A graph interpretation of the least squares ranking method," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 44(1), pages 51-69, January.
    13. Adachi, Tsuyoshi & Kongo, Takumi, 2015. "Further axiomatizations of Egghe's g-index," Journal of Informetrics, Elsevier, vol. 9(4), pages 839-844.
    14. Pavel Yu. Chebotarev & Elena Shamis, 1998. "Characterizations of scoring methodsfor preference aggregation," Annals of Operations Research, Springer, vol. 80(0), pages 299-332, January.
    15. Quesada, Antonio, 2011. "Axiomatics for the Hirsch index and the Egghe index," Journal of Informetrics, Elsevier, vol. 5(3), pages 476-480.
    16. Satterthwaite, Mark Allen, 1975. "Strategy-proofness and Arrow's conditions: Existence and correspondence theorems for voting procedures and social welfare functions," Journal of Economic Theory, Elsevier, vol. 10(2), pages 187-217, April.
    17. Woeginger, Gerhard J., 2008. "An axiomatic characterization of the Hirsch-index," Mathematical Social Sciences, Elsevier, vol. 56(2), pages 224-232, September.
    18. Antonio Quesada, 2011. "Further characterizations of the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 107-114, April.
    19. Motty Perry & Philip J. Reny, 2016. "How to Count Citations If You Must," American Economic Review, American Economic Association, vol. 106(9), pages 2722-2741, September.
    20. László Csató, 2019. "An impossibility theorem for paired comparisons," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 27(2), pages 497-514, June.
    21. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    22. Laszlo A. Koczy & Martin Strobel, 2010. "The World Cup of Economics Journals: A Ranking by a Tournament Method," CERS-IE WORKING PAPERS 1018, Institute of Economics, Centre for Economic and Regional Studies.
    23. Palacios-Huerta, Ignacio & Volij, Oscar, 2014. "Axiomatic measures of intellectual influence," International Journal of Industrial Organization, Elsevier, vol. 34(C), pages 85-90.
    24. Antonio Quesada, 2010. "More axiomatics for the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 413-418, February.
    25. Thierry Marchant, 2009. "An axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(2), pages 325-342, August.
    26. Gibbard, Allan, 1973. "Manipulation of Voting Schemes: A General Result," Econometrica, Econometric Society, vol. 41(4), pages 587-601, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bergantiños, Gustavo & Moreno-Ternero, Juan D., 2022. "Monotonicity in sharing the revenues from broadcasting sports leagues," European Journal of Operational Research, Elsevier, vol. 297(1), pages 338-346.
    2. Arlegi, Ritxar & Dimitrov, Dinko, 2020. "Fair elimination-type competitions," European Journal of Operational Research, Elsevier, vol. 287(2), pages 528-535.
    3. Gustavo Bergantiños & Juan D. Moreno-Ternero, 2023. "Broadcasting revenue sharing after cancelling sports competitions," Annals of Operations Research, Springer, vol. 328(2), pages 1213-1238, September.
    4. Petróczy, Dóra Gréta, 2021. "An alternative quality of life ranking on the basis of remittances," Socio-Economic Planning Sciences, Elsevier, vol. 78(C).
    5. Katchanov, Yurij L. & Markova, Yulia V. & Shmatko, Natalia A., 2019. "The distinction machine: Physics journals from the perspective of the Kolmogorov–Smirnov statistic," Journal of Informetrics, Elsevier, vol. 13(4).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. L'aszl'o Csat'o, 2019. "Journal ranking should depend on the level of aggregation," Papers 1904.06300, arXiv.org, revised Sep 2019.
    2. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    3. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    4. Sylvain Béal & Eric Rémila & Philippe Solal & Sylvain Ferrières, 2016. "An axiomatization of the iterated h-index and applications to sport rankings," Working Papers hal-01394818, HAL.
    5. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    6. Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
    7. Antonin Macé, 2023. "The Limits of Citation Counts," Working Papers halshs-01630095, HAL.
    8. Mark Levene & Trevor Fenner & Judit Bar-Ilan, 2019. "Characterisation of the $$\chi$$χ-index and the rec-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 885-896, August.
    9. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    10. Woeginger, Gerhard J., 2014. "Investigations on the step-based research indices of Chambers and Miller," Journal of Informetrics, Elsevier, vol. 8(3), pages 659-666.
    11. Lasso de la Vega, Casilda & Volij, Oscar, 2018. "Ranking scholars: A measure representation," Journal of Informetrics, Elsevier, vol. 12(2), pages 510-517.
    12. Mukherjee, Conan & Alam, Aftab, 2016. "On Evaluating Author's Performance by Publications: An Axiomatic Study," Working Papers 2016:14, Lund University, Department of Economics, revised 12 May 2017.
    13. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    14. J. Atsu Amegashie, 2020. "Citations And Incentives In Academic Contests," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1233-1244, July.
    15. Haley, M. Ryan & McGee, M. Kevin, 2020. "Jointly valuing journal visibility and author citation count: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 14(1).
    16. Conan Mukherjee & Ranojoy Basu & Aftab Alam, 2020. "A measure of authorship by publications," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 41(3), pages 354-361, April.
    17. Adachi, Tsuyoshi & Kongo, Takumi, 2015. "Further axiomatizations of Egghe's g-index," Journal of Informetrics, Elsevier, vol. 9(4), pages 839-844.
    18. Csató, László & Tóth, Csaba, 2020. "University rankings from the revealed preferences of the applicants," European Journal of Operational Research, Elsevier, vol. 286(1), pages 309-320.
    19. Antonin Mac'e, 2017. "The Limits of Citation Counts," Papers 1711.02695, arXiv.org, revised Sep 2023.
    20. Flores-Szwagrzak, Karol & Treibich, Rafael, 2015. "Co-authorship and the Measurement of Individual Productivity," Discussion Papers on Economics 17/2015, University of Southern Denmark, Department of Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:13:y:2019:i:4:s1751157719301397. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.