IDEAS home Printed from https://ideas.repec.org/p/rua/wpaper/apru175ye20161.html
   My bibliography  Save this paper

Ранжирование Российских Экономических Журналов: Научный Метод Или «Игра В Цыфирь»?
[Ran Ranking of Russian Economic Journals: The Scientific Method or “Numbers Game”?]

Author

Listed:
  • Рубинштейн Александр Яковлевич

Abstract

Статья посвящена общим проблемам ранжирования журналов на примере критического анализа трех рейтингов российских экономических журналов, предложенных в последние годы. Методология построения данных рейтингов связана с данными РИНЦ, результатами экспертных опросов и комбинацией этих подходов. Выявлены принципиальные недостатки каждого из рейтингов и показано, что уязвимым местом таких разработок являются относительно произвольный выбор библиометрических индикаторов и их слабая корреляция с научным авторитетом журналов, недостаточно обоснованная процедура агрегирования используемых показателей и/ или экспертных оценок, а также нерепрезентативность опросов экспертов. В работе представлен пассивный эксперимент, в рамках которого сопоставлены результаты ранжирования журналов по указанным трем рейтингам и трем дополнительным критериям. Сделан общий вывод о невысоком уровне развития подобных исследований и отсутствии реальных оснований для применения указанных рейтингов в практике управления наукой и стимулирования труда ученых. This article is devoted to general problems of ranking on the example of critical analysis of the three ratings of Russian economic journals suggested in recent years, the methodology of construction of which are connected with the Russian Science Citation Index (RSCI) data, results of expert surveys and a combination of these approaches. The fundamental disadvantages of each of them are revealed and it is shown that the vulnerable points of such developments are relatively arbitrary choice of bibliometric indicators and their weak correlation with academic authority of journals, insufficiently substantiated procedure of aggregation of used indicators and/or expert analysis, as well as surveys of experts are not representative. This paper presents a «passive experiment», in terms of which were mapped the results of the ranking of journals, based on the three ratings and three additional criteria. Made the overall conclusion about the low level of development of such researches and the lack of real grounds for the application of these ratings in the practice of science management and motivation of scientists.

Suggested Citation

  • Рубинштейн Александр Яковлевич, "undated". "Ранжирование Российских Экономических Журналов: Научный Метод Или «Игра В Цыфирь»? [Ran Ranking of Russian Economic Journals: The Scientific Method or “Numbers Game”?]," Working papers a:pru175:ye:2016:1, Institute of Economics.
    • Handle: RePEc:rua:wpaper:a:pru175:ye:2016:1
    as

    Download full text from publisher

    File URL: http://journal.econorus.org/pdf/NEA-30.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. van Eck, Nees Jan & Waltman, Ludo, 2014. "CitNetExplorer: A new software tool for analyzing and visualizing citation networks," Journal of Informetrics, Elsevier, vol. 8(4), pages 802-823.
    2. Fuad Aleskerov, 2006. "Power Indices Taking into Account Agents’ Preferences," Studies in Choice and Welfare, in: Bruno Simeone & Friedrich Pukelsheim (ed.), Mathematics and Democracy, pages 1-18, Springer.
    3. Michael H. MacRoberts & Barbara R. MacRoberts, 1989. "Problems of citation analysis: A critical review," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 40(5), pages 342-349, September.
    4. Loet Leydesdorff & Lutz Bornmann, 2011. "Integrated impact indicators compared with impact factors: An alternative research design with policy implications," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(11), pages 2133-2146, November.
    5. Katherine W. McCain, 1991. "Mapping economics through the journal literature: An experiment in journal cocitation analysis," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 42(4), pages 290-296, May.
    6. Henry Small, 1973. "Co‐citation in the scientific literature: A new measure of the relationship between two documents," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 24(4), pages 265-269, July.
    7. Wang, Zhongmin, 2009. "Station level gasoline demand in an Australian market with regular price cycles," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 53(4), pages 1-17.
    8. Nida Denson & Katalin Szelényi & Kate Bresonis, 2018. "Correlates of Work-Life Balance for Faculty Across Racial/Ethnic Groups," Research in Higher Education, Springer;Association for Institutional Research, vol. 59(2), pages 226-247, March.
    9. E. Balatsky & N. Ekimova., 2015. "The Experience of Ranking Russian Economic Journals," VOPROSY ECONOMIKI, N.P. Redaktsiya zhurnala "Voprosy Economiki", vol. 8.
    10. Loet Leydesdorff, 2008. "Caveats for the use of citation indicators in research and journal evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(2), pages 278-287, January.
    11. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    12. Guerrero-Bote, Vicente P. & Moya-Anegón, Félix, 2012. "A further step forward in measuring journals’ scientific prestige: The SJR2 indicator," Journal of Informetrics, Elsevier, vol. 6(4), pages 674-688.
    13. Loet Leydesdorff, 2006. "Can scientific journals be classified in terms of aggregated journal‐journal citation relations using the Journal Citation Reports?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(5), pages 601-613, March.
    14. A. Muravyev., 2013. "On Scientific Value of Russian Journals in Economics and Related Fields," VOPROSY ECONOMIKI, N.P. Redaktsiya zhurnala "Voprosy Economiki", vol. 4.
    15. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    16. Fuad Aleskerov & Irina Andrievskaya & Elena Permjakova, 2014. "Key Borrowers Detected By The Intensities Of Their Short-range Interactions," HSE Working papers WP BRP 33/FE/2014, National Research University Higher School of Economics.
    17. Loet Leydesdorff, 2007. "Visualization of the citation impact environments of scientific journals: An online mapping exercise," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(1), pages 25-38, January.
    18. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Рубинштейн Александр Яковлевич & Бураков Н. А. & Славинская О. А., "undated". "Сообщество Экономистов И Экономические Журналы (Социологические Измерения Vs Библиометрии). Научный Доклад [Community of economists and Russian economic journals (sociological measurements VS bibli," Working papers a:pru175:ye:2017:2, Institute of Economics.
    2. Kuklin, Alexander A. (Куклин, Александр) & Balyakina, Evgeniya A. (Балякина, Евгения), 2017. "Active policy as a key to success for an International Economic Periodical [Активная Политика — Залог Успеха Международного Экономического Журнала]," Ekonomicheskaya Politika / Economic Policy, Russian Presidential Academy of National Economy and Public Administration, vol. 6, pages 160-177, December.
    3. Balatsky, Evgeny V. (Балацкий, Евгений) & Ekimova, Natalia A. (Екимова, Наталья), 2017. "Competition of Economic Journals in Russia: Results of the Three Waves of Ranking [Конкуренция Экономических Журналов России: Итоги Трех Волн Рейтингования]," Ekonomicheskaya Politika / Economic Policy, Russian Presidential Academy of National Economy and Public Administration, vol. 6, pages 178-201, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aleskerov, F. & Badgaeva, D. & Pislyakov, V. & Sterligov, I. & Shvydun, S., 2016. "An Importance of Russian and International Economic Journals: a Network Approach," Journal of the New Economic Association, New Economic Association, vol. 30(2), pages 193-205.
    2. Lutz Bornmann & Alexander Tekles & Loet Leydesdorff, 2019. "How well does I3 perform for impact measurement compared to other bibliometric indicators? The convergent validity of several (field-normalized) indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1187-1205, May.
    3. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Costa, 2023. "Correlating article citedness and journal impact: an empirical investigation by field on a large-scale dataset," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1877-1894, March.
    4. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    5. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    6. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    7. Loet Leydesdorff & Paul Wouters & Lutz Bornmann, 2016. "Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators—a state-of-the-art report," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2129-2150, December.
    8. Schmal, W. Benedikt & Haucap, Justus & Knoke, Leon, 2023. "The role of gender and coauthors in academic publication behavior," Research Policy, Elsevier, vol. 52(10).
    9. Sergio Copiello, 2019. "Research Interest: another undisclosed (and redundant) algorithm by ResearchGate," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 351-360, July.
    10. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    11. Currie, Russell R. & Pandher, Gurupdesh S., 2020. "Finance journal rankings: Active scholar assessment revisited," Journal of Banking & Finance, Elsevier, vol. 111(C).
    12. Fargher, Neil & Wee, Marvin, 2019. "The impact of Ball and Brown (1968) on generations of research," Pacific-Basin Finance Journal, Elsevier, vol. 54(C), pages 55-72.
    13. Andreas Thor & Lutz Bornmann & Werner Marx & Rüdiger Mutz, 2018. "Identifying single influential publications in a research field: new analysis opportunities of the CRExplorer," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 591-608, July.
    14. Zhesi Shen & Liying Yang & Zengru Di & Jinshan Wu, 2019. "Large enough sample size to rank two groups of data reliably according to their means," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 653-671, February.
    15. Xu, Shuqi & Mariani, Manuel Sebastian & Lü, Linyuan & Medo, Matúš, 2020. "Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data," Journal of Informetrics, Elsevier, vol. 14(1).
    16. Bornmann, Lutz & Haunschild, Robin, 2016. "Citation score normalized by cited references (CSNCR): The introduction of a new citation impact indicator," Journal of Informetrics, Elsevier, vol. 10(3), pages 875-887.
    17. Zharova, Alona & Härdle, Wolfgang Karl & Lessmann, Stefan, 2017. "Is scientific performance a function of funds?," SFB 649 Discussion Papers 2017-028, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    18. Bornmann, Lutz & Haunschild, Robin & Mutz, Rüdiger, 2020. "Should citations be field-normalized in evaluative bibliometrics? An empirical analysis based on propensity score matching," Journal of Informetrics, Elsevier, vol. 14(4).
    19. Yao, Ye & Du, Huibin & Zou, Hongyang & Zhou, Peng & Antunes, Carlos Henggeler & Neumann, Anne & Yeh, Sonia, 2023. "Fifty years of Energy Policy: A bibliometric overview," Energy Policy, Elsevier, vol. 183(C).
    20. Loet Leydesdorff, 2012. "Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 355-365, August.

    More about this item

    Keywords

    journal ranking; bibliometric indicators; citation; expert analysis; aggregation; arranging;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:rua:wpaper:a:pru175:ye:2016:1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/derasru.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.