IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i9d10.1007_s11192-021-04045-3.html
   My bibliography  Save this article

Journal ratings: a paper affiliation methodology

Author

Listed:
  • Domingo Docampo

    (University of Vigo)

  • Vicente Safón

    (University of Valencia)

Abstract

Journal ratings are a key factor when individuals or institutions assess research and select journals. Despite their significant development in recent years, ratings are still not sufficiently precise or updated enough, depending on the subject and method used. In this paper, we present a new methodology, called paper affiliation index, with which to create subject journal ratings using expert judgment and research impact, both of which are based on secondary, objective measures, thus making it possible to produce lists every year without human manipulation at virtually no cost. We compare the results of our methodology with the selection of top journals by the ShanghaiRanking’s Global Ranking of Academic Subjects in the Social Sciences, the Excellence in Research for Australia, Australian Business Dean Council, the Academic Journal Guide release by Chartered Association of Business Schools lists, and the Journal Impact Factor calculated by Clarivate Analytics.

Suggested Citation

  • Domingo Docampo & Vicente Safón, 2021. "Journal ratings: a paper affiliation methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 8063-8090, September.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:9:d:10.1007_s11192-021-04045-3
    DOI: 10.1007/s11192-021-04045-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-021-04045-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-021-04045-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nianhang Xu & Kam C. Chan & Chih-Hsiang Chang, 2016. "A quality-based global assessment of financial research," Review of Quantitative Finance and Accounting, Springer, vol. 46(3), pages 605-631, April.
    2. James J. Heckman & Sidharth Moktan, 2020. "Publishing and promotion in economics - The tyranny of the Top Five," Vox eBook Chapters, in: Sebastian Galliani & Ugo Panizza (ed.), Publishing and Measuring Success in Economics, edition 1, volume 1, chapter 1, pages 23-32, Centre for Economic Policy Research.
    3. Serenko, Alexander & Bontis, Nick, 2013. "First in, best dressed: The presence of order-effect bias in journal ranking surveys," Journal of Informetrics, Elsevier, vol. 7(1), pages 138-144.
    4. Currie, Russell R. & Pandher, Gurupdesh S., 2011. "Finance journal rankings and tiers: An Active Scholar Assessment methodology," Journal of Banking & Finance, Elsevier, vol. 35(1), pages 7-20, January.
    5. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    6. Joseph Gerald Hirschberg & Jeanette Ngaire Lye, 2020. "Grading Journals In Economics: The Abcs Of The Abdc," Journal of Economic Surveys, Wiley Blackwell, vol. 34(4), pages 876-921, September.
    7. Bajo, Emanuele & Barbi, Massimiliano & Hillier, David, 2020. "Where should I publish to get promoted? A finance journal ranking based on business school promotions," Journal of Banking & Finance, Elsevier, vol. 114(C).
    8. Kam C. Chan & Carl R. Chen & Thomas L. Steiner, 2002. "Production in the Finance Literature, Institutional Reputation, and Labor Mobility in Academia: A Global Perspective," Financial Management, Financial Management Association, vol. 31(4), Winter.
    9. Serenko, Alexander & Dohan, Michael, 2011. "Comparing the expert survey and citation impact journal ranking methods: Example from the field of Artificial Intelligence," Journal of Informetrics, Elsevier, vol. 5(4), pages 629-648.
    10. Michael D. Gordon, 1982. "Citation ranking versus subjective evaluation in the determination of journal hierachies in the social sciences," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 33(1), pages 55-57, January.
    11. Rodney H. Mabry & Arthur D. Sharplin, 1985. "The Relative Importance Of Journals Used In Finance Research," Journal of Financial Research, Southern Finance Association;Southwestern Finance Association, vol. 8(4), pages 287-296, December.
    12. Alireza Tahai & Michael J. Meyer, 1999. "A revealed preference study of management journals’ direct influences," Strategic Management Journal, Wiley Blackwell, vol. 20(3), pages 279-296, March.
    13. Luh-Yu (Louie) Ren, 2016. "A Note about the Finance Journal Rankings and Citation Counts," Asian Academy of Management Journal of Accounting and Finance (AAMJAF), Penerbit Universiti Sains Malaysia, vol. 12(Suppl. 1), pages 183–194-1.
    14. Vincent Larivière & Yves Gingras, 2010. "The impact factor's Matthew Effect: A natural experiment in bibliometrics," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(2), pages 424-427, February.
    15. Bryce, Cormac & Dowling, Michael & Lucey, Brian, 2020. "The journal quality perception gap," Research Policy, Elsevier, vol. 49(5).
    16. Currie, Russell R. & Pandher, Gurupdesh S., 2020. "Finance journal rankings: Active scholar assessment revisited," Journal of Banking & Finance, Elsevier, vol. 111(C).
    17. Lawrence D. Brown & Ronald J. Huefner, 1994. "The Familiarity with and Perceived Quality of Accounting Journals: Views of Senior Accounting Faculty in Leading U.S. MBA Programs," Contemporary Accounting Research, John Wiley & Sons, vol. 11(1), pages 223-250, June.
    18. Imad Moosa, 2011. "The demise of the ARC journal ranking scheme: an ex post analysis of the accounting and finance journals," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 51(3), pages 809-836, September.
    19. Vivien Beattie & Alan Goodacre, 2006. "A new method for ranking academic journals in accounting and finance," Accounting and Business Research, Taylor & Francis Journals, vol. 36(2), pages 65-91.
    20. repec:bla:jfinan:v:53:y:1998:i:1:p:351-363 is not listed on IDEAS
    21. I Horowitz, 2003. "Preference-neutral attribute weights in the journal-ranking problem," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(5), pages 452-457, May.
    22. Cudd, Mike & Morris, Joe, 1988. "Bias in Journal Ratings," The Financial Review, Eastern Finance Association, vol. 23(1), pages 117-125, February.
    23. Eleftheriou, Konstantinos & Polemis, Michael, 2020. "One list to fit them all: What do we learn from journal ranking?," Finance Research Letters, Elsevier, vol. 35(C).
    24. Vincent Larivière & Yves Gingras, 2010. "The impact factor's Matthew Effect: A natural experiment in bibliometrics," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(2), pages 424-427, February.
    25. Vanclay, Jerome K., 2011. "An evaluation of the Australian Research Council's journal ranking," Journal of Informetrics, Elsevier, vol. 5(2), pages 265-274.
    26. Vogel, Rick & Hattke, Fabian & Petersen, Jessica, 2017. "Journal rankings in management and business studies: What rules do we play by?," Research Policy, Elsevier, vol. 46(10), pages 1707-1722.
    27. Fang Xu & Wenbin Liu & Ronald Rousseau, 2015. "Introducing sub-impact factor (SIF-) sequences and an aggregated SIF-indicator for journal ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1577-1593, February.
    28. Tom Arnold, 2003. "Impact: What Influences Finance Research?," The Journal of Business, University of Chicago Press, vol. 76(2), pages 343-362, April.
    29. Alberto Baccini & Giuseppe De Nicolao, 2016. "Do they agree? Bibliometric evaluation versus informed peer review in the Italian research assessment exercise," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1651-1671, September.
    30. Alexander, John C & Mabry, Rodney H, 1994. "Relative Significance of Journals, Authors, and Articles Cited in Financial Research," Journal of Finance, American Finance Association, vol. 49(2), pages 697-712, June.
    31. Stefano Benati & Silvana Stefani, 2011. "The Academic Journal Ranking Problem: A Fuzzy-Clustering Approach," Journal of Classification, Springer;The Classification Society, vol. 28(1), pages 7-20, April.
    32. Moussa, Salim & Touzani, Mourad, 2010. "Ranking marketing journals using the Google Scholar-based hg-index," Journal of Informetrics, Elsevier, vol. 4(1), pages 107-117.
    33. Chan, Kam C. & Chang, Chih-Hsiang & Chang, Yuanchen, 2013. "Ranking of finance journals: Some Google Scholar citation perspectives," Journal of Empirical Finance, Elsevier, vol. 21(C), pages 241-250.
    34. Borokhovich, Kenneth A. & Lee, Allissa A. & Simkins, Betty J., 2011. "A framework for journal assessment: The case of the Journal of Banking & Finance," Journal of Banking & Finance, Elsevier, vol. 35(1), pages 1-6, January.
    35. Gianfranco Ennas & Battista Biggio & Maria Chiara Di Guardo, 2015. "Data-driven journal meta-ranking in business and management," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1911-1929, December.
    36. Michael F. Gorman & John J. Kanet, 2005. "Evaluating Operations Management--Related Journals via the Author Affiliation Index," Manufacturing & Service Operations Management, INFORMS, vol. 7(1), pages 3-19, February.
    37. Imad A. Moosa, 2016. "A Critique of the Bucket Classification of Journals: The ABDC List as an Example," The Economic Record, The Economic Society of Australia, vol. 92(298), pages 448-463, September.
    38. Roland Herrmann & Ernst Berg & Stephan Dabbert & Siegfried Pöchtrager & Klaus Salhofer, 2011. "Going Beyond Impact Factors: A Survey‐based Journal Ranking by Agricultural Economists," Journal of Agricultural Economics, Wiley Blackwell, vol. 62(3), pages 710-732, September.
    39. Philip M. Podsakoff & Scott B. MacKenzie & Daniel G. Bachrach & Nathan P. Podsakoff, 2005. "The influence of management journals in the 1980s and 1990s," Strategic Management Journal, Wiley Blackwell, vol. 26(5), pages 473-488, May.
    40. Haddawy, Peter & Hassan, Saeed-Ul & Asghar, Awais & Amin, Sarah, 2016. "A comprehensive examination of the relation of three citation-based journal metrics to expert judgment of journal quality," Journal of Informetrics, Elsevier, vol. 10(1), pages 162-173.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nerantzidis, Michail & Tampakoudis, Ioannis & She, Chaoyuan, 2024. "Social media in accounting research: A review and future research agenda," Journal of International Accounting, Auditing and Taxation, Elsevier, vol. 54(C).
    2. Vicente Safón & Domingo Docampo, 2023. "What are you reading? From core journals to trendy journals in the Library and Information Science (LIS) field," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2777-2801, May.
    3. Gustavo Dambiski Gomes de Carvalho & Luis Mauricio Martins de Resende & Joseane Pontes & Hélio Gomes de Carvalho & Leozenir Mendes Betim, 2021. "Innovation and Management in MSMEs: A Literature Review of Highly Cited Papers," SAGE Open, , vol. 11(4), pages 21582440211, October.
    4. Gangan Prathap, 2023. "Letter to the Editor: Comments on the paper of Safón and Docampo: what are you reading? From core journals to trendy journals in the Library and Information Science (LIS) field," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(7), pages 4137-4142, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Currie, Russell R. & Pandher, Gurupdesh S., 2020. "Finance journal rankings: Active scholar assessment revisited," Journal of Banking & Finance, Elsevier, vol. 111(C).
    2. Grossmann, Axel & Lee, Allissa, 2022. "An analysis of finance journal accessibility: Author inclusivity and journal quality," Journal of Banking & Finance, Elsevier, vol. 138(C).
    3. Bajo, Emanuele & Barbi, Massimiliano & Hillier, David, 2020. "Where should I publish to get promoted? A finance journal ranking based on business school promotions," Journal of Banking & Finance, Elsevier, vol. 114(C).
    4. Kerl, Alexander & Miersch, Enrico & Walter, Andreas, 2018. "Evaluation of academic finance conferences," Journal of Banking & Finance, Elsevier, vol. 89(C), pages 26-38.
    5. Matthew D. Crook & Brian R. Walkup, 2016. "Rankings And Trends In Finance Publishing: An Iterative Approach," Journal of Financial Research, Southern Finance Association;Southwestern Finance Association, vol. 39(3), pages 291-322, September.
    6. Jon A. Garfinkel & Mosab Hammoudeh & James Weston, 2024. "Academic publishing behavior and pay across business fields," Financial Management, Financial Management Association International, vol. 53(1), pages 31-58, March.
    7. Rosenthal, Edward C. & Weiss, Howard J., 2017. "A data envelopment analysis approach for ranking journals," Omega, Elsevier, vol. 70(C), pages 135-147.
    8. Thelwall, Mike & Kousha, Kayvan & Stuart, Emma & Makita, Meiko & Abdoli, Mahshid & Wilson, Paul & Levitt, Jonathan, 2023. "Do bibliometrics introduce gender, institutional or interdisciplinary biases into research evaluations?," Research Policy, Elsevier, vol. 52(8).
    9. Berninger, Marc & Kiesel, Florian & Schiereck, Dirk & Gaar, Eduard, 2021. "Citations and the readers’ information-extracting costs of finance articles," Journal of Banking & Finance, Elsevier, vol. 131(C).
    10. Bryce, Cormac & Dowling, Michael & Lucey, Brian, 2020. "The journal quality perception gap," Research Policy, Elsevier, vol. 49(5).
    11. Currie, Russell R. & Pandher, Gurupdesh S., 2011. "Finance journal rankings and tiers: An Active Scholar Assessment methodology," Journal of Banking & Finance, Elsevier, vol. 35(1), pages 7-20, January.
    12. Konstantinos Eleftheriou & Patroklos Patsoulis & Michael Polemis, 2023. "Convergence among academic journals in accounting: a note," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(2), pages 1055-1069, February.
    13. Morris G. Danielson & Jean L. Heck, 2014. "Voting with Their Feet: In Which Journals Do the Most Prolific Finance Researchers Publish?," Financial Management, Financial Management Association International, vol. 43(1), pages 1-27, March.
    14. Luh-Yu (Louie) Ren, 2016. "A Note about the Finance Journal Rankings and Citation Counts," Asian Academy of Management Journal of Accounting and Finance (AAMJAF), Penerbit Universiti Sains Malaysia, vol. 12(Suppl. 1), pages 183–194-1.
    15. Thomas Lange, 2006. "The Imprecise Science of Evaluating Scholarly Performance," Evaluation Review, , vol. 30(4), pages 505-532, August.
    16. Chongyu Dang & Zhichuan (Frank) Li, 2020. "Drivers of research impact: evidence from the top three finance journals," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 60(3), pages 2759-2809, September.
    17. Drivas, Kyriakos & Kremmydas, Dimitris, 2020. "The Matthew effect of a journal's ranking," Research Policy, Elsevier, vol. 49(4).
    18. Osterloh, Margit & Frey, Bruno S., 2020. "How to avoid borrowed plumes in academia," Research Policy, Elsevier, vol. 49(1).
    19. Piotr Śpiewanowski & Oleksandr Talavera, 2021. "Journal rankings and publication strategy," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3227-3242, April.
    20. Mario Fernandes & Andreas Walter, 2023. "The times they are a-changin’: profiling newly tenured business economics professors in Germany over the past thirty years," Journal of Business Economics, Springer, vol. 93(5), pages 929-971, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:9:d:10.1007_s11192-021-04045-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.