IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v88y2011i3d10.1007_s11192-011-0421-y.html
   My bibliography  Save this article

Measuring economic journals’ citation efficiency: a data envelopment analysis approach

Author

Listed:
  • George Emm Halkos

    (University of Thessaly)

  • Nickolaos G. Tzeremes

    (University of Thessaly)

Abstract

This paper by using data envelopment analysis (DEA) and statistical inference evaluates the citation performance of 229 economic journals. The paper categorizes the journals into four main categories (A–D) based on their efficiency levels. The results are then compared to the 27 “core economic journals” as introduced by Diamond (Curr Contents 21(1):4–11, 1989). The results reveal that after more than 20 years Diamonds’ list of “core economic journals” is still valid. Finally, for the first time the paper uses data from four well-known databases (SSCI, Scopus, RePEc, Econlit) and two quality ranking reports (Kiel Institute internals ranking and ABS quality ranking report) in a DEA setting and in order to derive the ranking of 229 economic journals. The ten economic journals with the highest citation performance are Journal of Political Economy, Econometrica, Quarterly Journal of Economics, Journal of Financial Economics, Journal of Economic Literature, American Economic Review, Review of Economic Studies, Journal of Econometrics, Journal of Finance, Brookings Papers on Economic Activity.

Suggested Citation

  • George Emm Halkos & Nickolaos G. Tzeremes, 2011. "Measuring economic journals’ citation efficiency: a data envelopment analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 979-1001, September.
    • Handle: RePEc:spr:scient:v:88:y:2011:i:3:d:10.1007_s11192-011-0421-y
    DOI: 10.1007/s11192-011-0421-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-011-0421-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-011-0421-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Leopold Simar & Paul Wilson, 2000. "A general methodology for bootstrapping in non-parametric frontier models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 27(6), pages 779-802.
    2. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2011. "An updated ranking of academic journals in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 44(4), pages 1525-1538, November.
    3. Léopold Simar & Paul Wilson, 2000. "Statistical Inference in Nonparametric Frontier Models: The State of the Art," Journal of Productivity Analysis, Springer, vol. 13(1), pages 49-78, January.
    4. Kodrzycki Yolanda K. & Yu Pingkang, 2006. "New Approaches to Ranking Economics Journals," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(1), pages 1-44, August.
    5. Francesc Pujol, 2008. "Ranking Journals Following a Matching Model Approach: An Application to Public Economics Journals," Journal of Public Economic Theory, Association for Public Economic Theory, vol. 10(1), pages 55-76, February.
    6. Léopold Simar & Paul W. Wilson, 1998. "Sensitivity Analysis of Efficiency Scores: How to Bootstrap in Nonparametric Frontier Models," Management Science, INFORMS, vol. 44(1), pages 49-61, January.
    7. Goio Etxebarria & Mikel Gomez-Uranga, 2010. "Use of Scopus and Google Scholar to measure social sciences production in four major Spanish universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 333-349, February.
    8. Moed, Henk F., 2010. "Measuring contextual citation impact of scientific journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 265-277.
    9. Finn Førsund & Nikias Sarafoglou, 2002. "On the Origins of Data Envelopment Analysis," Journal of Productivity Analysis, Springer, vol. 17(1), pages 23-40, January.
    10. Friedrich Schneider & Heinrich W. Ursprung, 2008. "The 2008 GEA Journal‐Ranking for the Economics Profession," German Economic Review, Verein für Socialpolitik, vol. 9(4), pages 532-538, November.
    11. Kodrzycki Yolanda K. & Yu Pingkang, 2006. "New Approaches to Ranking Economics Journals," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(1), pages 1-44.
    12. George Emm. Halkos & Nickolaos G. Tzeremes, 2007. "International Competitiveness in the ICT Industry: Evaluating the Performance of the Top 50 Companies," Global Economic Review, Taylor & Francis Journals, vol. 36(2), pages 167-182.
    13. Simar, Leopold & Wilson, Paul W., 2002. "Non-parametric tests of returns to scale," European Journal of Operational Research, Elsevier, vol. 139(1), pages 115-132, May.
    14. Ignacio Palacios-Huerta & Oscar Volij, 2004. "The Measurement of Intellectual Influence," Econometrica, Econometric Society, vol. 72(3), pages 963-977, May.
    15. repec:bla:germec:v:9:y:2008:i::p:402-430 is not listed on IDEAS
    16. Laszlo A. Koczy & Martin Strobel, 2010. "The World Cup of Economics Journals: A Ranking by a Tournament Method," CERS-IE WORKING PAPERS 1018, Institute of Economics, Centre for Economic and Regional Studies.
    17. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    18. Norris, Michael & Oppenheim, Charles, 2007. "Comparing alternatives to the Web of Science for coverage of the social sciences’ literature," Journal of Informetrics, Elsevier, vol. 1(2), pages 161-169.
    19. Lokman I. Meho & Kiduk Yang, 2007. "Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(13), pages 2105-2125, November.
    20. Wolfgang Glänzel & Henk F. Moed, 2002. "Journal impact measures in bibliometric research," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 171-193, February.
    21. Loet Leydesdorff & Félix de Moya-Anegón & Vicente P. Guerrero-Bote, 2010. "Journal maps on the basis of Scopus data: A comparison with the Journal Citation Reports of the ISI," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(2), pages 352-369, February.
    22. Andrea Bonaccorsi & Cinzia Daraio & Léopold Simar, 2006. "Advanced indicators of productivity of universitiesAn application of robust nonparametric methods to Italian data," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(2), pages 389-410, February.
    23. Wade D. Cook & Tal Raviv & Alan J. Richardson, 2010. "Aggregating Incomplete Lists of Journal Rankings: An Application to Academic Accounting Journals," Accounting Perspectives, John Wiley & Sons, vol. 9(3), pages 217-235, September.
    24. H. L. Patterson, 1957. "Discussion," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 5(1), pages 20-23, March.
    25. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    26. Burton, M P & Phimister, Euan, 1995. "Core Journals: A Reappraisal of the Diamond List," Economic Journal, Royal Economic Society, vol. 105(429), pages 361-373, March.
    27. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    28. repec:bla:germec:v:9:y:2008:i::p:532-538 is not listed on IDEAS
    29. Førsund, Finn R. & Kittelsen, Sverre A. & Krivonozhko, Vladimir E., 2007. "Farrell Revisited: Visualising the DEA Production Frontier," Memorandum 15/2007, Oslo University, Department of Economics.
    30. Laband, David N & Piette, Michael J, 1994. "The Relative Impacts of Economics Journals: 1970-1990," Journal of Economic Literature, American Economic Association, vol. 32(2), pages 640-666, June.
    31. Johan Bollen & Herbert Van de Sompel, 2008. "Usage impact factor: The effects of sample characteristics on usage‐based impact metrics," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(1), pages 136-149, January.
    32. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, vol. 22(1), pages 77-88, March.
    33. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003. "Rankings of Academic Journals and Institutions in Economics," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1346-1366, December.
    34. A. W. Wood, 1957. "Discussion," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 5(2), pages 62-64, July.
    35. Michel Zitt & Henry Small, 2008. "Modifying the journal impact factor by fractional citation weighting: The audience factor," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1856-1860, September.
    36. Gaines Liner & Minesh Amin, 2004. "Methods of ranking economics journals," Atlantic Economic Journal, Springer;International Atlantic Economic Society, vol. 32(2), pages 140-149, June.
    37. Andrea Bonaccorsi & Cinzia Daraio, 2008. "The differentiation of the strategic profile of higher education institutions. New positioning indicators based on microdata," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 15-37, January.
    38. Ritzberger Klaus, 2008. "A Ranking of Journals in Economics and Related Fields," German Economic Review, De Gruyter, vol. 9(4), pages 402-430, December.
    39. Massimo Franceschet, 2010. "A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 243-258, April.
    40. George Halkos & Nickolaos Tzeremes, 2010. "The effect of foreign ownership on SMEs performance: An efficiency analysis perspective," Journal of Productivity Analysis, Springer, vol. 34(2), pages 167-180, October.
    41. Coelli, Tim & Perelman, Sergio, 1999. "A comparison of parametric and non-parametric distance functions: With application to European railways," European Journal of Operational Research, Elsevier, vol. 117(2), pages 326-339, September.
    42. Richard Klavans & Kevin W. Boyack, 2009. "Toward a consensus map of science," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(3), pages 455-476, March.
    43. Kayvan Kousha & Mike Thelwall, 2008. "Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 273-294, February.
    44. López-Illescas, Carmen & de Moya-Anegón, Félix & Moed, Henk F., 2008. "Coverage and citation impact of oncological journals in the Web of Science and Scopus," Journal of Informetrics, Elsevier, vol. 2(4), pages 304-316.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Halkos, George & Tzeremes, Nickolaos, 2012. "Ranking agricultural, environmental and natural resource economics journals: A note," MPRA Paper 36233, University Library of Munich, Germany.
    2. Delbianco, Fernando & Fioriti, Andrés & Hernandez-Chanto, Allan & Tohmé, Fernando, 2020. "A Markov-switching approach to the study of citations in academic journals," Journal of Informetrics, Elsevier, vol. 14(4).
    3. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    4. Biresh K Sahoo & Ramadhar Singh & Bineet Mishra & Krithiga Sankaran, 2015. "Research Productivity in Management Schools of India: A Directional Benefit-of-Doubt Model Analysis," Working Papers id:7139, eSocialSciences.
    5. Eleftheriou, Konstantinos & Polemis, Michael, 2020. "One list to fit them all: What do we learn from journal ranking?," Finance Research Letters, Elsevier, vol. 35(C).
    6. Marcel Clermont & Julia Schaefer, 2019. "Identification of Outliers in Data Envelopment Analysis," Schmalenbach Business Review, Springer;Schmalenbach-Gesellschaft, vol. 71(4), pages 475-496, October.
    7. Petridis, Konstantinos & Malesios, Chrisovalantis & Arabatzis, Garyfallos & Thanassoulis, Emmanuel, 2013. "Efficiency analysis of forestry journals: Suggestions for improving journals’ quality," Journal of Informetrics, Elsevier, vol. 7(2), pages 505-521.
    8. Rosenthal, Edward C. & Weiss, Howard J., 2017. "A data envelopment analysis approach for ranking journals," Omega, Elsevier, vol. 70(C), pages 135-147.
    9. Johan Lyhagen & Per Ahlgren, 2020. "Uncertainty and the ranking of economics journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2545-2560, December.
    10. Frederic S. Lee & Xuan Pham & Gyun Gu, 2013. "The UK Research Assessment Exercise and the narrowing of UK economics," Cambridge Journal of Economics, Cambridge Political Economy Society, vol. 37(4), pages 693-717.
    11. David I. Stern, 2013. "Uncertainty Measures for Economics Journal Impact Factors," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 173-189, March.
    12. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    13. Seiler, Christian & Wohlrabe, Klaus, 2013. "Archetypal scientists," Journal of Informetrics, Elsevier, vol. 7(2), pages 345-356.
    14. Tüselmann, Heinz & Sinkovics, Rudolf R. & Pishchulov, Grigory, 2016. "Revisiting the standing of international business journals in the competitive landscape," Journal of World Business, Elsevier, vol. 51(4), pages 487-498.
    15. Chen, Kun & Ren, Xian-tong & Yang, Guo-liang, 2021. "A novel approach for assessing academic journals: Application of integer DEA model for management science and operations research field," Journal of Informetrics, Elsevier, vol. 15(3).
    16. Campanario, Juan Miguel, 2014. "Analysis of the distribution of cited journals according to their positions in the h-core of citing journal listed in Journal Citation Reports," Journal of Informetrics, Elsevier, vol. 8(3), pages 534-545.
    17. Antonio García-Romero & Daniel Santín & Gabriela Sicilia, 2016. "Another brick in the wall: a new ranking of academic journals in Economics using FDH," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 91-101, April.
    18. Laura Vana & Ronald Hochreiter & Kurt Hornik, 2016. "Computing a journal meta-ranking using paired comparisons and adaptive lasso estimators," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 229-251, January.
    19. Sahoo, Biresh K. & Singh, Ramadhar & Mishra, Bineet & Sankaran, Krithiga, 2017. "Research productivity in management schools of India during 1968-2015: A directional benefit-of-doubt model analysis," Omega, Elsevier, vol. 66(PA), pages 118-139.
    20. Tüselmann, Heinz & Sinkovics, Rudolf R. & Pishchulov, Grigory, 2015. "Towards a consolidation of worldwide journal rankings – A classification using random forests and aggregate rating via data envelopment analysis," Omega, Elsevier, vol. 51(C), pages 11-23.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    2. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    3. Halkos, George & Tzeremes, Nickolaos, 2011. "A non-parametric analysis of the efficiency of the top European football clubs," MPRA Paper 31173, University Library of Munich, Germany.
    4. Halkos, George & Tzeremes, Nickolaos, 2012. "Ranking agricultural, environmental and natural resource economics journals: A note," MPRA Paper 36233, University Library of Munich, Germany.
    5. Chagas , André Luis Squarize, 2017. "Publish or Perish: um ranking de revistas da subárea de Economia Regional e Urbana para os pesquisadores brasileiros," Revista Brasileira de Estudos Regionais e Urbanos, Associação Brasileira de Estudos Regionais e Urbanos (ABER), vol. 11(4), pages 515-536.
    6. Ham, John C. & Wright, Julian & Ye, Ziqiu, 2023. "Documenting and Explaining the Dramatic Rise of the New Society Journals in Economics," IZA Discussion Papers 16337, Institute of Labor Economics (IZA).
    7. Chan, Ho Fai & Frey, Bruno S. & Gallus, Jana & Torgler, Benno, 2014. "Academic honors and performance," Labour Economics, Elsevier, vol. 31(C), pages 188-204.
    8. Halkos, George & Tzeremes, Nickolaos, 2011. "A nonparametric analysis of the Greek renewable energy sector," MPRA Paper 30467, University Library of Munich, Germany.
    9. Petridis, Konstantinos & Malesios, Chrisovalantis & Arabatzis, Garyfallos & Thanassoulis, Emmanuel, 2013. "Efficiency analysis of forestry journals: Suggestions for improving journals’ quality," Journal of Informetrics, Elsevier, vol. 7(2), pages 505-521.
    10. Halkos, George & Tzeremes, Nickolaos, 2011. "Applying conditional DEA to measure football clubs’ performance: Evidence from the top 25 European clubs," MPRA Paper 31278, University Library of Munich, Germany.
    11. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    12. Ho F. Chan & Franklin G. Mixon & Benno Torgler, 2018. "Relation of early career performance and recognition to the probability of winning the Nobel Prize in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1069-1086, March.
    13. Yu, Xiaohua & Gao, Zhifeng, 2010. "An updated ranking of the economic research institutions in China (2000-2009)," China Economic Review, Elsevier, vol. 21(4), pages 571-581, December.
    14. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2011. "An updated ranking of academic journals in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 44(4), pages 1525-1538, November.
    15. Eduardo A. Haddad & Jesus P. Mena-Chalco, Otávio J.G. Sidone, 2016. "Produção Científica e Redes de Colaboração dos Docentes Vinculados aos Programas de Pós-graduação em Economia no Brasil," Working Papers, Department of Economics 2016_10, University of São Paulo (FEA-USP).
    16. Halkos, George & Tzeremes, Nickolaos, 2012. "Evaluating professional tennis players’ career performance: A Data Envelopment Analysis approach," MPRA Paper 41516, University Library of Munich, Germany.
    17. Halkos, George & Tzeremes, Nickolaos, 2011. "The effect of national culture on countries’ innovation efficiency," MPRA Paper 30100, University Library of Munich, Germany.
    18. Halkos, George & Tzeremes, Nickolaos, 2011. "A conditional full frontier approach for investigating the Averch-Johnson effect," MPRA Paper 35491, University Library of Munich, Germany.
    19. Sebastian Kohl & Jan Schoenfelder & Andreas Fügener & Jens O. Brunner, 2019. "The use of Data Envelopment Analysis (DEA) in healthcare with a focus on hospitals," Health Care Management Science, Springer, vol. 22(2), pages 245-286, June.
    20. Halkos, George & Tzeremes, Nickolaos, 2010. "Measuring the effect of virtual mergers on banks’ efficiency levels:A non parametric analysis," MPRA Paper 23696, University Library of Munich, Germany.

    More about this item

    Keywords

    Ranking journals; Economic journals; Data envelopment analysis; Indexing techniques;
    All these keywords.

    JEL classification:

    • C02 - Mathematical and Quantitative Methods - - General - - - Mathematical Economics
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • C67 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Input-Output Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:88:y:2011:i:3:d:10.1007_s11192-011-0421-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.