IDEAS home Printed from https://ideas.repec.org/a/vrs/openec/v2y2019i1p1-18n1.html
   My bibliography  Save this article

Cite This Economics Paper! It Is Time for the House of Cards to Fall Down

Author

Listed:
  • Payson Steven

    (Freelance WriterMcLean)

Abstract

This paper takes a fresh look at citation counts and publications in top-rank journals, which the academic economics profession uses to evaluate and promote its members. It first examines how and why citations are mentioned in an article, and what this implies about their counts. The discussion then examines how average citation counts to articles are used to rank journals, and the paper reviews the concerns that have been expressed about this practice. These concerns identify the large variance in citation counts among articles of the same journal, implying that those articles themselves must vary greatly in quality (Engemann and Wall 2009). To address these concerns, the paper proposes the classification of citations into three categories: Fodder Citations (for references that contribute only trivially to a paper), Relevant Citations (which substantively contribute to the paper, though the paper would remain roughly the same without them), and Essential Citations (which have a major influence). The paper argues that counts of citations by the last two categories offers greater credibility in the application of citation counts to evaluate economic literature. Finally, the paper provides an opportunity for economists to participate in a new project that solicits information on citations by these categories.

Suggested Citation

  • Payson Steven, 2019. "Cite This Economics Paper! It Is Time for the House of Cards to Fall Down," Open Economics, De Gruyter, vol. 2(1), pages 1-18, January.
    • Handle: RePEc:vrs:openec:v:2:y:2019:i:1:p:1-18:n:1
    DOI: 10.1515/openec-2019-0001
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/openec-2019-0001
    Download Restriction: no
    File URL: https://libkey.io/10.1515/openec-2019-0001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. James J. Heckman & Sidharth Moktan, 2020. "Publishing and promotion in economics - The tyranny of the Top Five," Vox eBook Chapters, in: Sebastian Galliani & Ugo Panizza (ed.), Publishing and Measuring Success in Economics, edition 1, volume 1, chapter 1, pages 23-32, Centre for Economic Policy Research.
    2. Nattavudh Powdthavee & Yohanes E. Riyanto & Jack L. Knetsch, 2017. "Impact of Lower Rated Journals on Economists' Judgments of Publication Lists: Evidence from a Survey Experiment," PIER Discussion Papers 63, Puey Ungphakorn Institute for Economic Research.
    3. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    4. Christian Zimmermann, 2013. "Academic Rankings with RePEc," Econometrics, MDPI, vol. 1(3), pages 1-32, December.
    5. Tommaso Colussi, 2018. "Social Ties in Academia: A Friend Is a Treasure," The Review of Economics and Statistics, MIT Press, vol. 100(1), pages 45-50, March.
    6. David I. Stern, 2013. "Uncertainty Measures for Economics Journal Impact Factors," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 173-189, March.
    7. Johan Bollen & Herbert Van de Sompel & Aric Hagberg & Ryan Chute, 2009. "A Principal Component Analysis of 39 Scientific Impact Measures," PLOS ONE, Public Library of Science, vol. 4(6), pages 1-11, June.
    8. Colussi, Tommaso, 2015. "Social Ties in Academia: A Friend is a Treasure," IZA Discussion Papers 9414, Institute of Labor Economics (IZA).
    9. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    10. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    11. Laband, David N & Piette, Michael J, 1994. "The Relative Impacts of Economics Journals: 1970-1990," Journal of Economic Literature, American Economic Association, vol. 32(2), pages 640-666, June.
    12. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, vol. 22(1), pages 77-88, March.
    13. Moed, Henk F., 2010. "Measuring contextual citation impact of scientific journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 265-277.
    14. Powdthavee, Nattavudh & Riyanto, Yohanes E. & Knetsch, Jack L., 2018. "Lower-rated publications do lower academics’ judgments of publication lists: Evidence from a survey experiment of economists," Journal of Economic Psychology, Elsevier, vol. 66(C), pages 33-44.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Payson, Steven, 2019. "The Urgent Need for an Economics of “Hategoatism”," GLO Discussion Paper Series 365, Global Labor Organization (GLO).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matthias Aistleitner & Jakob Kapeller & Stefan Steinerberger, 2018. "Citation Patterns in Economics and Beyond," Working Papers Series 85, Institute for New Economic Thinking.
    2. María Victoria Anauati & Sebastian Galiani & Ramiro H. Gálvez, 2020. "Differences In Citation Patterns Across Journal Tiers: The Case Of Economics," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1217-1232, July.
    3. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    4. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).
    5. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    6. Carlo D'Ippoliti, 2021. "“Many‐Citedness”: Citations Measure More Than Just Scientific Quality," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1271-1301, December.
    7. Johannes König & David I. Stern & Richard S.J. Tol, 2022. "Confidence Intervals for Recursive Journal Impact Factors," Tinbergen Institute Discussion Papers 22-038/III, Tinbergen Institute.
    8. Lutz Bornmann & Klaus Wohlrabe, 2019. "Normalisation of citation impact in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 841-884, August.
    9. Andrew Hussey & Sheena Murray & Wendy Stock, 2022. "Gender, coauthorship, and academic outcomes in economics," Economic Inquiry, Western Economic Association International, vol. 60(2), pages 465-484, April.
    10. John Gibson & David L. Anderson & John Tressler, 2017. "Citations Or Journal Quality: Which Is Rewarded More In The Academic Labor Market?," Economic Inquiry, Western Economic Association International, vol. 55(4), pages 1945-1965, October.
    11. Erich Battistin & Marco Ovidi, 2022. "Rising Stars: Expert Reviews and Reputational Yardsticks in the Research Excellence Framework," Economica, London School of Economics and Political Science, vol. 89(356), pages 830-848, October.
    12. Simon Ek & Magnus Henrekson, 2019. "The Geography and Concentration of Authorship in the Top Five: Implications For European Economics," Scottish Journal of Political Economy, Scottish Economic Society, vol. 66(2), pages 215-245, May.
    13. Damien Besancenot & Abdelghani Maddi, 2019. "Should citations be weighted to assess the influence of an academic article?," Economics Bulletin, AccessEcon, vol. 435(1), pages 435-445.
    14. Joseph Gerald Hirschberg & Jeanette Ngaire Lye, 2020. "Grading Journals In Economics: The Abcs Of The Abdc," Journal of Economic Surveys, Wiley Blackwell, vol. 34(4), pages 876-921, September.
    15. María Victoria Anauati & Sebastian Galiani & Ramiro H. Gálvez, 2018. "Differences in citation patterns across journal tiers in economics," Documentos de Trabajo 16701, The Latin American and Caribbean Economic Association (LACEA).
    16. Püttmann, Vitus & Thomsen, Stephan L. & Trunzer, Johannes, 2020. "Zur Relevanz von Ausstattungsunterschieden für Forschungsleistungsvergleiche: Ein Diskussionsbeitrag für die Wirtschaftswissenschaften in Deutschland," Hannover Economic Papers (HEP) dp-679, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät, revised Mar 2021.
    17. J. Atsu Amegashie, 2020. "Citations And Incentives In Academic Contests," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1233-1244, July.
    18. Aistleitner, Matthias & Kapeller, Jakob & Kronberger, Dominik, 2023. "The authors of economics journals revisited: evidence from a large-scale replication of Hodgson and Rothman (1999)," Journal of Institutional Economics, Cambridge University Press, vol. 19(1), pages 86-101, February.
    19. Bethmann, Dirk & Bransch, Felix & Kvasnicka, Michael & Sadrieh, Abdolkarim, 2023. "Home Bias in Top Economics Journals," IZA Discussion Papers 15965, Institute of Labor Economics (IZA).
    20. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2021. "Editorial favoritism in the field of laboratory experimental economics (RM/20/014-revised-)," Research Memorandum 005, Maastricht University, Graduate School of Business and Economics (GSBE).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vrs:openec:v:2:y:2019:i:1:p:1-18:n:1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.