IDEAS home Printed from https://ideas.repec.org/a/zbw/espost/307995.html
   My bibliography  Save this article

Human intelligence versus artificial intelligence in classifying economics research articles: exploratory evidence

Author

Listed:
  • Heikkilä, Jussi T. S.

Abstract

Purpose We compare human intelligence to artificial intelligence (AI) in the choice of appropriate Journal of Economic Literature (JEL) codes for research papers in economics. Design/methodology/approach We compare the JEL code choices related to articles published in the recent issues of the Journal of Economic Literature and the American Economic Review and compare these to the original JEL code choices of the authors in earlier working paper versions and JEL codes recommended by various generative AI systems (OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini) based on the abstracts of the articles. Findings There are significant discrepancies and often limited overlap between authors’ choices of JEL codes, editors’ choices as well as the choices by contemporary widely used AI systems. However, the observations suggest that generative AI can augment human intelligence in the micro-task of choosing the JEL codes and, thus, save researchers time. Research limitations/implications Rapid development of AI systems makes the findings quickly obsolete. Practical implications AI systems may economize on classification costs and (semi-)automate the choice of JEL codes by recommending the most appropriate ones. Future studies may apply the presented approach to analyze whether the JEL code choices between authors, editors and AI systems converge and become more consistent as humans increasingly interact with AI systems. Originality/value We assume that the choice of JEL codes is a micro-task in which boundedly rational decision-makers rather satisfice than optimize. This exploratory experiment is among the first to compare human intelligence and generative AI in choosing and justifying the choice of optimal

Suggested Citation

  • Heikkilä, Jussi T. S., 2024. "Human intelligence versus artificial intelligence in classifying economics research articles: exploratory evidence," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 81(7), pages 18-30.
    • Handle: RePEc:zbw:espost:307995
    DOI: 10.1108/JD-05-2024-0104
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/307995/1/Heikkilae-Human-intelligence.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.1108/JD-05-2024-0104?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Joshua Angrist & Pierre Azoulay & Glenn Ellison & Ryan Hill & Susan Feng Lu, 2017. "Economic Research Evolves: Fields and Styles," American Economic Review, American Economic Association, vol. 107(5), pages 293-297, May.
    2. Beatrice Cherrier, 2017. "Classifying Economics: A History of the JEL Codes," Journal of Economic Literature, American Economic Association, vol. 55(2), pages 545-579, June.
    3. Herbert A. Simon, 1955. "A Behavioral Model of Rational Choice," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 69(1), pages 99-118.
    4. Anton Korinek, 2023. "Generative AI for Economic Research: Use Cases and Implications for Economists," Journal of Economic Literature, American Economic Association, vol. 61(4), pages 1281-1317, December.
    5. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    6. Ilia Shumailov & Zakhar Shumaylov & Yiren Zhao & Nicolas Papernot & Ross Anderson & Yarin Gal, 2024. "AI models collapse when trained on recursively generated data," Nature, Nature, vol. 631(8022), pages 755-759, July.
    7. Bartosz Maćkowiak & Filip Matějka & Mirko Wiederholt, 2023. "Rational Inattention: A Review," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 226-273, March.
    8. Lea†Rachel Kosnik, 2018. "A Survey Of Jel Codes: What Do They Mean And Are They Used Consistently?," Journal of Economic Surveys, Wiley Blackwell, vol. 32(1), pages 249-272, February.
    9. Alexandra Baumann & Klaus Wohlrabe, 2020. "Where have all the working papers gone? Evidence from four major economics working paper series," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2433-2441, September.
    10. Lutz Bornmann & Klaus Wohlrabe, 2024. "Recent Temporal Dynamics in Economics: Empirical Analyses of Annual Publications in Economic Fields," CESifo Working Paper Series 10881, CESifo.
    11. Lu, Wei & Liu, Zhifeng & Huang, Yong & Bu, Yi & Li, Xin & Cheng, Qikai, 2020. "How do authors select keywords? A preliminary study of author keyword selection behavior," Journal of Informetrics, Elsevier, vol. 14(4).
    12. Florian M. Artinger & Gerd Gigerenzer & Perke Jacobs, 2022. "Satisficing: Integrating Two Traditions," Journal of Economic Literature, American Economic Association, vol. 60(2), pages 598-635, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lutz Bornmann & Klaus Wohlrabe, 2019. "Normalisation of citation impact in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 841-884, August.
    2. Ali Sina Önder & Sergey V. Popov & Sascha Schweitzer, 2021. "Leadership in Scholarship: Editors’ Appointments and the Profession’s Narrative," Working Papers in Economics & Finance 2021-05, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    3. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    4. Ali Sina Önder & Sascha Schweitzer & Hakan Yilmazkuday, 2021. "Field Distance and Quality in Economists’ Collaborations," Working Papers in Economics & Finance 2021-04, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    5. Lorenz Gschwent & Bjorn Hammarfelt & Martin Karlsson & Mathias Kifmann, 2024. "The Rise of Health Economics: Transforming the Landscape of Economic Research," Papers 2410.06313, arXiv.org.
    6. Önder, Ali Sina & Schweitzer, Sascha & Yilmazkuday, Hakan, 2021. "Specialization, field distance, and quality in economists’ collaborations," Journal of Informetrics, Elsevier, vol. 15(4).
    7. Corbet, Shaen & Dowling, Michael & Gao, Xiangyun & Huang, Shupei & Lucey, Brian & Vigne, Samuel A., 2019. "An analysis of the intellectual structure of research on the financial economics of precious metals," Resources Policy, Elsevier, vol. 63(C), pages 1-1.
    8. Spiegel, Yossi & Toivanen, Otto, 2022. "From conference submission to publication and citations: Evidence from the EARIE conference," International Journal of Industrial Organization, Elsevier, vol. 84(C).
    9. Prashant Garg & Thiemo Fetzer, 2025. "Causal Claims in Economics," Papers 2501.06873, arXiv.org.
    10. Andrei Dubovik & Clemens Fiedler & Alexei Parakhonyak, 2022. "Temporal Patterns in Economics Research," CPB Discussion Paper 440, CPB Netherlands Bureau for Economic Policy Analysis.
    11. Onder, Ali Sina & Popov, Sergey V & Schweitzer, Sascha, 2018. "Leadership in Scholarship: Editors' Influence on the Profession's Narrative," Cardiff Economics Working Papers E2018/2, Cardiff University, Cardiff Business School, Economics Section.
    12. Jaque Herrera, Gabriela & Cárdenas-Retamal, Roberto & Barrales Henriquez, Daniel, 2022. "Tendencias en Publicaciones en Revistas Chilenas de Economía," Documentos de Trabajo 12, Estudios Nueva Economía.
    13. Jussi T. S. Heikkila, 2020. "Classifying economics for the common good: Connecting sustainable development goals to JEL codes," Papers 2004.04384, arXiv.org.
    14. Paldam Martin, 2021. "Methods Used in Economic Research: An Empirical Study of Trends and Levels," Economics - The Open-Access, Open-Assessment Journal, De Gruyter, vol. 15(1), pages 28-42, January.
    15. Constantin Bürgi & Klaus Wohlrabe, 2022. "The influence of Covid-19 on publications in economics: bibliometric evidence from five working paper series," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5175-5189, September.
    16. Peter Andre & Armin Falk, 2021. "What’s Worth Knowing? Economists’ Opinions about Economics," ECONtribute Discussion Papers Series 102, University of Bonn and University of Cologne, Germany.
    17. Matthias Aistleitner & Stephan Puehringer, 2020. "Exploring the trade (policy) narratives in economic elite discourse," ICAE Working Papers 110, Johannes Kepler University, Institute for Comprehensive Analysis of the Economy.
    18. Alexander Gutfraind, 2023. "Risk-reducing design and operations toolkit: 90 strategies for managing risk and uncertainty in decision problems," Papers 2309.03133, arXiv.org.
    19. Heinke, Steve, 2024. "Top–down and bottom–up information acquisition: Application to financial markets," Economics Letters, Elsevier, vol. 237(C).
    20. Jakob Kapeller & Stephan Puehringer & Christian Grimm, 2022. "Paradigms and policies: the state of economics in the German-speaking countries," Review of International Political Economy, Taylor & Francis Journals, vol. 29(4), pages 1183-1210, July.

    More about this item

    Keywords

    Artificial intelligence; Large language models; Search costs; Bounded rationality;
    All these keywords.

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • C88 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other Computer Software
    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:espost:307995. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.