IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_11002.html
   My bibliography  Save this paper

From Helping Hand to Stumbling Block: The ChatGPT Paradox in Competency Experiment

Author

Listed:
  • Duk Gyoo Kim
  • Ahram Moon

Abstract

We ran a controlled laboratory experiment to examine whether ChatGPT’s aid can increase the participants’ performance in three different—reading and writing, mathematical problem-solving, and computational thinking—tasks. We find that the math score significantly decreases with ChatGPT’s assistance. This result is mainly because the low-ability subjects couldn’t discern the hallucinated answers with the correct ones, and it contests the general idea that ChatGPT can complement the workers with less expertise.

Suggested Citation

  • Duk Gyoo Kim & Ahram Moon, 2024. "From Helping Hand to Stumbling Block: The ChatGPT Paradox in Competency Experiment," CESifo Working Paper Series 11002, CESifo.
    • Handle: RePEc:ces:ceswps:_11002
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp11002.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Maarten Goos & Alan Manning & Anna Salomons, 2014. "Explaining Job Polarization: Routine-Biased Technological Change and Offshoring," American Economic Review, American Economic Association, vol. 104(8), pages 2509-2526, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tommaso AGASISTI & Geraint JOHNES & Marco PACCAGNELLA, 2021. "Tasks, occupations and wages in OECD countries," International Labour Review, International Labour Organization, vol. 160(1), pages 85-112, March.
    2. Aleksandra Parteka & Joanna Wolszczak-Derlacz, 2020. "Wage response to global production links: evidence for workers from 28 European countries (2005–2014)," Review of World Economics (Weltwirtschaftliches Archiv), Springer;Institut für Weltwirtschaft (Kiel Institute for the World Economy), vol. 156(4), pages 769-801, November.
    3. Battisti, Michele & Gatto, Massimo Del & Parmeter, Christopher F., 2022. "Skill-biased technical change and labor market inefficiency," Journal of Economic Dynamics and Control, Elsevier, vol. 139(C).
    4. Antje Mertens & Laura Romeu-Gordo, 2023. "Retirement in Western Germany – How Workplace Tasks Influence Its Timing," Work, Employment & Society, British Sociological Association, vol. 37(2), pages 467-485, April.
    5. Werner Eichhorst & Ulf Rinne, 2017. "Digital Challenges for the Welfare State," CESifo Forum, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 18(04), pages 03-08, December.
    6. Lütkenhorst, Wilfried, 2018. "Creating wealth without labour? Emerging contours of a new techno-economic landscape," IDOS Discussion Papers 11/2018, German Institute of Development and Sustainability (IDOS).
    7. Parolin, Zachary & Schmitt, Rafael Pintro & Esping-Andersen, Gøsta & Fallesen, Peter, 2023. "The Intergenerational Persistence of Poverty in High-Income Countries," IZA Discussion Papers 16194, Institute of Labor Economics (IZA).
    8. Zhou, Yixiao & Tyers, Rod, 2019. "Automation and inequality in China," China Economic Review, Elsevier, vol. 58(C).
    9. Thiemo Fetzer & Stephan Kyburz, 2024. "Cohesive Institutions and Political Violence," The Review of Economics and Statistics, MIT Press, vol. 106(1), pages 133-150, January.
    10. T. Gries & R. Grundmann & I. Palnau & M. Redlin, 2017. "Innovations, growth and participation in advanced economies - a review of major concepts and findings," International Economics and Economic Policy, Springer, vol. 14(2), pages 293-351, April.
    11. Silvia Vannutelli & Sergio Scicchitano & Marco Biagetti, 2022. "Routine-biased technological change and wage inequality: do workers’ perceptions matter?," Eurasian Business Review, Springer;Eurasia Business and Economics Society, vol. 12(3), pages 409-450, September.
    12. Sascha O. Becker & Thiemo Fetzer, 2018. "Has Eastern European Migration Impacted UK-born Workers?," CAGE Online Working Paper Series 376, Competitive Advantage in the Global Economy (CAGE).
    13. Rossella Bozzon & Annalisa Murgia, 2022. "Independent or Dependent? European Labour Statistics and Their (In)ability to Identify Forms of Dependency in Self-employment," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 160(1), pages 199-226, February.
    14. Georg Graetz & Guy Michaels, 2017. "Is Modern Technology Responsible for Jobless Recoveries?," American Economic Review, American Economic Association, vol. 107(5), pages 168-173, May.
    15. Gordon H. Hanson, 2021. "Immigration and Regional Specialization in AI," NBER Working Papers 28671, National Bureau of Economic Research, Inc.
    16. Taiji Furusawa & Hideo Konishi & Duong Lam Anh Tran, 2020. "International Trade and Income Inequality," Scandinavian Journal of Economics, Wiley Blackwell, vol. 122(3), pages 993-1026, July.
    17. Santana, Monica & Cobo, Manuel J., 2020. "What is the future of work? A science mapping analysis," European Management Journal, Elsevier, vol. 38(6), pages 846-862.
    18. Eichhorst, Werner, 2015. "Do We Have to Be Afraid of the Future World of Work?," IZA Policy Papers 102, Institute of Labor Economics (IZA).
    19. Dirk Antonczyk & Thomas DeLeire & Bernd Fitzenberger, 2018. "Polarization and Rising Wage Inequality: Comparing the U.S. and Germany," Econometrics, MDPI, vol. 6(2), pages 1-33, April.
    20. David J. Deming, 2017. "The Growing Importance of Social Skills in the Labor Market," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(4), pages 1593-1640.

    More about this item

    Keywords

    laboratory experiment; ChatGPT; labor productivity;
    All these keywords.

    JEL classification:

    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_11002. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.