IDEAS home Printed from https://ideas.repec.org/p/pui/dpaper/229.html
   My bibliography  Save this paper

Experimenting with Generative AI: Does ChatGPT Really Increase Everyone’s Productivity?

Author

Listed:
  • Voraprapa Nakavachara
  • Tanapong Potipiti
  • Thanee Chaiwat

Abstract

Generative AI technologies such as ChatGPT, Gemini, and MidJourney have made remarkable progress in recent years. Recent literature has documented ChatGPT’s positive impact on productivity in areas where it has strong expertise—attributable to extensive training datasets—such as the English language and Python/SQL programming. However, the literature is still limited regarding ChatGPT’s performance in areas where its capabilities could still be further enhanced. In this paper, we asked participants to perform writing analysis tasks in a non-English language (specifically, Thai) and math & data analysis tasks using a less frequently used programming package (specifically Stata). The findings suggest that, on average, participants performed better using ChatGPT in terms of scores and time taken to complete the tasks. However, a detailed examination reveals that 34% of participants saw no improvement in writing analysis tasks, and 42% did not improve in math & data analysis tasks when employing ChatGPT. Further investigation indicated that higher-ability participants, as proxied by their econometrics grades, were the ones who performed worse in writing analysis tasks when using ChatGPT. We also found evidence that participants with better digital skills performed better with ChatGPT. This research provides insights on the impact of generative AI. Thus, relevant parties can make informed decisions regarding appropriate strategies, policies, and educational systems. It also highlights the critical role of human skills in addressing and complementing the limitations of AI.

Suggested Citation

  • Voraprapa Nakavachara & Tanapong Potipiti & Thanee Chaiwat, 2025. "Experimenting with Generative AI: Does ChatGPT Really Increase Everyone’s Productivity?," PIER Discussion Papers 229, Puey Ungphakorn Institute for Economic Research.
    • Handle: RePEc:pui:dpaper:229
    as

    Download full text from publisher

    File URL: https://www.pier.or.th/files/dp/pier_dp_229.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alex Kim & Maximilian Muhn & Valeri Nikolaev, 2023. "From Transcripts to Insights: Uncovering Corporate Risks Using Generative AI," Papers 2310.17721, arXiv.org.
    2. Duk Gyoo Kim & Ahram Moon, 2024. "From Helping Hand to Stumbling Block: The ChatGPT Paradox in Competency Experiment," CESifo Working Paper Series 11002, CESifo.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Voraprapa Nakavachara & Tanapong Potipiti & Thanee Chaiwat, 2024. "Experimenting with Generative AI: Does ChatGPT Really Increase Everyone's Productivity?," Papers 2403.01770, arXiv.org.
    2. Edward Li & Zhiyuan Tu & Dexin Zhou, 2024. "The Promise and Peril of Generative AI: Evidence from GPT-4 as Sell-Side Analysts," Papers 2412.01069, arXiv.org.
    3. Yikai Zhao & Jun Nagayasu & Xinyi Geng, 2024. "Measuring Climate Policy Uncertainty with LLMs: New Insights into Corporate Bond Credit Spreads," DSSR Discussion Papers 143, Graduate School of Economics and Management, Tohoku University.

    More about this item

    Keywords

    ChatGPT; Generative AI; Large Language Models; Labor Productivity;
    All these keywords.

    JEL classification:

    • A20 - General Economics and Teaching - - Economic Education and Teaching of Economics - - - General
    • D24 - Microeconomics - - Production and Organizations - - - Production; Cost; Capital; Capital, Total Factor, and Multifactor Productivity; Capacity
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pui:dpaper:229. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/pierbth.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.