IDEAS home Printed from https://ideas.repec.org/p/bis/biswps/1208.html
   My bibliography  Save this paper

Generative AI and labour productivity: a field experiment on coding

Author

Listed:
  • Leonardo Gambacorta
  • Han Qiu
  • Shuo Shan
  • Daniel M Rees

Abstract

In this paper we examine the effects of generative artificial intelligence (gen AI) on labour productivity. In September 2023, Ant Group introduced CodeFuse, a large language model (LLM) designed to assist programmer teams with coding. While one group of programmers used it, other programmer teams were not informed about this LLM. Leveraging this event, we conducted a field experiment on these two groups of programmers. We identified employees who used CodeFuse as the treatment group and paired them with comparable employees in the control group, to assess the impact of AI on their productivity. Our findings indicate that the use of gen AI increased code output by more than 50%. However, productivity gains are statistically significant only among entry-level or junior staff, while the impact on more senior employees is less pronounced.

Suggested Citation

  • Leonardo Gambacorta & Han Qiu & Shuo Shan & Daniel M Rees, 2024. "Generative AI and labour productivity: a field experiment on coding," BIS Working Papers 1208, Bank for International Settlements.
    • Handle: RePEc:bis:biswps:1208
    as

    Download full text from publisher

    File URL: https://www.bis.org/publ/work1208.pdf
    File Function: Full PDF document
    Download Restriction: no
    File URL: https://www.bis.org/publ/work1208.htm
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Erik Brynjolfsson & Danielle Li & Lindsey Raymond, 2023. "Generative AI at Work," Papers 2304.11771, arXiv.org, revised Nov 2024.
    2. Zied Bahroun & Chiraz Anane & Vian Ahmed & Andrew Zacca, 2023. "Transforming Education: A Comprehensive Review of Generative Artificial Intelligence in Educational Settings through Bibliometric and Content Analysis," Sustainability, MDPI, vol. 15(17), pages 1-40, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carvajal, Daniel & Franco, Catalina & Isaksson, Siri, 2024. "Will Artificial Intelligence Get in the Way of Achieving Gender Equality?," Discussion Paper Series in Economics 3/2024, Norwegian School of Economics, Department of Economics, revised 31 Oct 2024.
    2. Gaétan de Rassenfosse & Adam B. Jaffe & Joel Waldfogel, 2024. "Intellectual Property and Creative Machines," NBER Chapters, in: Entrepreneurship and Innovation Policy and the Economy, volume 4, National Bureau of Economic Research, Inc.
    3. Alexander Cuntz & Carsten Fink & Hansueli Stamm, 2024. "Artificial Intelligence and Intellectual Property : An Economic Perspective," WIPO Economic Research Working Papers 77, World Intellectual Property Organization - Economics and Statistics Division.
    4. Lilia Patrignani, 2024. "Understanding digital trade," Questioni di Economia e Finanza (Occasional Papers) 841, Bank of Italy, Economic Research and International Relations Area.
    5. Amali Matharaarachchi & Wishmitha Mendis & Kanishka Randunu & Daswin De Silva & Gihan Gamage & Harsha Moraliyage & Nishan Mills & Andrew Jennings, 2024. "Optimizing Generative AI Chatbots for Net-Zero Emissions Energy Internet-of-Things Infrastructure," Energies, MDPI, vol. 17(8), pages 1-19, April.
    6. Bohren, Noah & Hakimov, Rustamdjan & Lalive, Rafael, 2024. "Creative and Strategic Capabilities of Generative AI: Evidence from Large-Scale Experiments," IZA Discussion Papers 17302, Institute of Labor Economics (IZA).
    7. Becker, Dominik & Deck, Luca & Feulner, Simon & Gutheil, Niklas & Schüll, Moritz & Decker, Stefan & Eymann, Torsten & Gimpel, Henner & Pippow, Andreas & Röglinger, Maximilian & Urbach, Nils, 2024. "Lohnt sich Microsoft 365 Copilot? Eine Potenzialanalyse für Unternehmen und Bildungseinrichtungen," Bayreuth Reports on Information Systems Management 72, University of Bayreuth, Chair of Information Systems Management.
    8. Christian Peukert & Florian Abeillon & Jérémie Haese & Franziska Kaiser & Alexander Staub, 2024. "Strategic Behavior and AI Training Data," CESifo Working Paper Series 11099, CESifo.
    9. Stankov, Petar, 2024. "The death of exams? Grade inflation and student satisfaction when coursework replaces exams," International Review of Economics Education, Elsevier, vol. 46(C).
    10. Iñaki Aldasoro & Leonardo Gambacorta & Anton Korinek & Vatsala Shreeti & Merlin Stein, 2024. "Intelligent financial system: how AI is transforming finance," BIS Working Papers 1194, Bank for International Settlements.
    11. Samuel Chang & Andrew Kennedy & Aaron Leonard & John A. List, 2024. "12 Best Practices for Leveraging Generative AI in Experimental Research," NBER Working Papers 33025, National Bureau of Economic Research, Inc.
    12. James Bono & Alec Xu, 2024. "Randomized Controlled Trials for Security Copilot for IT Administrators," Papers 2411.01067, arXiv.org, revised Nov 2024.
    13. Nikolaos Charalampidis, 2024. "Frictions and the diffusion of automation," Manchester School, University of Manchester, vol. 92(2), pages 148-170, March.
    14. Morgan Blangeois, 2023. "Generative AI: Revolution or Threat for Digital Service Companies ? [IA générative : révolution ou menace pour les entreprises de services du numérique ?]," Post-Print hal-04355219, HAL.
    15. Spencer Bastani & Daniel Waldenström, 2024. "AI, Automation and Taxation," CESifo Working Paper Series 11084, CESifo.
    16. Raphael Auer & David Köpfer & Josef Švéda & Raphael A. Auer, 2024. "The Rise of Generative AI: Modelling Exposure, Substitution, and Inequality Effects on the US Labour Market," CESifo Working Paper Series 11410, CESifo.
    17. Ozge Demirci & Jonas Hannane & Xinrong Zhu, 2024. "Who Is AI Replacing? The Impact of Generative AI on Online Freelancing Platforms," CESifo Working Paper Series 11276, CESifo.
    18. Mahdi Ahmadi & Neda Khosh Kheslat & Adebola Akintomide, 2024. "Generative AI Impact on Labor Market: Analyzing ChatGPT's Demand in Job Advertisements," Papers 2412.07042, arXiv.org.
    19. Aldasoro, Iñaki & Armantier, Olivier & Doerr, Sebastian & Gambacorta, Leonardo & Oliviero, Tommaso, 2024. "The gen AI gender gap," Economics Letters, Elsevier, vol. 241(C).
    20. Bastani, Spencer & Waldenström, Daniel, 2024. "AI, Automation and Taxation," IZA Policy Papers 212, Institute of Labor Economics (IZA).

    More about this item

    Keywords

    artificial intelligence; productivity; field experiment; big tech;
    All these keywords.

    JEL classification:

    • D22 - Microeconomics - - Production and Organizations - - - Firm Behavior: Empirical Analysis
    • G31 - Financial Economics - - Corporate Finance and Governance - - - Capital Budgeting; Fixed Investment and Inventory Studies
    • R30 - Urban, Rural, Regional, Real Estate, and Transportation Economics - - Real Estate Markets, Spatial Production Analysis, and Firm Location - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bis:biswps:1208. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Martin Fessler (email available below). General contact details of provider: https://edirc.repec.org/data/bisssch.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.