IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp17302.html
   My bibliography  Save this paper

Creative and Strategic Capabilities of Generative AI: Evidence from Large-Scale Experiments

Author

Listed:
  • Bohren, Noah

    (University of Lausanne)

  • Hakimov, Rustamdjan

    (University of Lausanne)

  • Lalive, Rafael

    (University of Lausanne)

Abstract

Generative artificial intelligence (AI) has made substantial progress, but its full capabilities remain unclear, and we still lack a comprehensive understanding of how people augment productivity with AI and perceive AI-generated outputs. This study compares the ability of AI to a representative population of US adults in creative and strategic tasks. The creative ideas produced by AI chatbots are rated more creative than those created by humans. Moreover, ChatGPT is substantially more creative than humans, while Bard lags behind. Augmenting humans with AI improves human creativity, albeit not as much as ideas created by ChatGPT alone. Competition from AI does not significantly reduce the creativity of men, but it decreases the creativity of women. Humans who rate the text cannot discriminate well between ideas created by AI or other humans but assign lower scores to the responses they believe to be AI-generated. As for strategic capabilities, while ChatGPT shows a clear ability to adjust its moves in a strategic game to the play of the opponent, humans are, on average, more successful in this adaptation.

Suggested Citation

  • Bohren, Noah & Hakimov, Rustamdjan & Lalive, Rafael, 2024. "Creative and Strategic Capabilities of Generative AI: Evidence from Large-Scale Experiments," IZA Discussion Papers 17302, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp17302
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp17302.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Erik Brynjolfsson & Danielle Li & Lindsey Raymond, 2023. "Generative AI at Work," Papers 2304.11771, arXiv.org, revised Nov 2024.
    2. Berkeley J. Dietvorst & Joseph P. Simmons & Cade Massey, 2018. "Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them," Management Science, INFORMS, vol. 64(3), pages 1155-1170, March.
    3. Muriel Niederle & Lise Vesterlund, 2011. "Gender and Competition," Annual Review of Economics, Annual Reviews, vol. 3(1), pages 601-630, September.
    4. Dargnies, Marie-Pierre & Hakimov, Rustamdjan & Kübler, Dorothea, 2022. "Aversion to hiring algorithms: Transparency, gender profiling, and self-confidence," Discussion Papers, Research Unit: Market Behavior SP II 2022-202, WZB Berlin Social Science Center.
    5. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David B. Huffman & Uwe Sunde, 2017. "Global Evidence on Economic Preferences," NBER Working Papers 23943, National Bureau of Economic Research, Inc.
    6. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David Huffman & Uwe Sunde, 2018. "Global Evidence on Economic Preferences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1645-1692.
    7. Melanie Arntz & Terry Gregory & Ulrich Zierahn, 2016. "The Risk of Automation for Jobs in OECD Countries: A Comparative Analysis," OECD Social, Employment and Migration Working Papers 189, OECD Publishing.
    8. Greiner, Ben & Grünwald, Philipp & Lindner, Thomas & Lintner, Georg & Wiernsperger, Martin, 2024. "Incentives, Framing, and Reliance on Algorithmic Advice: An Experimental Study," Department for Strategy and Innovation Working Paper Series 01/2024, WU Vienna University of Economics and Business.
    9. David H. Autor, 2015. "Why Are There Still So Many Jobs? The History and Future of Workplace Automation," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 3-30, Summer.
    10. Gary Charness & Daniela Grieco, 2019. "Creativity and Incentives," Journal of the European Economic Association, European Economic Association, vol. 17(2), pages 454-496.
    11. Ed Felten & Manav Raj & Robert Seamans, 2023. "How will Language Modelers like ChatGPT Affect Occupations and Industries?," Papers 2303.01157, arXiv.org, revised Mar 2023.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carvajal, Daniel & Franco, Catalina & Isaksson, Siri, 2024. "Will Artificial Intelligence Get in the Way of Achieving Gender Equality?," Discussion Paper Series in Economics 3/2024, Norwegian School of Economics, Department of Economics, revised 31 Oct 2024.
    2. Innocenti, Stefania & Golin, Marta, 2022. "Human capital investment and perceived automation risks: Evidence from 16 countries," Journal of Economic Behavior & Organization, Elsevier, vol. 195(C), pages 27-41.
    3. Alain Cohn & Tobias Gesche & Michel André Maréchal, 2022. "Honesty in the Digital Age," Management Science, INFORMS, vol. 68(2), pages 827-845, February.
    4. Eber, Nicolas & François, Abel & Weill, Laurent, 2021. "Gender, age, and attitude toward competition," Journal of Economic Behavior & Organization, Elsevier, vol. 192(C), pages 668-690.
    5. Brandts, Jordi & El Baroudi, Sabrine & Huber, Stefanie J. & Rott, Christina, 2021. "Gender differences in private and public goal setting," Journal of Economic Behavior & Organization, Elsevier, vol. 192(C), pages 222-247.
    6. Kevin Bauer & Andrej Gill, 2024. "Mirror, Mirror on the Wall: Algorithmic Assessments, Transparency, and Self-Fulfilling Prophecies," Information Systems Research, INFORMS, vol. 35(1), pages 226-248, March.
    7. Kinnl, Klara & Möller, Jakob & Walter, Anna, 2023. "Borrowed Plumes:," Department of Economics Working Paper Series 345, WU Vienna University of Economics and Business.
    8. D’Exelle, Ben & Gutekunst, Christine & Riedl, Arno, 2023. "The effect of gender and gender pairing on bargaining: Evidence from an artefactual field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 205(C), pages 237-269.
    9. Raphael Auer & David Köpfer & Josef Švéda & Raphael A. Auer, 2024. "The Rise of Generative AI: Modelling Exposure, Substitution, and Inequality Effects on the US Labour Market," CESifo Working Paper Series 11410, CESifo.
    10. Kuhn, Andreas & Wolter, Stefan C., 2022. "Things versus People: Gender Differences in Vocational Interests and in Occupational Preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 203(C), pages 210-234.
    11. Klara Kinnl & Jakob Möller & Anna Walter, 2023. "The Gender Gap in Claiming Credit for Teamwork," Department of Economics Working Papers wuwp345, Vienna University of Economics and Business, Department of Economics.
    12. Ulf Rinne & Hendrik Sonnabend, 2022. "Female workers, male managers: Gender, leadership, and risk‐taking," Southern Economic Journal, John Wiley & Sons, vol. 88(3), pages 906-930, January.
    13. Horn, Dániel & Kiss, Hubert Janos & Lénárd, Tünde, 2020. "Economic preferences in the classroom - research documentation," MPRA Paper 100815, University Library of Munich, Germany.
    14. Kinnl, Klara & Möller, Jakob & Walter, Anna, 2023. "Borrowed Plumes: The Gender Gap in Claiming Credit for Teamwork," Department for Strategy and Innovation Working Paper Series 01/2023, WU Vienna University of Economics and Business.
    15. Pablo Casas & Concepción Román, 2024. "The impact of artificial intelligence in the early retirement decision," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 51(3), pages 583-618, August.
    16. Gorny, Paul M. & Groos, Eva & Strobel, Christina, 2024. "Do Personalized AI Predictions Change Subsequent Decision-Outcomes? The Impact of Human Oversight," MPRA Paper 121065, University Library of Munich, Germany.
    17. Martin Holmén & Felix Holzmeister & Michael Kirchler & Matthias Stefan & Erik Wengström, 2023. "Economic Preferences and Personality Traits Among Finance Professionals and the General Population," The Economic Journal, Royal Economic Society, vol. 133(656), pages 2949-2977.
    18. Daniel Horn & Hubert Kiss Janos & Sára Khayouti, 2020. "Does trust associate with political regime?," CERS-IE WORKING PAPERS 2013, Institute of Economics, Centre for Economic and Regional Studies.
    19. Lütkenhorst, Wilfried, 2018. "Creating wealth without labour? Emerging contours of a new techno-economic landscape," IDOS Discussion Papers 11/2018, German Institute of Development and Sustainability (IDOS).
    20. Gutmann, Jerg & Metelska-Szaniawska, Katarzyna & Voigt, Stefan, 2024. "Leader characteristics and constitutional compliance," European Journal of Political Economy, Elsevier, vol. 84(C).

    More about this item

    Keywords

    creativity; ChatGPT; artificial intelligence; strategic skill; experiment; algorithm-aversion;
    All these keywords.

    JEL classification:

    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp17302. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.