IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/hdjpk.html
   My bibliography  Save this paper

The Uneven Impact of Generative AI on Entrepreneurial Performance

Author

Listed:
  • Otis, Nicholas G.
  • Clarke, Rowan Philip
  • Delecourt, Solene
  • Holtz, David

    (University of California, Berkeley)

  • Koning, Rembrand

    (Harvard Business School)

Abstract

There is a growing belief that scalable and low-cost AI assistance can improve firm decision-making and economic performance. However, running a business involves a myriad of open-ended problems, making it hard to generalize from recent studies showing that generative AI improves performance on well-defined writing tasks. In our five-month field experiment with 640 Kenyan entrepreneurs, we assessed the impact of AI-generated advice on small business revenues and profits. Participants were randomly assigned to a control group that received a standard business guide or to a treatment group that received a GPT-4 powered AI business mentor via WhatsApp. While we find no average treatment effect, this is because the causal effect of generative AI access varied with the baseline business performance of the entrepreneur: high performers benefited by just over 20% from AI advice, whereas low performers did roughly 10% worse with AI assistance. Exploratory analysis of the WhatsApp interaction logs shows that both groups sought the AI mentor’s advice, but that low performers did worse because they sought help on much more challenging business tasks. These findings highlight how the tasks selected by firms and entrepreneurs for AI assistance fundamentally shape who will benefit from generative AI.

Suggested Citation

  • Otis, Nicholas G. & Clarke, Rowan Philip & Delecourt, Solene & Holtz, David & Koning, Rembrand, 2023. "The Uneven Impact of Generative AI on Entrepreneurial Performance," OSF Preprints hdjpk, Center for Open Science.
    • Handle: RePEc:osf:osfxxx:hdjpk
    DOI: 10.31219/osf.io/hdjpk
    as

    Download full text from publisher

    File URL: https://osf.io/download/658386417094e91347a177e5/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/hdjpk?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Erik Brynjolfsson & Daniel Rock & Chad Syverson, 2021. "The Productivity J-Curve: How Intangibles Complement General Purpose Technologies," American Economic Journal: Macroeconomics, American Economic Association, vol. 13(1), pages 333-372, January.
    2. Nicholas Bloom & John Van Reenen, 2007. "Measuring and Explaining Management Practices Across Firms and Countries," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(4), pages 1351-1408.
    3. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
    4. Jing Cai & Adam Szeidl, 2018. "Interfirm Relationships and Business Performance," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1229-1282.
    5. Edward P. Lazear, 2004. "Balanced Skills and Entrepreneurship," American Economic Review, American Economic Association, vol. 94(2), pages 208-211, May.
    6. Kevin A. Bryan & András Tilcsik & Brooklynn Zhu, 2017. "Which Entrepreneurs Are Coachable and Why?," American Economic Review, American Economic Association, vol. 107(5), pages 312-316, May.
    7. Arnaldo Camuffo & Alessandro Cordova & Alfonso Gambardella & Chiara Spina, 2020. "A Scientific Approach to Entrepreneurial Decision Making: Evidence from a Randomized Control Trial," Management Science, INFORMS, vol. 66(2), pages 564-586, February.
    8. David McKenzie & Christopher Woodruff, 2017. "Business Practices in Small Firms in Developing Countries," Management Science, INFORMS, vol. 63(9), pages 2967-2981, September.
    9. Fafchamps, Marcel & McKenzie, David & Quinn, Simon & Woodruff, Christopher, 2012. "Using PDA consistency checks to increase the precision of profits and sales measurement in panels," Journal of Development Economics, Elsevier, vol. 98(1), pages 51-57.
    10. Solène Delecourt & Anne Fitzpatrick, 2021. "Childcare Matters: Female Business Owners and the Baby-Profit Gap," Management Science, INFORMS, vol. 67(7), pages 4455-4474, July.
    11. Jeffrey R Kling & Jeffrey B Liebman & Lawrence F Katz, 2007. "Experimental Analysis of Neighborhood Effects," Econometrica, Econometric Society, vol. 75(1), pages 83-119, January.
    12. Tyna Eloundou & Sam Manning & Pamela Mishkin & Daniel Rock, 2023. "GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models," Papers 2303.10130, arXiv.org, revised Aug 2023.
    13. Wyatt Brooks & Kevin Donovan & Terence R. Johnson, 2018. "Mentors or Teachers? Microenterprise Training in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 10(4), pages 196-221, October.
    14. Anderson, Stephen J. & Lazicky, Christy & Zia, Bilal, 2021. "Measuring the unmeasured: Aggregating, anchoring, and adjusting to estimate small business performance," Journal of Development Economics, Elsevier, vol. 151(C).
    15. Eric Bartelsman & John Haltiwanger & Stefano Scarpetta, 2013. "Cross-Country Differences in Productivity: The Role of Allocation and Selection," American Economic Review, American Economic Association, vol. 103(1), pages 305-334, February.
    16. Stefan Dimitriadis & Rembrand Koning, 2022. "Social Skills Improve Business Performance: Evidence from a Randomized Control Trial with Entrepreneurs in Togo," Management Science, INFORMS, vol. 68(12), pages 8635-8657, December.
    17. Stephen J. Anderson & David McKenzie, 2022. "Improving Business Practices and the Boundary of the Entrepreneur: A Randomized Experiment Comparing Training, Consulting, Insourcing, and Outsourcing," Journal of Political Economy, University of Chicago Press, vol. 130(1), pages 157-209.
    18. Erik Brynjolfsson & Lorin M. Hitt, 2000. "Beyond Computation: Information Technology, Organizational Transformation and Business Performance," Journal of Economic Perspectives, American Economic Association, vol. 14(4), pages 23-48, Fall.
    19. Rema Hanna & Sendhil Mullainathan & Joshua Schwartzstein, 2014. "Learning Through Noticing: Theory and Evidence from a Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(3), pages 1311-1353.
    20. McKenzie, David, 2012. "Beyond baseline and follow-up: The case for more T in experiments," Journal of Development Economics, Elsevier, vol. 99(2), pages 210-221.
    21. Aaron Chatterji & Solène Delecourt & Sharique Hasan & Rembrand Koning, 2019. "When does advice impact startup performance?," Strategic Management Journal, Wiley Blackwell, vol. 40(3), pages 331-356, March.
    22. David Autor, 2014. "Polanyi's Paradox and the Shape of Employment Growth," NBER Working Papers 20485, National Bureau of Economic Research, Inc.
    23. Patricio S Dalton & Julius Rüschenpöhler & Burak Uras & Bilal Zia, 2021. "Curating Local Knowledge: Experimental Evidence from Small Retailers in Indonesia," Journal of the European Economic Association, European Economic Association, vol. 19(5), pages 2622-2657.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Otis, Nicholas G. & Cranney, Katelyn & Delecourt, Solene & Koning, Rembrand, 2024. "Global Evidence on Gender Gaps and Generative AI," OSF Preprints h6a7c, Center for Open Science.
    2. Jason P Davis & Jian Bai Li, 2024. "Early Adoption of Generative AI by Global Business Leaders: Insights from an INSEAD Alumni Survey," Papers 2404.04543, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Islam, Asadul & Lee, Wang-Sheng & Triyana, Margaret & Xia, Xing, 2023. "Improving Health and Safety in the Informal Sector: Evidence from a Randomized Trial in Bangladesh," IZA Discussion Papers 16150, Institute of Labor Economics (IZA).
    2. Gonzalez-Uribe, Juanita & Hmaddi, Ouafaa, 2022. "The multi-dimensional impacts of business accelerators: what does the research tell us?," LSE Research Online Documents on Economics 115461, London School of Economics and Political Science, LSE Library.
    3. Anwar Adem & Richard Kneller & Cher Li, 2023. "Information constraints and technology efficiency: Field experiments benchmarking firms website performance," Discussion Papers 2023-07, University of Nottingham, GEP.
    4. Rodríguez-Lesmes, Paul & Gutierrez, Luis H. & Urueña-Mejia, Juan Carlos & Ortiz, Andres & Medina Rojas, Ivan & Romero, Mauricio, 2023. "The role of local promoters in helping microentrepreneurs engage in digital business training. The case of Expertienda," Documentos de Trabajo 20902, Universidad del Rosario.
    5. Dalton, Patricio & Rüschenpöhler, Julius & Uras, Burak & Zia, Bilal, 2019. "Local Best Practices for Business Growth," Other publications TiSEM fc650e2f-88cf-4d75-8257-f, Tilburg University, School of Economics and Management.
    6. Bloom, Nicholas & Iacovone, Leonardo & Pereira-Lopez, Mariana & Van Reenen, John, 2022. "Management and Misallocation in Mexico," CEPR Discussion Papers 16979, C.E.P.R. Discussion Papers.
    7. Cettolin, Elena & Cole, Kym & Dalton, Patricio, 2022. "Improving Workers’ Performance in Small Firms : A Randomized Experiment on Goal Setting in Ghana," Discussion Paper 2022-028, Tilburg University, Center for Economic Research.
    8. Claudia Custodio & Diogo Mendes & Daniel Metzger, 2021. "The impact of financial education of executives on financial practices of medium and large enterprises," NOVAFRICA Working Paper Series wp2105, Universidade Nova de Lisboa, Nova School of Business and Economics, NOVAFRICA.
    9. Mckenzie,David J., 2020. "Small Business Training to Improve Management Practices in Developing Countries: Reassessingthe Evidence for 'Training Doesn’t Work'," Policy Research Working Paper Series 9408, The World Bank.
    10. Diego Javier Ubfal, 2023. "What Works in Supporting Women-led Businesses?," World Bank Publications - Reports 38564, The World Bank Group.
    11. Beber, Bernd & Lakemann, Tabea & Schnars, Regina & Lay, Jann, 2024. "Employment effects of skills trainings in Sub-Saharan Africa: A systematic review of recent randomized controlled trials," Ruhr Economic Papers 1080, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    12. Anderson, Stephen J. & Chintagunta, Pradeep & Vilcassim, Naufel J., 2023. "Virtual collaboration technology and international business coaching: examining the impact on marketing strategies and sales," LSE Research Online Documents on Economics 120474, London School of Economics and Political Science, LSE Library.
    13. Stephen J. Anderson & Pradeep K. Chintagunta & Naufel Vilcassim, 2024. "Virtual Collaboration Technology and International Business Coaching: Examining the Impact on Marketing Strategies and Sales," Marketing Science, INFORMS, vol. 43(3), pages 637-672, May.
    14. Dammert, Ana C. & Nansamba, Aisha, 2023. "Skills training and business outcomes: Experimental evidence from Liberia," World Development, Elsevier, vol. 162(C).
    15. Stephen J. Anderson & Rajesh Chandy & Bilal Zia, 2018. "Pathways to Profits: The Impact of Marketing vs. Finance Skills on Business Performance," Management Science, INFORMS, vol. 64(12), pages 5559-5583, December.
    16. Kilian Huber & Volker Lindenthal & Fabian Waldinger, 2021. "Discrimination, Managers, and Firm Performance: Evidence from “Aryanizations” in Nazi Germany," Journal of Political Economy, University of Chicago Press, vol. 129(9), pages 2455-2503.
    17. Ari Bronsoler & John Van Reenen & Joseph Doyle, 2022. "The Impact of Health Information and Communication Technology on Clinical Quality, Productivity, and Workers," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 23-46, August.
    18. Cristiano Antonelli & Francesco Crespi & Giuseppe Scellato, 2018. "Productivity growth persistence: firm strategies, size and system properties," Chapters, in: The Evolutionary Complexity of Endogenous Innovation, chapter 8, pages 176-202, Edward Elgar Publishing.
    19. Daniela Scur & Raffaella Sadun & John Van Reenen & Renata Lemos & Nicholas Bloom, 2021. "The World Management Survey at 18: lessons and the way forward," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 37(2), pages 231-258.
    20. Stefan Dimitriadis & Rembrand Koning, 2022. "Social Skills Improve Business Performance: Evidence from a Randomized Control Trial with Entrepreneurs in Togo," Management Science, INFORMS, vol. 68(12), pages 8635-8657, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:hdjpk. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.