IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/hdjpk_v1.html
   My bibliography  Save this paper

The Uneven Impact of Generative AI on Entrepreneurial Performance

Author

Listed:
  • Otis, Nicholas G.
  • Clarke, Rowan Philip
  • Delecourt, Solene
  • Holtz, David

    (University of California, Berkeley)

  • Koning, Rembrand

    (Harvard Business School)

Abstract

Scalable and low-cost AI assistance has the potential to improve firm decision-making and economic performance. However, running a business involves a myriad of open-ended problems, making it difficult to know whether recent AI advances can help business owners make better decisions in real-world markets. In a field experiment with Kenyan entrepreneurs, we assessed the impact of AI advice on small business revenues and profits by randomizing access to a GPT-4-powered AI business assistant via WhatsApp. While we are unable to reject the null hypothesis that there is no average treatment effect, we find the treatment effect for entrepreneurs who were high performing at baseline to be 0.27 standard deviations greater than for low performers. Sub-sample analyses show high performers benefited by just over 15% from the AI assistant, whereas low performers did about 8% worse. This increase in performance inequality does not stem from differences in the questions posed to or advice received from the AI, but from how entrepreneurs selected from and implemented the AI advice they received. More broadly, our findings demonstrate that generative AI is already capable of impacting—though in uneven and unexpected ways—real, open-ended, and unstructured business decisions.

Suggested Citation

  • Otis, Nicholas G. & Clarke, Rowan Philip & Delecourt, Solene & Holtz, David & Koning, Rembrand, 2023. "The Uneven Impact of Generative AI on Entrepreneurial Performance," OSF Preprints hdjpk_v1, Center for Open Science.
    • Handle: RePEc:osf:osfxxx:hdjpk_v1
    DOI: 10.31219/osf.io/hdjpk_v1
    as

    Download full text from publisher

    File URL: https://osf.io/download/658386417094e91347a177e5/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/hdjpk_v1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. David McKenzie & Christopher Woodruff, 2017. "Business Practices in Small Firms in Developing Countries," Management Science, INFORMS, vol. 63(9), pages 2967-2981, September.
    2. Jeffrey R Kling & Jeffrey B Liebman & Lawrence F Katz, 2007. "Experimental Analysis of Neighborhood Effects," Econometrica, Econometric Society, vol. 75(1), pages 83-119, January.
    3. Stefan Dimitriadis & Rembrand Koning, 2022. "Social Skills Improve Business Performance: Evidence from a Randomized Control Trial with Entrepreneurs in Togo," Management Science, INFORMS, vol. 68(12), pages 8635-8657, December.
    4. Davies, Elwyn & Deffebach, Peter & Iacovone, Leonardo & McKenzie, David, 2024. "Training microentrepreneurs over Zoom: Experimental evidence from Mexico," Journal of Development Economics, Elsevier, vol. 167(C).
    5. Emma van Inwegen & Zanele Munyikwa & John J. Horton, 2023. "Algorithmic Writing Assistance on Jobseekers' Resumes Increases Hires," Papers 2301.08083, arXiv.org.
    6. Natalie A. Carlson, 2023. "Differentiation in microenterprises," Strategic Management Journal, Wiley Blackwell, vol. 44(5), pages 1141-1167, May.
    7. Grimmer, Justin & Stewart, Brandon M., 2013. "Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts," Political Analysis, Cambridge University Press, vol. 21(3), pages 267-297, July.
    8. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
    9. Erik Brynjolfsson & Lorin M. Hitt, 2000. "Beyond Computation: Information Technology, Organizational Transformation and Business Performance," Journal of Economic Perspectives, American Economic Association, vol. 14(4), pages 23-48, Fall.
    10. Karlan, Dean & Knight, Ryan & Udry, Christopher, 2015. "Consulting and capital experiments with microenterprise tailors in Ghana," Journal of Economic Behavior & Organization, Elsevier, vol. 118(C), pages 281-302.
    11. Erik Brynjolfsson & Daniel Rock & Chad Syverson, 2021. "The Productivity J-Curve: How Intangibles Complement General Purpose Technologies," American Economic Journal: Macroeconomics, American Economic Association, vol. 13(1), pages 333-372, January.
    12. Kevin A. Bryan & András Tilcsik & Brooklynn Zhu, 2017. "Which Entrepreneurs Are Coachable and Why?," American Economic Review, American Economic Association, vol. 107(5), pages 312-316, May.
    13. Wyatt Brooks & Kevin Donovan & Terence R. Johnson, 2018. "Mentors or Teachers? Microenterprise Training in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 10(4), pages 196-221, October.
    14. Stephen J. Anderson & David McKenzie, 2022. "Improving Business Practices and the Boundary of the Entrepreneur: A Randomized Experiment Comparing Training, Consulting, Insourcing, and Outsourcing," Journal of Political Economy, University of Chicago Press, vol. 130(1), pages 157-209.
    15. McKenzie, David, 2012. "Beyond baseline and follow-up: The case for more T in experiments," Journal of Development Economics, Elsevier, vol. 99(2), pages 210-221.
    16. Emma Wiles & Zanele T. Munyikwa & John J. Horton, 2023. "Algorithmic Writing Assistance on Jobseekers’ Resumes Increases Hires," NBER Working Papers 30886, National Bureau of Economic Research, Inc.
    17. David Autor, 2014. "Polanyi's Paradox and the Shape of Employment Growth," NBER Working Papers 20485, National Bureau of Economic Research, Inc.
    18. Victor Chernozhukov & Mert Demirer & Esther Duflo & Iván Fernández-Val, 2018. "Generic Machine Learning Inference on Heterogeneous Treatment Effects in Randomized Experiments, with an Application to Immunization in India," NBER Working Papers 24678, National Bureau of Economic Research, Inc.
    19. Avi Goldfarb, 2024. "Pause artificial intelligence research? Understanding AI policy challenges," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 57(2), pages 363-377, May.
    20. Cusolito, Ana P. & Darova, Ornella & McKenzie, David, 2023. "Capacity building as a route to export market expansion: A six-country experiment in the Western Balkans," Journal of International Economics, Elsevier, vol. 144(C).
    21. Anderson, Stephen J. & Lazicky, Christy & Zia, Bilal, 2021. "Measuring the unmeasured: Aggregating, anchoring, and adjusting to estimate small business performance," Journal of Development Economics, Elsevier, vol. 151(C).
    22. Jorge Guzman & Aishen Li, 2023. "Measuring Founding Strategy," Management Science, INFORMS, vol. 69(1), pages 101-118, January.
    23. Aaron Chatterji & Solène Delecourt & Sharique Hasan & Rembrand Koning, 2019. "When does advice impact startup performance?," Strategic Management Journal, Wiley Blackwell, vol. 40(3), pages 331-356, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Otis, Nicholas G. & Clarke, Rowan Philip & Delecourt, Solene & Holtz, David & Koning, Rembrand, 2023. "The Uneven Impact of Generative AI on Entrepreneurial Performance," OSF Preprints hdjpk, Center for Open Science.
    2. Islam, Asadul & Lee, Wang-Sheng & Triyana, Margaret & Xia, Xing, 2023. "Improving Health and Safety in the Informal Sector: Evidence from a Randomized Trial in Bangladesh," IZA Discussion Papers 16150, Institute of Labor Economics (IZA).
    3. Mckenzie,David J., 2020. "Small Business Training to Improve Management Practices in Developing Countries: Reassessingthe Evidence for 'Training Doesn’t Work'," Policy Research Working Paper Series 9408, The World Bank.
    4. Higuchi, Yuki & Mhede, Edwin P. & Sonobe, Tetsushi, 2019. "Short- and medium-run impacts of management training: An experiment in Tanzania," World Development, Elsevier, vol. 114(C), pages 220-236.
    5. Diego Javier Ubfal, 2023. "What Works in Supporting Women-led Businesses?," World Bank Publications - Reports 38564, The World Bank Group.
    6. Beber, Bernd & Lakemann, Tabea & Schnars, Regina & Lay, Jann, 2024. "Employment effects of skills trainings in Sub-Saharan Africa: A systematic review of recent randomized controlled trials," Ruhr Economic Papers 1080, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    7. Anderson, Stephen J. & Chintagunta, Pradeep & Vilcassim, Naufel J., 2023. "Virtual collaboration technology and international business coaching: examining the impact on marketing strategies and sales," LSE Research Online Documents on Economics 120474, London School of Economics and Political Science, LSE Library.
    8. Stephen J. Anderson & Pradeep K. Chintagunta & Naufel Vilcassim, 2024. "Virtual Collaboration Technology and International Business Coaching: Examining the Impact on Marketing Strategies and Sales," Marketing Science, INFORMS, vol. 43(3), pages 637-672, May.
    9. Dammert, Ana C. & Nansamba, Aisha, 2023. "Skills training and business outcomes: Experimental evidence from Liberia," World Development, Elsevier, vol. 162(C).
    10. Dalton, Patricio & Rüschenpöhler, Julius & Uras, Burak & Zia, Bilal, 2019. "Local Best Practices for Business Growth," Other publications TiSEM fc650e2f-88cf-4d75-8257-f, Tilburg University, School of Economics and Management.
    11. Bloom, Nicholas & Iacovone, Leonardo & Pereira-Lopez, Mariana & Van Reenen, John, 2022. "Management and Misallocation in Mexico," CEPR Discussion Papers 16979, C.E.P.R. Discussion Papers.
    12. Catia Batista & Sandra Sequeira & Pedro C. Vicente, 2022. "Closing the Gender Profit Gap?," Management Science, INFORMS, vol. 68(12), pages 8553-8567, December.
    13. Lafortune, Jeanne & Pugatch, Todd & Tessada, José & Ubfal, Diego, 2024. "Can gamified online training make high school students more entrepreneurial? Experimental evidence from Rwanda," Economics of Education Review, Elsevier, vol. 101(C).
    14. Sedlmayr, Richard & Shah, Anuj & Sulaiman, Munshi, 2020. "Cash-plus: Poverty impacts of alternative transfer-based approaches," Journal of Development Economics, Elsevier, vol. 144(C).
    15. Claudia Custodio & Diogo Mendes & Daniel Metzger, 2021. "The impact of financial education of executives on financial practices of medium and large enterprises," NOVAFRICA Working Paper Series wp2105, Universidade Nova de Lisboa, Nova School of Business and Economics, NOVAFRICA.
    16. Antonia Grohmann & Lukas Menkhoff & Helke Seitz, 2022. "The Effect of Personalized Feedback on Small Enterprises’ Finances in Uganda," Economic Development and Cultural Change, University of Chicago Press, vol. 70(3), pages 1197-1227.
    17. Ubfal, Diego & Arráiz, Irani & Beuermann, Diether W. & Frese, Michael & Maffioli, Alessandro & Verch, Daniel, 2022. "The impact of soft-skills training for entrepreneurs in Jamaica," World Development, Elsevier, vol. 152(C).
    18. Campos, Francisco & Goldstein, Markus & McKenzie, David, 2023. "How should the government bring small firms into the formal system? Experimental evidence from Malawi," Journal of Development Economics, Elsevier, vol. 161(C).
    19. Davies, Elwyn & Deffebach, Peter & Iacovone, Leonardo & McKenzie, David, 2024. "Training microentrepreneurs over Zoom: Experimental evidence from Mexico," Journal of Development Economics, Elsevier, vol. 167(C).
    20. Jeffrey D. Michler & Anna Josephson, 2022. "Recent developments in inference: practicalities for applied economics," Chapters, in: A Modern Guide to Food Economics, chapter 11, pages 235-268, Edward Elgar Publishing.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:hdjpk_v1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.