IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2410.02091.html
   My bibliography  Save this paper

The Impact of Generative AI on Collaborative Open-Source Software Development: Evidence from GitHub Copilot

Author

Listed:
  • Fangchen Song
  • Ashish Agarwal
  • Wen Wen

Abstract

Generative artificial intelligence (AI) has opened the possibility of automated content production, including coding in software development, which can significantly influence the participation and performance of software developers. To explore this impact, we investigate the role of GitHub Copilot, a generative AI pair programmer, on software development in open-source community, where multiple developers voluntarily collaborate on software projects. Using GitHub's dataset for open-source repositories and a generalized synthetic control method, we find that Copilot significantly enhances project-level productivity by 6.5%. Delving deeper, we dissect the key mechanisms driving this improvement. Our findings reveal a 5.5% increase in individual productivity and a 5.4% increase in participation. However, this is accompanied with a 41.6% increase in integration time, potentially due to higher coordination costs. Interestingly, we also observe the differential effects among developers. We discover that core developers achieve greater project-level productivity gains from using Copilot, benefiting more in terms of individual productivity and participation compared to peripheral developers, plausibly due to their deeper familiarity with software projects. We also find that the increase in project-level productivity is accompanied with no change in code quality. We conclude that AI pair programmers bring benefits to developers to automate and augment their code, but human developers' knowledge of software projects can enhance the benefits. In summary, our research underscores the role of AI pair programmers in impacting project-level productivity within the open-source community and suggests potential implications for the structure of open-source software projects.

Suggested Citation

  • Fangchen Song & Ashish Agarwal & Wen Wen, 2024. "The Impact of Generative AI on Collaborative Open-Source Software Development: Evidence from GitHub Copilot," Papers 2410.02091, arXiv.org.
    • Handle: RePEc:arx:papers:2410.02091
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2410.02091
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alberto Abadie & Alexis Diamond & Jens Hainmueller, 2015. "Comparative Politics and the Synthetic Control Method," American Journal of Political Science, John Wiley & Sons, vol. 59(2), pages 495-510, February.
    2. Mohammed Alyakoob & Mohammad S. Rahman, 2022. "Shared Prosperity (or Lack Thereof) in the Sharing Economy," Information Systems Research, INFORMS, vol. 33(2), pages 638-658, June.
    3. Xu, Yiqing, 2017. "Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models," Political Analysis, Cambridge University Press, vol. 25(1), pages 57-76, January.
    4. Jushan Bai, 2009. "Panel Data Models With Interactive Fixed Effects," Econometrica, Econometric Society, vol. 77(4), pages 1229-1279, July.
    5. Yixin Lu & Alok Gupta & Wolfgang Ketter & Eric van Heck, 2019. "Information Transparency in Business-to-Business Auction Markets: The Role of Winner Identity Disclosure," Management Science, INFORMS, vol. 65(9), pages 4261-4279, September.
    6. David H. Autor, 2003. "Outsourcing at Will: The Contribution of Unjust Dismissal Doctrine to the Growth of Employment Outsourcing," Journal of Labor Economics, University of Chicago Press, vol. 21(1), pages 1-42, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Qili Wang & Liangfei Qiu & Wei Xu, 2024. "Informal Payments and Doctor Engagement in an Online Health Community: An Empirical Investigation Using Generalized Synthetic Control," Information Systems Research, INFORMS, vol. 35(2), pages 706-726, June.
    2. Xiong, Ruoxuan & Pelger, Markus, 2023. "Large dimensional latent factor modeling with missing observations and applications to causal inference," Journal of Econometrics, Elsevier, vol. 233(1), pages 271-301.
    3. Michał Marcin Kobierecki & Michał Pierzgalski, 2022. "Sports Mega-Events and Economic Growth: A Synthetic Control Approach," Journal of Sports Economics, , vol. 23(5), pages 567-597, June.
    4. Taylor K. Odle & Jennifer A. Delaney, 2022. "You are Admitted! Early Evidence on Enrollment from Idaho’s Direct Admissions System," Research in Higher Education, Springer;Association for Institutional Research, vol. 63(6), pages 899-932, September.
    5. Bai, Jushan & Wang, Peng, 2024. "Causal inference using factor models," MPRA Paper 120585, University Library of Munich, Germany.
    6. Hans-Bernd Schaefer & Rok Spruk, 2024. "Islamic Law, Western European Law and the Roots of Middle East's Long Divergence: a Comparative Empirical Investigation (800-1600)," Papers 2401.14435, arXiv.org, revised Mar 2024.
    7. Alexander S. Skorobogatov, 2021. "The effect of alcohol sales restrictions on alcohol poisoning mortality: Evidence from Russia," Health Economics, John Wiley & Sons, Ltd., vol. 30(6), pages 1417-1442, June.
    8. Chakraborty Avinandan & Doremus Jacqueline & Stith Sarah, 2021. "The effects of recreational cannabis access on labor markets: evidence from Colorado," IZA Journal of Labor Economics, Sciendo & Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 10(1), pages 1-86, January.
    9. Victor Chernozhukov & Kaspar Wüthrich & Yinchu Zhu, 2021. "An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1849-1864, October.
    10. Shu He & Jing Peng & Jianbin Li & Liping Xu, 2020. "Impact of Platform Owner’s Entry on Third-Party Stores," Information Systems Research, INFORMS, vol. 31(4), pages 1467-1484, December.
    11. Bruno Ferman, 2021. "On the Properties of the Synthetic Control Estimator with Many Periods and Many Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1764-1772, October.
    12. Dallas Dotter & Duncan Chaplin & Maria Bartlett, "undated". "Impacts of School Reforms in Washington, DC on Student Achievement," Mathematica Policy Research Reports 44e95d7566434a21b8d57f951, Mathematica Policy Research.
    13. Susan Athey & Mohsen Bayati & Nikolay Doudchenko & Guido Imbens & Khashayar Khosravi, 2021. "Matrix Completion Methods for Causal Panel Data Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1716-1730, October.
    14. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    15. Bennato, Anna Rita & Davies, Stephen & Mariuzzo, Franco & Ormosi, Peter, 2021. "Mergers and innovation: Evidence from the hard disk drive market," International Journal of Industrial Organization, Elsevier, vol. 77(C).
    16. Trang My Tran, 2022. "International Environmental Agreement and Trade in Environmental Goods: The Case of Kyoto Protocol," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 83(2), pages 341-379, October.
    17. Taylor K. Odle, 2022. "Free to Spend? Institutional Autonomy and Expenditures on Executive Compensation, Faculty Salaries, and Research Activities," Research in Higher Education, Springer;Association for Institutional Research, vol. 63(1), pages 1-32, February.
    18. Dmitry Arkhangelsky & Susan Athey & David A. Hirshberg & Guido W. Imbens & Stefan Wager, 2021. "Synthetic Difference-in-Differences," American Economic Review, American Economic Association, vol. 111(12), pages 4088-4118, December.
    19. Eli Ben-Michael & Avi Feller & Jesse Rothstein, 2021. "The Augmented Synthetic Control Method," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1789-1803, October.
    20. Bruno Ferman & Cristine Pinto, 2021. "Synthetic controls with imperfect pretreatment fit," Quantitative Economics, Econometric Society, vol. 12(4), pages 1197-1221, November.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2410.02091. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.