IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2024i5p629-d1342623.html
   My bibliography  Save this article

Computer Science Education in ChatGPT Era: Experiences from an Experiment in a Programming Course for Novice Programmers

Author

Listed:
  • Tomaž Kosar

    (Faculty of Electrical Engineering and Computer Science, University of Maribor, Koroška cesta 46, 2000 Maribor, Slovenia)

  • Dragana Ostojić

    (Faculty of Electrical Engineering and Computer Science, University of Maribor, Koroška cesta 46, 2000 Maribor, Slovenia)

  • Yu David Liu

    (Department of Computer Science, State University of New York at Binghamton (SUNY), 4400 Vestal Parkway East, Binghamton, NY 13902, USA)

  • Marjan Mernik

    (Faculty of Electrical Engineering and Computer Science, University of Maribor, Koroška cesta 46, 2000 Maribor, Slovenia)

Abstract

The use of large language models with chatbots like ChatGPT has become increasingly popular among students, especially in Computer Science education. However, significant debates exist in the education community on the role of ChatGPT in learning. Therefore, it is critical to understand the potential impact of ChatGPT on the learning, engagement, and overall success of students in classrooms. In this empirical study, we report on a controlled experiment with 182 participants in a first-year undergraduate course on object-oriented programming. Our differential study divided students into two groups, one using ChatGPT and the other not using it for practical programming assignments. The study results showed that the students’ performance is not influenced by ChatGPT usage (no statistical significance between groups with a p -value of 0.730), nor are the grading results of practical assignments ( p -value 0.760) and midterm exams ( p -value 0.856). Our findings from the controlled experiment suggest that it is safe for novice programmers to use ChatGPT if specific measures and adjustments are adopted in the education process.

Suggested Citation

  • Tomaž Kosar & Dragana Ostojić & Yu David Liu & Marjan Mernik, 2024. "Computer Science Education in ChatGPT Era: Experiences from an Experiment in a Programming Course for Novice Programmers," Mathematics, MDPI, vol. 12(5), pages 1-22, February.
    • Handle: RePEc:gam:jmathe:v:12:y:2024:i:5:p:629-:d:1342623
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/5/629/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/12/5/629/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chris Stokel-Walker & Richard Van Noorden, 2023. "What ChatGPT and generative AI mean for science," Nature, Nature, vol. 614(7947), pages 214-216, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Turgut Karakose & Murat Demirkol & Ramazan Yirci & Hakan Polat & Tuncay Yavuz Ozdemir & Tijen Tülübaş, 2023. "A Conversation with ChatGPT about Digital Leadership and Technology Integration: Comparative Analysis Based on Human–AI Collaboration," Administrative Sciences, MDPI, vol. 13(7), pages 1-19, June.
    2. Ion-Danut LIXANDRU, 2024. "The Use of Artificial Intelligence for Qualitative Data Analysis: ChatGPT," Informatica Economica, Academy of Economic Studies - Bucharest, Romania, vol. 28(1), pages 57-67.
    3. Xing, Yunfei & Zhang, Justin Zuopeng & Teng, Guangqing & Zhou, Xiaotang, 2024. "Voices in the digital storm: Unraveling online polarization with ChatGPT," Technology in Society, Elsevier, vol. 77(C).
    4. Chong Lan & Yongsheng Wang & Chengze Wang & Shirong Song & Zheng Gong, 2023. "Application of ChatGPT-Based Digital Human in Animation Creation," Future Internet, MDPI, vol. 15(9), pages 1-18, September.
    5. Brady D. Lund & Ting Wang & Nishith Reddy Mannuru & Bing Nie & Somipam Shimray & Ziang Wang, 2023. "ChatGPT and a new academic reality: Artificial Intelligence‐written research papers and the ethics of the large language models in scholarly publishing," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(5), pages 570-581, May.
    6. Manu Suvarna & Alain Claude Vaucher & Sharon Mitchell & Teodoro Laino & Javier Pérez-Ramírez, 2023. "Language models and protocol standardization guidelines for accelerating synthesis planning in heterogeneous catalysis," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    7. Kim, Jungkeun & Kim, Jeong Hyun & Kim, Changju & Park, Jooyoung, 2023. "Decisions with ChatGPT: Reexamining choice overload in ChatGPT recommendations," Journal of Retailing and Consumer Services, Elsevier, vol. 75(C).
    8. Wahyono, Budi & Rapih, Subroto & Boungou, Whelsy, 2023. "Unleashing the wordsmith: Analysing the stock market reactions to the launch of ChatGPT in the US Education sector," Finance Research Letters, Elsevier, vol. 58(PC).
    9. Daniel Souza & Aldo Geuna & Jeff Rodr'iguez, 2024. "How Small is Big Enough? Open Labeled Datasets and the Development of Deep Learning," Papers 2408.10359, arXiv.org.
    10. Giordano, Vito & Spada, Irene & Chiarello, Filippo & Fantoni, Gualtiero, 2024. "The impact of ChatGPT on human skills: A quantitative study on twitter data," Technological Forecasting and Social Change, Elsevier, vol. 203(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2024:i:5:p:629-:d:1342623. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.