IDEAS home Printed from https://ideas.repec.org/a/pal/palcom/v11y2024i1d10.1057_s41599-024-03991-6.html
   My bibliography  Save this article

Investigating students’ programming behaviors, interaction qualities and perceptions through prompt-based learning in ChatGPT

Author

Listed:
  • Dan Sun

    (Chinese Education Modernization Research Institute of Hangzhou Normal University (Zhejiang Provincial Key Think Tank))

  • Azzeddine Boudouaia

    (Southwest Jiaotong University)

  • Junfeng Yang

    (Hangzhou Normal University)

  • Jie Xu

    (Zhejiang University)

Abstract

ChatGPT has proven to facilitate computer programming tasks through the strategic use of prompts, which effectively steer the interaction with the language model towards eliciting relevant information. However, the impact of specifically designed prompts on programming learning outcomes has not been rigorously examined through empirical research. This study adopted a quasi-experimental framework to investigate the differential effects of prompt-based learning (PbL) versus unprompted learning (UL) conditions on the programming behaviors, interaction qualities, and perceptions of college students. The study sample consisted of 30 college students who were randomly assigned to two groups. A mixed-methods approach was employed to gather multi-faceted data. Results revealed notable distinctions between the two learning conditions. First, the PbL group students frequently engaged in coding with Python and employed debugging strategies to verify their work, whereas their UL counterparts typically transferred Python code from PyCharm into ChatGPT and posed new questions within ChatGPT. Second, PbL participants were inclined to formulate more complex queries independently, prompted by the guiding questions, and consequently received more precise feedback from ChatGPT compared to the UL group. UL students tended to participate in more superficial-level interactions with ChatGPT, yet they also obtained accurate feedback. Third, there were noticeable differences in perception observed before and after the ChatGPT implementation, UL group reported a more favorable perception in the perceived ease of use in the pre-test, while the PbL group experienced an improvement in their mean scores for perceived usefulness, ease of use, behavioral intention to utilize, and a significant difference regarding the attitude towards utilizing ChatGPT. Specifically, the use of structured output and delimiters enhanced learners’ understanding of problem-solving steps and made learning more efficient with ChatGPT. Drawing on these outcomes, the study offers recommendations for the incorporation of ChatGPT into future instructional designs, highlighting the structured prompting benefits in enhancing programming learning experience.

Suggested Citation

  • Dan Sun & Azzeddine Boudouaia & Junfeng Yang & Jie Xu, 2024. "Investigating students’ programming behaviors, interaction qualities and perceptions through prompt-based learning in ChatGPT," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-14, December.
    • Handle: RePEc:pal:palcom:v:11:y:2024:i:1:d:10.1057_s41599-024-03991-6
    DOI: 10.1057/s41599-024-03991-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1057/s41599-024-03991-6
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1057/s41599-024-03991-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yi Xie & Azzeddine Boudouaia & Jinfen Xu & Abdo Hasan AL-Qadri & Asma Khattala & Yan Li & Ya Min Aung, 2023. "A Study on Teachers’ Continuance Intention to Use Technology in English Instruction in Western China Junior Secondary Schools," Sustainability, MDPI, vol. 15(5), pages 1-18, February.
    2. Renate Andersen & Anders I. Mørch & Kristina Torine Litherland, 2022. "Collaborative learning with block-based programming: investigating human-centered artificial intelligence in education," Behaviour and Information Technology, Taylor & Francis Journals, vol. 41(9), pages 1830-1847, July.
    3. Viswanath Venkatesh & Fred D. Davis, 2000. "A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies," Management Science, INFORMS, vol. 46(2), pages 186-204, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Renzhong Peng & Qiqin Hu & Bochra Kouider, 2023. "Teachers’ Acceptance of Online Teaching and Emotional Labor in the EFL Context," Sustainability, MDPI, vol. 15(18), pages 1-19, September.
    2. Saeideh Sharifi fard & Ezhar Tamam & Md Salleh Hj Hassan & Moniza Waheed & Zeinab Zaremohzzabieh, 2016. "Factors affecting Malaysian university students’ purchase intention in social networking sites," Cogent Business & Management, Taylor & Francis Journals, vol. 3(1), pages 1182612-118, December.
    3. Chou, Jui-Sheng & Gusti Ayu Novi Yutami, I, 2014. "Smart meter adoption and deployment strategy for residential buildings in Indonesia," Applied Energy, Elsevier, vol. 128(C), pages 336-349.
    4. Philippe Cohard, 2020. "Information Systems Values: A Study of the Intranet in Three French Higher Education Institutions," Post-Print hal-02987225, HAL.
    5. Melih Engin & Fatih Gürses, 2019. "Adoption of Hospital Information Systems in Public Hospitals in Turkey: An Analysis with the Unified Theory of Acceptance and Use of Technology Model," International Journal of Innovation and Technology Management (IJITM), World Scientific Publishing Co. Pte. Ltd., vol. 16(06), pages 1-19, October.
    6. Morosan, Cristian, 2016. "An empirical examination of U.S. travelers’ intentions to use biometric e-gates in airports," Journal of Air Transport Management, Elsevier, vol. 55(C), pages 120-128.
    7. Abdesamad Zouine & Pierre Fenies, 2014. "The Critical Success Factors Of The ERP System Project: A Meta-Analysis Methodology," Post-Print hal-01419785, HAL.
    8. Talukder, Md. Shamim & Sorwar, Golam & Bao, Yukun & Ahmed, Jashim Uddin & Palash, Md. Abu Saeed, 2020. "Predicting antecedents of wearable healthcare technology acceptance by elderly: A combined SEM-Neural Network approach," Technological Forecasting and Social Change, Elsevier, vol. 150(C).
    9. Sarv Devaraj & Robert F. Easley & J. Michael Crant, 2008. "Research Note ---How Does Personality Matter? Relating the Five-Factor Model to Technology Acceptance and Use," Information Systems Research, INFORMS, vol. 19(1), pages 93-105, March.
    10. Claudio Vitari & Elisabetta Raguseo, 2016. "Big data value and financial performance: an empirical investigation [Digital data, dynamic capability and financial performance: an empirical investigation in the era of Big Data]," Post-Print halshs-01923271, HAL.
    11. Chen-Yuan Chen & Bih-Yaw Shih & Shih-Hsien Yu, 2012. "Disaster prevention and reduction for exploring teachers’ technology acceptance using a virtual reality system and partial least squares techniques," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 62(3), pages 1217-1231, July.
    12. Mäntymäki, Matti & Salo, Jari, 2013. "Purchasing behavior in social virtual worlds: An examination of Habbo Hotel," International Journal of Information Management, Elsevier, vol. 33(2), pages 282-290.
    13. Fatima Zahra Barrane & Gahima Egide Karuranga & Diane Poulin, 2018. "Technology Adoption and Diffusion: A New Application of the UTAUT Model," International Journal of Innovation and Technology Management (IJITM), World Scientific Publishing Co. Pte. Ltd., vol. 15(06), pages 1-19, December.
    14. Mary D. Adu & Usman H. Malabu & Aduli E. O. Malau-Aduli & Aaron Drovandi & Bunmi S. Malau-Aduli, 2020. "Efficacy and Acceptability of My Care Hub Mobile App to Support Self-Management in Australians with Type 1 or Type 2 Diabetes," IJERPH, MDPI, vol. 17(7), pages 1-18, April.
    15. Ofir Turel & Catherine E. Connelly, 2012. "Team Spirit: The Influence of Psychological Collectivism on the Usage of E-Collaboration Tools," Group Decision and Negotiation, Springer, vol. 21(5), pages 703-725, September.
    16. Baicu Claudia Gabriela & Gârdan Iuliana Petronela & Gârdan Daniel Adrian & Epuran Gheorghe, 2020. "The impact of COVID-19 on consumer behavior in retail banking. Evidence from Romania," Management & Marketing, Sciendo, vol. 15(s1), pages 534-556, October.
    17. Cong Cheng & Hongfang Cui, 2024. "Combining digital and legacy technologies: firm digital transformation strategies—evidence from Chinese manufacturing companies," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-14, December.
    18. Joan Torrent-Sellens & Cristian Salazar-Concha & Pilar Ficapal-Cusí & Francesc Saigí-Rubió, 2021. "Using Digital Platforms to Promote Blood Donation: Motivational and Preliminary Evidence from Latin America and Spain," IJERPH, MDPI, vol. 18(8), pages 1-17, April.
    19. Zamrudi Zakky & Wicaksono Teguh, 2018. "Promoting the Use of Social Commerce on SME in the Context of Logistics: UTAUT Model Examination," LOGI – Scientific Journal on Transport and Logistics, Sciendo, vol. 9(2), pages 73-82, November.
    20. Yuting Zhang & Xiaofen Yu & Ning Cai & Yong Li, 2020. "Analyzing the Employees’ New Media Use in the Energy Industry：The Role of Creative Self-Efficacy, Perceived Usefulness and Leaders’ Use," Sustainability, MDPI, vol. 12(3), pages 1-13, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pal:palcom:v:11:y:2024:i:1:d:10.1057_s41599-024-03991-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: https://www.nature.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.