IDEAS home Printed from https://ideas.repec.org/a/pal/palcom/v11y2024i1d10.1057_s41599-024-02751-w.html
   My bibliography  Save this article

What factors will affect the effectiveness of using ChatGPT to solve programming problems? A quasi-experimental study

Author

Listed:
  • Yuhui Jing

    (Zhejiang University of Technology)

  • Haoming Wang

    (East China Normal University)

  • Xiaojiao Chen

    (Zhejiang University of Technology)

  • Chengliang Wang

    (East China Normal University)

Abstract

The emergence of ChatGPT has sparked new expectations for AI-empowered educational transformation. However, it remains unknown which factors affect its effectiveness in empowering learners to solve programming problems. Therefore, this study employed a quasi-experimental research design and used Python graphing in programming education as an example to investigate the factors influencing the effectiveness of learners in applying ChatGPT to problem-solving. Findings: AI literacy significantly influences learners’ effectiveness in using ChatGPT to solve problems, with AI awareness and AI usage being key factors. The knowledge base of programming language significantly affects learners’ effectiveness in applying ChatGPT to solve programming problems. Learners’ cognitive level of ChatGPT significantly influences their effectiveness in applying ChatGPT to problem-solving, while usage intention does not have a significant impact. However, learners’ intention to use ChatGPT significantly improves after application. Based on these findings, this study proposes that in the process of empowering education with Artificial Intelligence Generated Content (AIGC) products, the focus on learners should shift from cultivating their AI usage to AI literacy, laying the foundation for empowering learning with AIGC products. It is suggested to shift from mastering specific knowledge to graph-based rules as a method for empowering learning with AIGC products. Additionally, the focus should shift from enhancing learners’ intention to use the technology to strengthen their technological awareness, thereby creating practical pathways for empowering learning with AIGC products.

Suggested Citation

  • Yuhui Jing & Haoming Wang & Xiaojiao Chen & Chengliang Wang, 2024. "What factors will affect the effectiveness of using ChatGPT to solve programming problems? A quasi-experimental study," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-12, December.
    • Handle: RePEc:pal:palcom:v:11:y:2024:i:1:d:10.1057_s41599-024-02751-w
    DOI: 10.1057/s41599-024-02751-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1057/s41599-024-02751-w
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1057/s41599-024-02751-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dowling, Michael & Lucey, Brian, 2023. "ChatGPT for (Finance) research: The Bananarama Conjecture," Finance Research Letters, Elsevier, vol. 53(C).
    2. Gaur, Ajai & Kumar, Mukesh, 2018. "A systematic approach to conducting review studies: An assessment of content analysis in 25years of IB research," Journal of World Business, Elsevier, vol. 53(2), pages 280-289.
    3. Viswanath Venkatesh, 2000. "Determinants of Perceived Ease of Use: Integrating Control, Intrinsic Motivation, and Emotion into the Technology Acceptance Model," Information Systems Research, INFORMS, vol. 11(4), pages 342-365, December.
    4. Teng Yu & Jian Dai & Chengliang Wang, 2023. "Adoption of blended learning: Chinese university students’ perspectives," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-16, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuhui Jing & Xiaojiao Chen & Keke Zhu & Shusheng Shen & Chengliang Wang, 2023. "The Implementation Path and Problems Encountered During Emergency Remote Teaching in Vocational Colleges: A Qualitative Study in China," SAGE Open, , vol. 13(4), pages 21582440231, November.
    2. Wenzel, Stefan, 2014. "App'ification of Enterprise Software - Evaluating Mobile App Characteristics Enabling Online Purchase And Their Portability To Enterprise Application Software," EconStor Preprints 146785, ZBW - Leibniz Information Centre for Economics.
    3. Cristina Oneț & Dana Georgeta Alexandru, 2023. "Revenues Sharing in Mineral Exploration: Local Authorities’ Incentives towards Economic Diversification in Romania," Sustainability, MDPI, vol. 15(4), pages 1-14, February.
    4. Irina Heimbach & Oliver Hinz, 2018. "The Impact of Sharing Mechanism Design on Content Sharing in Online Social Networks," Information Systems Research, INFORMS, vol. 29(3), pages 592-611, September.
    5. de Camargo Fiorini, Paula & Roman Pais Seles, Bruno Michel & Chiappetta Jabbour, Charbel Jose & Barberio Mariano, Enzo & de Sousa Jabbour, Ana Beatriz Lopes, 2018. "Management theory and big data literature: From a review to a research agenda," International Journal of Information Management, Elsevier, vol. 43(C), pages 112-129.
    6. Mäntymäki, Matti & Salo, Jari, 2013. "Purchasing behavior in social virtual worlds: An examination of Habbo Hotel," International Journal of Information Management, Elsevier, vol. 33(2), pages 282-290.
    7. Bilgihan, Anil & Barreda, Albert & Okumus, Fevzi & Nusair, Khaldoon, 2016. "Consumer perception of knowledge-sharing in travel-related Online Social Networks," Tourism Management, Elsevier, vol. 52(C), pages 287-296.
    8. Cong Cheng & Hongfang Cui, 2024. "Combining digital and legacy technologies: firm digital transformation strategies—evidence from Chinese manufacturing companies," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-14, December.
    9. Alaassar, Ahmad & Mention, Anne-Laure & Aas, Tor Helge, 2021. "Exploring a new incubation model for FinTechs: Regulatory sandboxes," Technovation, Elsevier, vol. 103(C).
    10. Waqas Mehmood & Rasidah Mohd-Rashid & Yasir Abdullah & Ataul Karim Patwary & Attia Aman-Ullah, 2023. "Inclusive mapping of initial public offerings: a bibliometric and literature review study," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(1), pages 655-700, February.
    11. Mansoureh Beheshti Nejad & Seyed Mahmoud Zanjirchi & Seyed Mojtaba Hosseini Bamakan & Negar Jalilian, 2024. "Blockchain Adoption in Operations Management: A Systematic Literature Review of 14 Years of Research," Annals of Data Science, Springer, vol. 11(4), pages 1361-1389, August.
    12. Chengliang Wang & Xiaojiao Chen & Teng Yu & Yidan Liu & Yuhui Jing, 2024. "Education reform and change driven by digital technology: a bibliometric study from a global perspective," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-17, December.
    13. Globisch, Joachim & Dütschke, Elisabeth & Schleich, Joachim, 2018. "Acceptance of electric passenger cars in commercial fleets," Transportation Research Part A: Policy and Practice, Elsevier, vol. 116(C), pages 122-129.
    14. Michael Addotey-Delove & Richard E. Scott & Maurice Mars, 2023. "Healthcare Workers’ Perspectives of mHealth Adoption Factors in the Developing World: Scoping Review," IJERPH, MDPI, vol. 20(2), pages 1-27, January.
    15. Yi Sun & Shihui Li & Lingling Yu, 2022. "The dark sides of AI personal assistant: effects of service failure on user continuance intention," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(1), pages 17-39, March.
    16. Nistor, Cristian, 2013. "A conceptual model for the use of social media in companies," MPRA Paper 44224, University Library of Munich, Germany.
    17. Zhang, Wenqing & Liu, Liangliang, 2022. "Exploring non-users' intention to adopt ride-sharing services: Taking into account increased risks due to the COVID-19 pandemic among other factors," Transportation Research Part A: Policy and Practice, Elsevier, vol. 158(C), pages 180-195.
    18. Jabbar, Abdul & Geebren, Ahmed & Hussain, Zahid & Dani, Samir & Ul-Durar, Shajara, 2023. "Investigating individual privacy within CBDC: A privacy calculus perspective," Research in International Business and Finance, Elsevier, vol. 64(C).
    19. Klaus E. Meyer & Chengguang Li & Andreas P. J. Schotter, 0. "Managing the MNE subsidiary: Advancing a multi-level and dynamic research agenda," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 0, pages 1-39.
    20. Kuldeep Baishya & Harsh Vardhan Samalia, 2020. "Factors Influencing Smartphone Adoption: A Study in the Indian Bottom of the Pyramid Context," Global Business Review, International Management Institute, vol. 21(6), pages 1387-1405, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pal:palcom:v:11:y:2024:i:1:d:10.1057_s41599-024-02751-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: https://www.nature.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.