Investigating the Effects of ChatGPT on Student Learning in Programming Courses

Abstract Authors intend to follow up with a full paper. One of the latest developments impacting computer science education is the availability of high-quality generative AI (Artificial Intelligence) tools like ChatGPT. These tools provide a range of functionalities, such as code creation, debugging assistance, and optimization. From the students' perspective, these systems can be virtual tutoring aids that complement the learning process. From the educators' perspective, the availability of AI tools necessitates adjustments to the course material and raises ethical concerns, primarily related to the potential for code plagiarism. Rather than disregarding or imposing prohibitive restrictions on these tools, our research group encouraged students to leverage AI tools to further their understanding of coursework. Instructors emphasized the importance of submitting original work and academic integrity to address ethical plagiarism concerns. To gauge the impact of AI tools on students' learning experiences, the group conducted a preliminary study to assess student perceptions and usage of ChatGPT utilizing a custom questionnaire. The initial study results showed that about half of the students had used ChatGPT before the beginning of the Fall 2023 semester. While consensus was not reached on questions concerning learning outcomes, material retention, and the impact on future job opportunities, there were significant agreements on certain aspects: ChatGPT cannot replace the need for in-person instruction and tutoring; code generated by ChatGPT is not always accurate; graded assignments should not be removed from the course assessment; AI platforms are helpful for challenging or selected class assignments; and requiring students to submit their original code alongside AI-generated code and/or flowcharts is beneficial.