Evaluating the impact of students' generative AI use in educational contexts

Purpose The purpose of the study was to evaluate the impact of generative artificial intelligence (GenAI) on students' learning experiences and perceptions through a master’s-level course. The study specifically focused on student engagement, comfort with GenAI and ethical considerations. Design/methodology/approach The study used an action research methodology employing qualitative data collection methods, including pre- and post-course surveys, reflective assignments, class discussions and a questionnaire. The AI-Ideas, Connections, Extensions (ICE) Framework, combining the ICE Model and AI paradigms, is used to assess students' cognitive engagement with GenAI. Findings The study revealed that incorporating GenAI in a master’s-level instructional design course increased students' comfort with GenAI and their understanding of its ethical implications. The AI-ICE Framework demonstrated most students were at the initial engagement level, with growing awareness of GenAI’s limitations and ethical issues. Course reflections highlighted themes of improved teaching strategies, personal growth and the practical challenges of integrating GenAI responsibly. Research limitations/implications The small sample size poses challenges to the analytical power of the findings, potentially limiting the breadth and applicability of conclusions. This constraint may affect the generalizability of the results, as the participants may not fully represent the broader population of interest. The researchers are mindful of these limitations and suggest caution in interpreting the findings, acknowledging that they may offer more exploratory insights than definitive conclusions. Future research endeavors should aim to recruit a larger cohort to validate and expand upon the initial observations, ensuring a more robust understanding. Originality/value The study is original in its integration of GenAI into a master's-level instructional design course, assessing both the practical and ethical implications of its use in education. By utilizing the AI-ICE Framework to evaluate students' cognitive engagement and employing action research methodology, the study provides insights into how GenAI influences learning experiences and perceptions. This approach bridges the gap between theoretical understanding and the real-world application of GenAI, offering actionable strategies for its responsible use in educational settings.