Computer-Based Authentic Assessment in Engineering: A Case of Group Practice

Traditional assessment practices that focus on lower-level cognitive skills no longer align with the knowledge and social practices students will need as professionals in the era of generative artificial intelligence. In this qualitative study, we analyzed students’ approaches and participation in a computer-based group exam for second-year university engineering students in the US. Students were positioned as an engineering group in a hypothetical candy factory collecting reactor data within a Unity 3D computer simulation, representing the industrial setting. Four groups were video recorded as they completed the two-hour exam. Videos and transcripts were analyzed via interaction analysis. In addition, their exam answers and other course assessment scores were collected as secondary data. More successful groups in this study tended to share material tools on their Zoom screens and toggle between engineering decisions, while the least successful group rarely shared material tools and took a sequential approach to decision-making. We observed both single and dual leadership modes, with most students clearly participating in ways to move the solution forward. However, some students did not appear to contribute substantially, raising concerns about using such exams for credentialing purposes. Taking a theoretical lens of group practice within figured worlds, we observed that the individual students became entangled in the group’s approach implying that, in addition to the individual, the group should be a fundamental unit of analysis for engineering assessments. In the cases studied, the computer-based tool provided an authentic task that allowed them to demonstrate such group practices.