Case Study on Using AI for Feedback and Evaluation in an Undergraduate Statistics Course: A Pragmatic Workflow Approach

Artificial Intelligence (AI) is increasingly utilized in student assessment, particularly for creating and grading tests. However, current applications largely focus on multiple-choice questionnaires. In this contribution, we present an innovative case study employing AI, specifically natural language processing techniques, for the evaluation and feedback of research papers submitted by students. We designed and implemented a structured experiment involving second-year undergraduate students enrolled in an introductory statistics course for social sciences. Students were required to submit a 20-page research paper, including a literature review, descriptive statistics, hypothesis formulation, and testing. The aim was not to fully automate evaluation, but rather to create a pragmatic workflow that meaningfully complements human judgment while addressing institutional, ethical, and pedagogical considerations. Through iterative testing with actual student assignments and previously graded submissions, we assessed AI’s ability to deliver motivational feedback, pinpoint areas for improvement, provide rubric-aligned scoring, and detect inconsistencies or possible academic misconduct. In an initial implementation, AI reliably delivered surface-level praise and basic rubric-driven evaluations but struggled with deeper contextual judgments and accurate fraud detection without explicit guidance. To overcome these limitations, we established a four-step workflow: (1) defining red flags based on prior subject matter expert (SME) knowledge; (2) scanning submissions for red flags, inconsistencies, and suspicious data; (3) rubric-based evaluation supported by justifications and specific quotes; and (4) SME intervention to finalize feedback using the AI-generated insights. Key insights include the effectiveness of “chain-of-verification” prompting, the necessity of developing domain-specific red-flag rubrics collaboratively with faculty, and navigating the strategic balance between delivering substantial feedback and ensuring time efficiency. Properly guided AI should help reduce expert timeload, enhancing efficiency and foster student engagement. Instead of replacing human expertise, our proposed model strategically leverages AI to highlight areas where expert intervention is most beneficial. In environments where students increasingly expect detailed feedback but faculty face time constraints, AI can offer a supportive, motivational, and pedagogically appropriate "light-touch" solution. This presentation provides practical examples, detailed workflow diagrams, and valuable insights for educators aiming to responsibly and effectively integrate AI into statistics education.