A Systematic Review of Approaches for Evaluating Students’ Function-Level Code Structure Knowledge with a Catalog of All Discussed Patterns

Code structure, an important aspect of code quality, involves stylistic and structural decisions within individual functions, such as selecting appropriate control flow structures for a specific task. Despite over forty years of research on student performance in various code structure topics, a comprehensive overview of these studies remains absent. Therefore, we conducted a systematic literature review to understand which function-level code structure topics were considered essential for student learning and what assessment practices were used to evaluate students’ knowledge of code structure. Our review identified approximately 70 code-structure topics, which we classified into seven categories: expressions, variables and parameters, Boolean expressions, conditionals, loops, functions, and others. We created a catalog of these topics that includes their definitions, an example defect for each topic, and a corresponding solution. This catalog can serve as a resource for instructors seeking to incorporate code structure into introductory programming courses, as well as for researchers developing educational tools or investigating the extent and causes of code-structure violations. Regarding assessment practices in prior studies, we found that various assessment tasks and student artifacts were utilized, with half of the studies examining students’ code-writing submissions for the occurrence of violations. While this approach is scalable through the use of automated detectors, it provides limited insight into the underlying causes of these violations or into students’ performance on non-writing skills such as code refactoring. Additionally, we found very little research that focuses on specific assessment tasks such as code recall and code editing (i.e., refactoring others’ code). Moreover, while many studies suggested that student code structure defects stem from knowledge gaps in programming concepts, only a few empirically tested this hypothesis. Finally, we conclude this study by highlighting open areas for future research on code structure.