Abstract Teachers’ use of criterial and individual reference norms offers advantages for formative assessment. Therefore, in this randomized (mixed within-between-person) replication study, we investigate which reference norms teachers apply when interpreting learning progress data, whether the graphical representation has an influence on this and whether the general reference norm orientation can predict the situationally applied reference norm. This conceptual replication study not only offers the opportunity to confirm an effect and its direction already demonstrated in the original study, but we also strengthen the internal, external and construct validity by conducting the study with in-service teachers instead of pre-service teachers and avoiding carry-over effects through design changes. In the study, teachers interpret learning progress data of a school class first in a neutral graphical representation using an open-ended item. The basis for the second interpretation is a graphical representation of the learning progress data of a second class, in which one of the three reference norms (individual, criterial or social) is graphically highlighted. To measure the applied reference norm, we use an open-ended and a single-choice item. Finally, we assess the teachers’ reference norm orientation. In a neutral graphical representation of learning progress data, N = 157 teachers most frequently used one reference norm in their interpretation, namely the individual reference norm. The general reference norm orientation could predict the reference norm used in a specific assessment situation only to a limited extent. In addition, graphical representation showed consistent, conclusive effects of small to large magnitude (2.38 ≤ OR ≤ 7.05) on the situationally applied reference norm.
Lunowa et al. (2026) studied this question.