Introduction Differences in prior knowledge among incoming medical students pose a persistent challenge for universities. To promote more individualized and equitable preparation, a large language model-based learning platform is being developed at the University Medical Center Hamburg-Eppendorf. A central component of this platform is the automated generation of multiple-choice questions (MCQs) from curated medical materials. However, ensuring their educational quality remains difficult, particularly when relying on smaller, locally deployed language models. Methods This study introduces Node-Sampling, a self-optimizing multi-agent approach for improving MCQ quality. The method identifies efficient refinement strategies by modeling agents as an adaptive sequence optimized through the REINFORCE algorithm. Results Expert evaluations showed that Node-Sampling enhances the quality of question stems significantly compared to a fixed baseline. Importantly, Node-Sampling achieved this performance using an effective three-agent configuration, requiring only 33% of the original resources. Results for answer options were less consistent. Discussion The results highlight the potential of adaptive multi-agent optimization to strengthen automated question refinement. Node-Sampling therefore presents a sustainable and promising approach to better MCQ quality and supports more effective and personalized preparation for medical students.
Düsterbeck et al. (Thu,) studied this question.