Appropriate performance metrics are essential for evaluating machine learning (ML) algorithm performance in dental image analysis. A broad variety of performance metrics are used in dental artificial intelligence (AI) research which could cause confusion regarding model performance. This study aimed to assess the knowledge, awareness, and self-reported confidence of oral health researchers and clinicians regarding AI performance metrics. This cross-sectional, questionnaire-based survey initially recruited participants ( n = 100) from the ITU/WHO/WIPO Global Initiative on AI for Health Topic Group – Oral Health and affiliated professional global networks. An online self-administered questionnaire evaluated theoretical knowledge and applied knowledge. Participants also reported basic demographics and their confidence in using different metrics. Overall performance was poor with participants achieving a mean score of 17.1% (SD: 15.2%, median: 16.7%, IQR: 16.7%) for fully correct responses. Responses to theoretical questions were rarely fully correct (range: 1%-15%; median: 5.5%). Applied scenario performance ranged from 17% to 37% correct responses (median: 29%). Participants demonstrated greater knowledge of common metrics (accuracy, specificity) but struggled with advanced measures (Intersection over Union & Dice Similarity Coefficient). Common errors included terminology confusion, familiarity bias towards accuracy, and task-metric misalignment. Substantial gaps exist in knowledge and awareness of both theory and practical applications of AI performance metrics amongst dental professionals and researchers. Frequent misapplication of commonly known metrics suggests overgeneralisation without consideration of task-specific requirements. Inadequate knowledge or awareness of metric selection and interpretation may lead to the deployment of suboptimal algorithms, which can compromise diagnostic accuracy and patient outcomes. Enhanced understanding of performance metrics is critical for ensuring reliable, fair, safe, and effective AI systems in clinical dentistry.
Ramani et al. (Wed,) studied this question.