Impact of Test Set Composition on AI Performance for Pediatric Radiograph Appendicular Skeleton Fracture Detection

Key Points

• To evaluate how the composition of test sets impacts the performance of AI in detecting fractures in pediatric radiographs.
• Analyzed AI performance during fracture detection tasks.
• Utilized internal test sets with varying image complexity.
• Measured prediction accuracy and odds of correct predictions against test set composition.
• Difficult radiographs in the test set correlated with lower prediction accuracy.
• Odds of correct predictions decreased with more complex images.