Large Language Models Outperform PGY-5 Residents on the Orthopaedic In-Training Examination: A Comparative Analysis of Six Cutting-Edge Large Language Models

Key Points

• This analysis aims to compare the performance of large language models to postgraduate residents on an orthopaedic examination.
• Compared performance of ChatGPT and other large language models against PGY-5 resident scores on the orthopaedic examination.
• Evaluated accuracy and reasoning quality across multiple metrics.
• ChatGPT scored highest in accuracy, matching PGY-5 resident level.
• Performance across all metrics was superior in large language models compared to resident averages.

Abstract