ChatGPT’s progress over time: A longitudinal enhancing biostatistical problem-solving in medical education

Objective: ChatGPT has been recognised as a potentially transformative tool in higher education by enhancing the teaching and learning process. Cross-sectional evaluations have acknowledged this potential. This study evaluates ChatGPT’s performance in solving specific biostatistical problems, focusing on accuracy, stability, and reproducibility, and explores its potential as a reliable educational tool in medical education. Methods: The correlation analysis task from Statistics at Square One by Swinscow and Campbell was chosen for its foundational role in biostatistics. Between October 2023 and March 2024, and July 2024, GPT-3.5 and GPT-4 were tested for accuracy in 12 parameters. Results: A statistically significant change in correct response rates was established in repeated measurements in the period October 2023, March 2024, and July 2024 for GPT-3.5 (Q = 100.99, p < 0.001), GPT-4.0 (Q = 89.55, p < 0.001), respectively. The significant GPT-3.5 improvement was established between March 2024/July 2024 ( p = 0.004), and between October 2023 and July 2024 ( p = 0.008). The significant GPT-4.0 improvement was established between October 2023 and March 2024 ( p = 0.004), and between October 2023 and July 2024 ( p = 0.026). Conclusion: Over 9 months, GPT-4 demonstrated rapid and consistent improvements, achieving perfect accuracy by March 2024. Although this study documented ChatGPT’s advancement within 9 months, ChatGPT should be positioned as a supplementary tool in higher education classrooms, in the presence of educators, to enhance the learning process.