Quality assessment for medical AI

Artificial Intelligence (AI) has the potential to make healthcare systems more efficient by supporting healthcare professionals in labor-intensive tasks. However, the underlying Machine Learning (ML) algorithms carry inherent biases from the training data that may cause failures during deployment when these algorithms encounter data that differ from the training distribution, which may ultimately pose risks to patient safety. While trustworthy AI guidelines grounded in ethical principles have been developed, fully realizing these principles in practice requires concrete, practical quality assessment approaches to ensure safe and responsible use in real-world healthcare settings. With the aim to realize the potential of AI in healthcare, I investigate approaches to assess whether AI algorithms are safe for use in medical practice. This thesis contributes to the operationalisation of trustworthy AI in healthcare, in particular through three key studies. Chapter 3 investigates how algorithmic bias impacts real-world deployment within a global health context by analyzing the commercial AI tool ‘CAD4TB’. CAD4TB analyzes chest x-rays for tuberculosis to triage individuals for diagnostic work-up. The study reveals performance limitations, version inconsistencies, and a lack of transparency around training data and intended use, all of which pose challenges for the deployment of this tool. Chapter 4 addresses the transparency gap by presenting a structured assessment approach to assess reported information on medical AI tools in relation to trustworthy AI principles. Applied to 14 authorized radiology tools in Europe, the study shows that current documentation is often insufficient to judge safety, bias, or suitability for specific settings. Chapter 5 addresses the challenge of performance robustness when encountering data differing from the training distribution and presents a method to test prediction models in synthetic environments before deployment. Using Alzheimer’s disease prediction as a case study, it demonstrates how synthetic data can help assess the robustness across varying data distributions. In summary, this thesis demonstrates how algorithmic biases can pose challenges to the practical use of medical AI tools and introduces practical methods to address these risks through quality assessment approaches that promote transparency and robustness, two core pillars of trustworthy medical AI. These methods aim to equip stakeholders with insights to decide whether a medical AI tool is safe to use in their specific context. Together, the methods and applications presented in this thesis help bridge the gap between technical machine learning and clinical practice, contributing to the responsible integration of AI in healthcare. Beyond the technical contributions, I critically reflect on the governance and oversight processes needed to ensure the safe and ethical deployment of AI. Navigating the risks of algorithmic bias, this work highlights that advancing trustworthy medical AI demands a multifaceted, interdisciplinary research and governance approach that enables technical innovation while ensuring clinical benefit, ethical scrutiny, and regulatory foresight.