From data to diagnosis: Overcoming challenges in the clinical machine learning lifecycle

The promise of Artificial Intelligence in modern healthcare lies in its potential to transform patient care by uncovering novel insights within the vast, multi-modal data generated by digitalized clinical workflows, ranging from clinical notes to volumetric images. Understanding and harnessing these sources to facilitate the generation of novel insights is a primary objective in contemporary research. However, despite this potential, there is a striking dissonance between the capabilities Artificial Intelligence seemingly offers and the lack of clinical translation of such solutions. Although the medical domain is among the most strictly regulated worldwide, challenges aside from regulation hinder successful adoption in clinical workflows. The key to unlocking the potential of clinical Artificial Intelligence is not exclusively in the pursuit of incremental algorithmic novelty, but in solving the logistical hurdles that slow translation. This work outlines and confronts three critical challenges head-on. First is the "Data Pipeline Problem", concerning the aggregation of diverse, multi-center datasets; this is addressed by two contributions introducing open-source software tools for efficient and data protection compliant collection and processing. Second is the "Analysis and Annotation Bottleneck" of efficiently exploring and annotating massive, unstructured data corpora, to which a contribution is made via a novel approach for interactive data exploration that moves beyond simple sample-by-sample analysis, fostering a holistic understanding of entire datasets. Finally, this research addresses the "Clinical Utility Gap", the final hurdle between a functioning model and a truly useful clinical tool. Building on the preceding efforts, a culminating clinical deployment in ophthalmology is presented to demonstrate a comprehensive, end-to-end pathway in an attempt to bridge this final gap. Together, these contributions form a complete framework that directly addresses the logistical pipeline required to successfully bring Machine Learning projects from initial data collection and ingestion through to robust, clinically useful endpoints, which is subject to a critical discussion.