Towards a clinical large language model: an ethico-legal case study analysis

Large Language Models (LLMs) have significant potential for transforming healthcare. However, there is a limited number of LLMs trained on clinical routine data globally and in Switzerland, no models have been trained using extensive electronic health record data yet. While this approach holds promise, it raises significant regulatory and ethical challenges. This paper explores these challenges through a case study of a Swiss initiative leveraging high-performance computing to train a clinical LLM using unstructured medical data. We begin by exploring the applicable regulatory framework and the challenges of bridging technical and human research. We then analyse challenges related to data security and compliance with data minimization requirements. This is followed by an examination of the complexities of communicating with study participants to ensure informed consent and meaningful public engagement. Our discussion is framed by data protection principles and offers recommendations for researchers, ethics committees, and policymakers within the Swiss research landscape and beyond. We argue that the responsible development of clinical LLMs requires a coordinated effort among all stakeholders, emphasizing the need for harmonized practices and a clear regulatory framework. Additionally, lagging regulations and practices, such as those related to anonymization and general consent, must be updated to address emerging challenges. Ultimately, a holistic approach is needed to balance ethico-legal considerations with the effective development of clinical LLMs. Lessons learned in Switzerland are likely relevant on a global scale and can provide insights to guide the evolution of ethical and regulatory frameworks governing the development of clinical LLMs.