Development and validation of a deep learning-based emergency triage model: a feasibility and effectiveness study

Due to insufficient triage accuracy, general purpose Large Language Models (LLMs) cannot be directly applied to clinical pre-triage. At the same time, the existing emergency pre-triage systems also have certain limitations. Dedicated self-trained models, with their advantages of high performance, easy scalability, and low cost, have become the primary development direction in the field of medical diagnosis. We aimed to propose a method for training a dedicated emergency triage model to improve the accuracy of triage level classification for emergency patients. This study employed emergency triage data from December 2023 to December 2024 (raw data: 270,298 entries, cleaned data: 113,302 entries). The data was temporally partitioned into a training set (n = 91,743; 8:2 split for training and validation), a test set (n = 11,559), and an online diagnostic set (n = 10,000). The data was uploaded to the EasyDL platform to develop and deploy an emergency triage model, followed by the evaluation of model performance and online diagnosis. The emergency triage model developed on the EasyDL platform demonstrated reliable performance within defined parameters. The model achieved 97.2% accuracy (95% CI: 96.2–98.2%) on the training set and 97.7% (95% CI: 96.8–98.6%) on the test set. Post-deployment, the overall triage accuracy reached 89.45%, with > 90% accuracy across all disease severity levels (I–IV). The Matthews Correlation Coefficient (MCC) was 0.92. External validation achieved an accuracy of 78.3% (95% CI: 76.9–79.8%), representing a modest decrease compared to the Online Diagnostic set. The development process was simple and efficient; the model can be deployed to mobile terminals via an Application Programming Interface (API), enabling direct clinical application. The deep learning-based emergency triage model demonstrates potential for clinical translation, and its development plan is operationally feasible. However, to realize its practical value, subsequent research and efficacy validation in real-world clinical settings are necessary prior to implementation.