We developed model systems for detecting medical, legal, and ethical risks in medical chatbot answers by using BERT and ChatGPT language models. The ChatGPT model system, which refers to external medical knowledge, performed best in detecting medical risk, while the BERT model system performed well in detecting legal and ethical risks. The hybrid model system reduces missed risks by combining the best of the BERT and ChatGPT model systems and has the best recall values for all risk determination models. This study demonstrates the usefulness of utilizing external medical knowledge and the effectiveness of the hybrid approach.
Ohara et al. (2025) studied this question.