OURad at the NTCIR-18 RadNLP Task: Predicting Lung Cancer Clinical Staging from Radiology Reports Using Few-Shot Prompting of Large Language Models

In this paper, we describe our proposed systems for the Japanese main task and sub task in Natural Language Processing for Radiology 2024 shared task. We employed Generative Pre-trained Transformer models and applied a few-shot prompting approach to tackle the classification task for lung cancer TNM staging from free-text radiology reports. Our method first performs zero-shot prompting using training data and then refines the final predictions by incorporating examples of incorrect predictions into the prompt. We demonstrate that this approach outperforms several BERT-based models and other open-source large language models. On the test data, our method achieved a Joint Accuracy (fine) of 0.732 for the main task and an overall micro F2.0 of 0.688 for the sub task, ranking 3rd in both categories.