Scientific Claim Recognition via Staged Fine-Tuning with LoRA

Scientific claims, which present propositions as facts, are fundamental to scientific knowledge. Despite their significance, current methods for scientific claim recognition are hindered by the scarcity of annotated datasets, particularly those covering full-text documents rather than just abstracts. To bridge this gap, this study aims to enhance scientific claim recognition by leveraging transfer learning through a staged fine-tuning approach. Specifically, we employ a large move prediction dataset (RCMR 280k) alongside the smaller SciClaim dataset we developed, to enhance our scientific claim recognition model’s ability to distinguish between various types of scientific narratives and their roles within research papers. We converted the labeled sentences from both datasets into a question-answer format, aligning them with the fine-tuning requirements of large language models. During the fine-tuning process, we explore two distinct strategies for incorporating knowledge from previous phases. Results indicate that re-integrating LoRA trained on the RCMR 280k dataset into the original model, followed by the creation of a new LoRA specifically for SciClaim training, produces the best outcomes. This staged fine-tuning approach efficiently adapts the model to the task of scientific claim recognition. Our model, SciClaim Miner, outperforms state-of-the-art approaches, achieving an F1-score of 90.96%. The ablation study demonstrates that both the dataset and prompt design, as well as the model training strategies, significantly enhance performance. This work advances scientific claim recognition by introducing a robust methodology that bridges the gap between limited data and effective model training.