Mitigating Hallucination in Small Language Models via Contrastive Chain-of-Thought Fine-Tuning

Small Language Models (SLMs), typically comprising fewer than 3 billion parameters, offer efficient deployment for edge computing but are susceptible to reasoning hallucinations: they generate plausible but logically unsound multi-step solutions. While Chain-of-Thought (CoT) prompting enhances reasoning in larger models, SLMs often lack the capacity to maintain coherent reasoning chains. This paper introduces Contrastive Chain-of-Thought (CCoT) Fine-Tuning, a novel parameter-efficient training method that pairs correct reasoning paths with explicitly labeled logical fallacies during fine-tuning. Using Low-Rank Adaptation (LoRA) on the Phi-2 model, we show that exposing SLMs to curated negative reasoning examples sharpens their decision boundaries between valid and hallucinatory logic. Comprehensive evaluation on arithmetic (GSM8K) and symbolic reasoning (BBH) benchmarks shows that CCoT significantly reduces hallucination rates, measured by stepwise logical consistency, and improves final-answer accuracy by 12.5% relative to standard fine-tuning. This work provides a scalable, hardware-accessible framework for improving the reliability of resource-constrained language models in edge AI applications.