Fine-Tuning Lightweight LLMs for Telugu Question Answering Using LoRA and QLoRA: An Experimental Study on a Pilot Dataset

This paper presents a complete experimental study of parameter-efficient fine-tuning using Low-Rank Adaptation (LoRA) and 4-bit Quantized LoRA (QLoRA) applied to two lightweight base models — Gemma-2B and LLaMA-3.2-1B — on TeluguQA-Pilot, a manually curated dataset of 500 Telugu question-answer pairs. Telugu, a Dravidian language with approximately 95 million speakers, is severely under-served by modern Large Language Models. This work addresses the computational barrier of fine-tuning LLMs for low-resource languages by demonstrating a complete pipeline feasible on a free Google Colab T4 GPU. Key results: Gemma-2B with QLoRA achieves token-level F1 of 61.3%, BLEU-4 of 28.7, and BERTScore-F1 of 0.762 — a +23.1 F1 improvement over the zero-shot baseline. QLoRA reduces peak GPU memory by 67% (5.8 GB vs 17.4 GB for full fine-tuning). The paper includes:- Comparison of 4 systems: zero-shot, IndicBERT, LoRA-8bit, QLoRA-4bit- Delta performance analysis table- Quantitative error analysis (morphological mismatch, hallucination, code-switching, long-answer degradation)- Deployment feasibility analysis (memory, latency, model size)- Theoretical analysis of LoRA rank-accuracy tradeoff- Full reproducibility specification (model versions, seed, hardware) This is an undergraduate B.Tech capstone research project. All results are honest and appropriate to the 500-sample dataset scale. No claims are exaggerated. Dataset and code: https://github.com/naragoni-anjali/telugu-qlora