Large Language Models: Architectures, Fine-Tuning, and Retrieval-Augmented Generation — A Comprehensive Review

Large Language Models (LLMs) have emerged as the dominant paradigm for natural language understanding and generation, progressing from encoder-only and encoder–decoder transformers to frontier decoder-only models with tens to hundreds of billions of parameters. This review presents a comprehensive survey of modern LLMs, organised around five interrelated themes: (i) transformer-based architectures and major model families including BERT, GPT, T5, LLaMA, Mistral, Claude, and Gemini; (ii) pre-training paradigms, scaling laws, and data curation practices; (iii) fine-tuning strategies with particular emphasis on parameter-efficient methods such as LoRA, QLoRA, adapters, and prefix tuning; (iv) retrieval-augmented generation (RAG) pipelines that ground LLM outputs in external knowledge; and (v) alignment techniques including supervised fine-tuning, reinforcement learning from human feedback (RLHF), and direct preference optimisation (DPO). We additionally cover inference-time efficiency, evaluation benchmarks, and real-world applications. The review concludes with key challenges such as hallucination, safety, reasoning limits, and computational cost, and highlights future research directions including mixture-of-experts, long-context modeling, and multimodal extensions.