Fine-tuning large language models (LLMs) for specialized tasks is resource-intensive and challenging to scale across multiple domains. Model merging, which directly combines parameters from fine-tuned LLMs without additional training, has emerged as an efficient alternative. However, traditional merging methods, such as linear interpolation, often degrade performance due to destructive interference between conflicting parameters. To address these limitations, we propose Layer-Adaptive Spherical Linear Interpolation (Layer-Adaptive SLERP), a novel merging strategy that i) follows a geometry-preserving SLERP path and ii) assigns layer-specific coefficients that respect the distinct roles of attention, feed-forward and embedding blocks. Across 50+ merges spanning six architectures and seven parameter scales (0. 5 ~B-7 ~B), we demonstrate that our method significantly improves merging stability and task-specific performance. Results indicate that the merged 7B variant attains competitive leaderboard performance and supports production-scale deployments, confirming the method's robustness and applicability to realworld adaptation tasks.
Rage et al. (Mon,) studied this question.