ARCHIVES

This paper presents a comprehensive technical survey of large language model (LLM) fine-tuning, spanning the complete methodological landscape from foundational techniques to frontier advances as of early 2026. We organize the field along two orthogonal axes: the training objective—what the model learns (SFT, DPO, RLHF, GRPO, ORPO, SimPO, and KTO)—and the parameter-update strategy—how weights are modified (Full Fine-Tuning, LoRA, QLoRA, DoRA, GaLore, Spectrum). We trace the theoretical evolution from classical RLHF with its three-model PPO pipeline, through the DPO reparameterization that collapsed preference learning into a single supervised objective, to the reasoning-focused GRPO/RLVR paradigm that enabled DeepSeek-R1 to achieve 71.0% Pass@1 on AIME 2024 through emergent reasoning without supervised reasoning traces. The paper further provides rigorous treatment of model merging techniques (TIES, DARE, and SLERP) that compose capabilities in weight space without gradient computation, knowledge distillation methods including cross-tokenizer and comparative approaches, Mixture-of-Experts fine-tuning with sparse routing, multimodal adaptation of vision-language models, and the modern framework ecosystem. We present formal loss functions, convergence properties, and computational complexity analysis for each method, accompanied by empirical benchmark comparisons. The survey concludes with a unified taxonomy and actionable guidance for selecting technique combinations based on compute budget, data availability, and task requirements.