ARCHIVES

1. [ref:lora] Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2022). “LoRA: Low-Rank Adaptation of Large Language Models.” ICLR 2022. https://arxiv.org/abs/2106.09685
2. [ref:qlora] Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. (2023). “QLoRA: Efficient Finetuning of Quantized Language Models.” NeurIPS 2023. https://arxiv.org/abs/2305.14314
3. [ref:dpo] Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. (2023). “Direct Preference Optimization: Your Language Model is Secretly a Reward Model.” NeurIPS 2023. https://arxiv.org/abs/2305.18290
4. [ref:deepseek] DeepSeek-AI. (2025). “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.” Nature. https://arxiv.org/abs/2501.12948
5. [ref:dora] Liu, S.-Y., Wang, C.-Y., Yin, H., Molchanov, P., Wang, Y.-C. F., Cheng, K.-T., and Chen, M.-H. (2024). “DoRA: Weight-Decomposed Low-Rank Adaptation.” ICLR 2025. https://arxiv.org/abs/2402.09353
6. [ref:galore] Zhao, J., Zhang, Z., Chen, B., Wang, Z., Anandkumar, A., and Tian, Y. (2024). “GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection.” ICML 2024. https://arxiv.org/abs/2403.03507
7. [ref:orpo] Hong, J., Lee, N., and Thorne, J. (2024). “ORPO: Monolithic Preference Optimization without Reference Model.” https://arxiv.org/abs/2403.07691
8. [ref:kto] Ethayarajh, K., Xu, W., Muennighoff, N., Jurafsky, D., and Kiela, D. (2024). “KTO: Model Alignment as Prospect Theoretic Optimization.” https://arxiv.org/abs/2402.01306
9. [ref:simpo] Meng, Y., Xia, M., and Chen, D. (2024). “SimPO: Simple Preference Optimization with a Reference-Free Reward.” NeurIPS 2024. https://arxiv.org/abs/2405.14734


10. [ref:neftune] Jain, N., Chiang, P.-y., Wen, Y., Kirchenbauer, J., Chu, H.-M., Somepalli, G., Bartoldson, B. R., Kailkhura, B., Schwarzschild, A., Saha, A., Goldblum, M., Geiping, J., and Goldstein, T. (2024). “NEFTune: Noisy Embeddings Improve Instruction Finetuning.” ICLR 2024. https://arxiv.org/abs/2310.05914
11. [ref:dare] Yu, L., Yu, B., Yu, H., Huang, F., and Li, Y. (2024). “Language Model Surgery: Efficient Knowledge Unlearning and Editing via Weight Disentanglement.” https://arxiv.org/abs/2311.03099
12. [ref:ties] Yadav, P., Tam, D., Choshen, L., Raffel, C., and Bansal, M. (2023). “TIES-Merging: Resolving Interference When Merging Models.” NeurIPS 2023. https://arxiv.org/abs/2306.01708
13. [ref:dapo] Yu, Q., et al. (2025). “DAPO: An Open-Source LLM Reinforcement Learning System.” https://arxiv.org/abs/2503.14476
14. [ref:instructgpt] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022). “Training Language Models to Follow Instructions with Human Feedback.” NeurIPS 2022. https://arxiv.org/abs/2203.02155
15. [ref:ppo] Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). “Proximal Policy Optimization Algorithms.” https://arxiv.org/abs/1707.06347
16. [ref:selfinstruct] Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., and Hajishirzi, H. (2023). “Self-Instruct: Aligning Language Models with Self-Generated Instructions.” ACL 2023. https://arxiv.org/abs/2212.10560
17. [ref:lima] Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., Zhang, S., Ghosh, G., Lewis, M., Zettlemoyer, L., and Levy, O. (2023). “LIMA: Less Is More for Alignment.” NeurIPS 2023. https://arxiv.org/abs/2305.11206
18. HuggingFace PEFT Library. https://github.com/huggingface/peft
19. Axolotl Fine-Tuning Framework. https://github.com/axolotl-ai-cloud/axolotl
20. Unsloth. https://github.com/unslothai/unsloth
21. LLaMA-Factory. https://github.com/hiyouga/LLaMA-Factory
22. TRL: Transformer Reinforcement Learning. https://github.com/huggingface/trl
23. MergeKit. https://github.com/arcee-ai/mergekit
24. Ollama. https://github.com/ollama/ollama
25. TorchTune. https://github.com/pytorch/torchtune