QLoRA: Fine-Tuning a 7B Model on a 16GB GPU (It Shrank to 5.4GB in Front of Me)

Series — Fine-Tuning, Smallest to Largest: QLoRA (7B) ← you are here In , LoRA let me fine-tune a 1.5B model by freezing it and training tiny adapters. But the frozen base still sat in memory in 16-bit (~3GB). Now I wanted to go to Qwen2.5-7B — and hit a wall that LoRA alone doesn’t solve. The probl…

View Post on Scour ↗

More from kaggle.com on Scour.

Discussed on DEV