96. LoRA: Fine-Tune a Billion-Parameter Model on a Laptop (opens in new tab)

GPT-2 has 117M parameters. LLaMA-2 has 7B. GPT-3 has 175B. Full fine-tuning means updating every single parameter. For GPT-2 that's manageable. For LLaMA-2 it needs 28GB of GPU memory just to store the gradients. For GPT-3 it's basically impossible without a cluster. LoRA (Low-Rank Adaptation) solves this. Instead of updating the full weight matrices, it adds tiny trainable modules next to them. The original weights stay frozen. Only the tiny modules train. At the end you merge them back. You...