Running Gemma 4 26B on an Old GTX 1080 with llama.cpp (opens in new tab)

How to get Google's Gemma 4 26B-A4B Mixture-of-Experts model running locally — including speculative decoding — on hardware that has no business running it. Google's Gemma 4 26B-A4B is a Mixture-of-Experts model: 25.2 billion total parameters, but only 3.8 billion are active per token. That distinction matters enormously for running it locally, because it means you can keep the cold expert weights in system RAM and stream them over PCIe, while a much smaller working set lives on the GPU. This...