Running Qwen 35B MoE at 450k Context on a Single 32GB GPU (opens in new tab) 🦙Ollama
A complete technical report on extreme LLM local inference using llama.cpp, TurboQuant, and YaRN scaling on a 32GB RTX 5090.
Read the original articleA complete technical report on extreme LLM local inference using llama.cpp, TurboQuant, and YaRN scaling on a 32GB RTX 5090.
Read the original article