Quantized LoRA Adapters for On-Device LLMs: Hot-Swapping Task-Specific Behaviors on Android Without Reloading the Base Model (opens in new tab)

--- title: "QLoRA Adapters on Android: Hot-Swap LLM Tasks in Under 100ms" published: true description: "Load a 4-bit quantized base model once on Android and hot-swap 2MB LoRA adapters for different tasks using llama.cpp, Kotlin, and NEON optimizations." tags: kotlin, android, architecture, mobile canonical_url: --- ## What We Will Build Let me show you a pattern I use in every project that runs on-device LLMs: loading a single 4-bit quantized base model into memory via `mmap`, then dynamical...