First Gemma 4 ExecuTorch Deployment on Raspberry Pi 5 — and Why It's 7.7 Slower Than llama.cpp (opens in new tab)

On April 2, ARM published a blog post announcing Gemma 4 optimised for ARM devices via XNNPACK + KleidiAI, reporting 5.5× prefill speedup and 1.6× faster decode. Those numbers target Armv9 chips with SME2 — flagship phone silicon. I wanted to see what happens on the broader ARM ecosystem. So I took Gemma 4 E2B through the full PyTorch edge deployment pipeline — torch.export → torchao quantization (INT8 dynamic activations + INT4 weights) → ExecuTorch XNNPACK backend → KleidiAI — and deployed ...