Deploying Gemma 4 26B on Proxmox: IaC Setup with Terraform, Ansible & AMD iGPU (opens in new tab)

Originally published at woitzik.dev Running large language models (LLMs) like Gemma 4 26B locally usually requires massive Nvidia clusters. But what if you want to run it in a home lab or a constrained edge environment using Infrastructure as Code (IaC)? In this guide, I will show you how to automate a complete local AI stack on Proxmox VE using Terraform for the infrastructure and Ansible for provisioning. We will cover the quirks of the Proxmox Terraform provider, setting up Ollama, and dep...