Deploying Disaggregated LLM Inference Workloads on Kubernetes (opens in new tab)

As large language model (LLM) inference workloads grow in complexity, a single monolithic serving process starts to hit its limits. Prefill and decode stages have fundamentally different compute…