How NetEase Games achieved 30-second LLM cold starts on Kubernetes (opens in new tab)

At NetEase Games, we learned a hard lesson about large language model (LLM) inference in production: elastic compute is only useful if data can move just as fast. “Elastic compute is only useful if data can...