sgl-project/sglang-omni: SGLang Omni: High-Performance Multi-Stage Pipeline Framework for Omni Models (opens in new tab)  🗄️Database Sharding  Content type: Code

SGLang Omni: High-Performance Multi-Stage Pipeline Framework for Omni Models About SGLang-Omni is a high-performance serving framework for omni and multimodal models, built on top of Modern omni models — such as speech-output LLMs and multimodal generation systems — decompose into heterogeneous stages with fundamentally different computational profiles: a compute-bound thinker, a memory-bound talker, a latency-sensitive codec. SGLang-Omni is built around a computation-centric design: each sta...