https://www.together.ai/blog/flexgen-high-throughput-generative-inference-of-large-language-models-with-a-single-gpu (opens in new tab)

--- title: FlexGen: High-throughput generative inference of large language models with a single GPU --- ⚡️ FlashAttention-4: up to 1.3× faster than cuDNN on NVIDIA Blackwell → Introducing Together AI's new look → 🔎 ATLAS: runtime-learning accelerators delivering up to 4x faster LLM inference → ⚡ Together GPU Clusters: self-service NVIDIA GPUs, now generally available → 📦 Batch Inference API: Process billions of tokens at 50% lower cost for most models → 🪛 Fine-Tuning Platform Upgrades: ...