Kog hits 3K t/s on MI300X, no kernel switches — test it now (opens in new tab)

AMD's MI300X has long had more single-request inference headroom than the default ROCm stack exposes. A Paris startup just showed how much — by deleting the per-token kernel launch entirely. How the monokernel eliminates kernel-launch overhead A monokernel is a single, persistent GPU-resident program that runs an entire LLM decode pass — prefill, decode, LM-head sampling, and the EOS stop check — without returning to the host CPU or launching a new kernel per token. Kog AI reports 3,000+ outp...