⚡ Continuous Batching · Scour

⚡ Continuous BatchingSpecific

dynamic batching, vLLM, inference throughput, iteration-level scheduling

mstar.stanford.edu·

M* (M-Star): A Modular, Extensible, Serving System for Multimodal Models

Discussed on Hacker News

Achieve state-of-the-art inference latencies with speculative decoding

Covers DFlash: Block Diffusion for Flash Speculative Decoding

baidu/Unlimited-OCR

Covered by 5 sources including The Rundown AI, VentureBeat

We built the fastest API for GLM-5.2 (280 TPS)

Covers GLM-5.2 (6 minute read)

Discussed on Hacker News

Monitor and debug generative AI inference with SageMaker detailed metrics and Insights dashboard on CloudWatch

fitservers.com·

The Production-Ready Guide to Self-Hosting LLaMA 3 on a GPU Dedicated Server

ByteByteGo Newsletter·

Large Language Models vs Small Language Models

Covers 6 stories including Attention is all you need (2017)

For users with 4x-8x 6000 PROs, how is your experience with bigger models lately? (GLM 5.2, Kimi 2.7, DeepSeek V4 Pro)

Discussed on r/LocalLLaMA

Developing web apps with local LLM inference

blog.chuanxilu.net·

An analysis on why LLMs perform bad on long loop tasks

Covers 2 stories including Lost in the Middle: How Language Models Use Long Contexts

Discussed on Hacker News

·

HPE Discover 2026: Lessons in optimizing AI economics

·

Debugging Deployments with Gemma 12B, TPU v6e-1, MCP, and Antigravity CLI

notes.dsebastien.net·

GLM-5.2

Covers 6 stories including zai-org/GLM-5.2 is here!

Covered by Sébastien Dubois

graphsignal.com·

CUDA Profiler for Production Inference

Discussed on Hacker News

Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG

·

vLLM, Function Calling, and World Models explained

Red Hat Developer·

Designing distributed AI inference: Core concepts and scaling dimensions

not much happened today | AINews

Covers 5 stories including Bernie Sanders unveils $7 trillion plan to give Americans control of AI industry

AI inference provider Baseten reportedly raising $1.5B in funding

Rack to Cloud·

GPU Scarcity Isn't the Problem Anymore. GPU Allocation Governance Is.

Discussed on DEV

Log in to enable infinite scrolling