LLM Inference

Feeds to Scour
SubscribedAll
Scoured 188 posts in 8.1 ms

The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking

 ⚡Inference Optimization
edn.com·

Xiaomi MiMo-V2.5-Pro Just Hit 1,000 Tokens Per Second!

 ⚡Inference Optimization
gizchina.com·

147th airhacks tv: Local LLMs, LightMetal, ZSmith Agents, AI Rails, Saving Tokens

 🔄Agentic Systems  Content type: Blog
adambien.blog·

Running LLM Inference on Kubernetes: What It Actually Takes

 ⚙️AI Infrastructure  Content type: Blog
fairwinds.com·

MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 TPS

 ⚡Inference Optimization  Content type: Blog
mimo.xiaomi.com··Hacker News, r/LocalLLaMA

Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation

 ⚡Inference Optimization  Content type: Academic
arxiv.org·

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

 ⚡Inference Optimization  Content type: News  Content type: Blog
blog.google··Hacker News

The economics of speculative decoding

 ⚡Inference Optimization  Content type: Blog
fergusfinn.com··Hacker News
Less-relevant results

The Bill Arrives: How to Manage Agentic AI Costs at Scale

 🔄Agentic Systems  Content type: Blog
cockroachlabs.com·

BeeLlama.cpp DFlash on Strix Halo: 2.7x Gemma 31B, But MTP Is Still Faster

 ⚡Inference Optimization
sleepingrobots.com·

Two Leaps to 1000 Tokens/s on a 1T-Parameter Model: On Inference Systems, Execution Boundaries, and Co-Design

 🔄Agentic Systems  Content type: Blog
tilert.ai··Hacker News

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

 ⚡Inference Optimization  Content type: Code
github.com··Hacker News

DiffusionGemma: The Developer Guide

 🎯Post-Training  Content type: Blog
developers.googleblog.com·

Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and 4-Bit Shouldn’t Be This Good

 ⚡Inference Optimization  Content type: Blog
towardsai.net·

MLPerf and the rise of latency-aware LLM benchmarking

 ⚡Inference Optimization
edn.com·

Infrastructure Options for Scalable AI Inference

 ⚙️AI Infrastructure  Content type: Blog
mirantis.com·

google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation

 ⚙️AI Infrastructure
huggingface.co··r/LocalLLaMA

Show HN: Ext-Infer

 ⚡Inference Optimization
infer.displace.tech··Hacker News

🇳🇱 Go/Golang job: Senior Backend Engineer (Go) | Studio AI at Creative Fabrica (Amsterdam, Netherlands)

 ⚙️AI Infrastructure
golangprojects.com·

DiffusionGemma: 4x Faster Text Generation

 ⚙️AI Infrastructure  Content type: News  Content type: Blog
blog.google··Hacker News, r/LocalLLaMA, r/singularity

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help