Inference Serving

Feeds to Scour
SubscribedAll
Scoured 149 posts in 34.7 ms

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

 🧠LLM Inference  Content type: Code
github.com··Hacker News, r/LLM

MiniPIC: Flexible Position-Independent Caching in <100LOC

 💾Prompt Caching  Content type: Academic
arxiv.org·
Less-relevant results

detects when ML research consensus is shifting using Bayesian CUSUM

 🎓Advanced content
tattvaai.org··Hacker News

Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%

 💾Prompt Caching
zozo123.github.io··Hacker News

Unsloth Minimax M3 GGUF

 🤖AI
huggingface.co··r/LocalLLaMA

DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200

 ⚡Fast AI Inference  Content type: News
newsletter.semianalysis.com
··Hacker News·Cited by 1 article

Optimizing Local LLM Inference on Constrained Hardware

 🤖AI
pub.towardsai.net
·

DiffusionGemma: Discrete diffusion in a large language model

 🆕New AI
idlemachines.co.uk··Hacker News

DiffusionGemma: 4x Faster Text Generation

 🤖AI  Content type: News  Content type: Blog  21 articles covering this post
blog.google··Hacker News, r/LocalLLaMA, r/singularity·Cited by 21 articles

heterodoxin/graphkv: Graph-guided KV cache compression for memory-efficient LLM inference.

 💾Prompt Caching  Content type: Code
github.com··r/LocalLLaMA

google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation

 ⚡Fast AI Inference
huggingface.co··r/LocalLLaMA·Cited by 1 article

Why Transformer Models Get Costlier as Context Grows

 🧩MoE
siliconopera.com·

Making FlashAttention-4 faster for inference

 🤖AI  Content type: Blog
modal.com··Hacker News

RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference

 💾Prompt Caching  Content type: Academic
arxiv.org·

Modular: Day Zero: MiniMax M3 Open Weights on Modular Cloud

 🆕New AI  Content type: Blog
modular.com·

Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit

 🪄Prompt Engineering
venturebeat.com··r/LocalLLaMA

KJLdefeated/RL.cu: RLVR training for LLM in CUDA/C++

 🏗️LLM Infrastructure  Content type: Code
github.com··Hacker News

Mi50 32GB / GFX906 - vLLM Qwen 3.5 Configuration for Qwen 3.5:9B AWQ-4bit

 🤖AI
huggingface.co··r/LocalLLaMA

AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support

 🤖AI
phoronix.com··r/artificial·Cited by 1 article

Youssof Altoukhi (@Youssofal_)

 🤖AI
xcancel.com··r/LocalLLaMA

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help