Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
LLM Inference
🤖 LLM Inference
Specific
Model Serving, Quantization, vLLM, ONNX Runtime
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
203
posts in
6.8
ms
harshuljain13/llm-inference-at-scale
: A Practitioner handbook for production
llm
serving
.
⚡
Vllm
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
Big Blue’s Redbook on Storage Scale
KV
Cache
management
⚡
Vllm
Content type:
News
blocksandfiles.com
·
1d
1 day ago
Actions for Big Blue’s Redbook on Storage Scale KV Cache management
RKSC: Reasoning-Aware
KV
Cache
Sharing and Confident Early Exit for Multi-Step
LLM
Inference
🤖
LLM
Content type:
Academic
arxiv.org
·
20h
20 hours ago
Actions for RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference
PagedAttention
vs Traditional
KV
Cache
: How vLLM Reinvented GPU Memory for LLM Inference
⚡
Vllm
Content type:
Blog
medium.com
·
2d
2 days ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
Inferoa
AI harness claimed 90%
cache
savings. We ran it and measured 97.8%
🤖
LLM
zozo123.github.io
·
14h
14 hours ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
2x GH200 for
LLM
inference
, Part 2:
vLLM
, DeepSeek V4 Flash, and MTP
🤖
LLM
Content type:
Blog
dnhkng.github.io
·
3d
3 days ago
Actions for 2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
🤖
LLM
Content type:
News
newsletter.semianalysis.com
·
1d
1 day ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
How we fight GPU scarcity without compromise
🤖
LLM
Content type:
Blog
equixly.com
·
5d
5 days ago
·
Hacker News
Actions for How we fight GPU scarcity without compromise
google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation
🤖
LLM
huggingface.co
·
2d
2 days ago
·
r/LocalLLaMA
Actions for google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation
Less-relevant results
The
Inference
Alpha: Maximizing Frontier
Models
on AMD
🤖
LLM
Content type:
Blog
digitalocean.com
·
10h
10 hours ago
Actions for The Inference Alpha: Maximizing Frontier Models on AMD
The economics of
speculative
decoding
⚡
Vllm
Content type:
Blog
fergusfinn.com
·
3d
3 days ago
·
Hacker News
Actions for The economics of speculative decoding
Here's a
llama.cpp
CLI Command builder.
🤖
LLM
llamabuilding.com
·
2d
2 days ago
·
r/LocalLLaMA
Actions for Here's a llama.cpp CLI Command builder.
AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support
🤖
LLM
phoronix.com
·
8h
8 hours ago
Actions for AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support
A system programmer’s guide to
LLM
inference
⚡
Vllm
Content type:
Blog
blog.xiangpeng.systems
·
3d
3 days ago
·
Hacker News
Actions for A system programmer’s guide to LLM inference
DiffusionGemma: The Developer Guide
⚡
Vllm
Content type:
Blog
developers.googleblog.com
·
1d
1 day ago
Actions for DiffusionGemma: The Developer Guide
Running Qwen 35B MoE at 450k Context on a Single 32GB GPU
🤖
LLM
local-llm.utop.workers.dev
·
3d
3 days ago
·
Hacker News
Actions for Running Qwen 35B MoE at 450k Context on a Single 32GB GPU
KaiFelixBennett/gemma4-turboquant-rdna4:
Run
Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant
KV
cache
+ HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.
⚡
Vllm
Content type:
Code
github.com
·
8h
8 hours ago
·
Hacker News
Actions for KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.
OpenCV 5.0 Computer Vision Library Released with Rewritten DNN Engine
⚡
Vllm
linuxiac.com
·
2d
2 days ago
Actions for OpenCV 5.0 Computer Vision Library Released with Rewritten DNN Engine
Alignment Collapse Under
KV
Cache
Quantization
: Diagnosis and Mitigation
🤖
LLM
Content type:
Academic
arxiv.org
·
20h
20 hours ago
Actions for Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
The hidden bottleneck in
LLM
inference
and the impact on MLPerf benchmarking
⚡
Vllm
edn.com
·
6d
6 days ago
Actions for The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help