Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Inference
⚡ Inference
LLM inference, model serving, vLLM, TensorRT, latency
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
317
posts in
7.6
ms
harshuljain13/llm-inference-at-scale
: A Practitioner handbook for production
llm
serving
.
🧠
LLMs
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
A system programmer’s guide to
LLM
inference
🧠
LLMs
Content type:
Blog
blog.xiangpeng.systems
·
2d
2 days ago
·
Hacker News
Actions for A system programmer’s guide to LLM inference
Alignment Collapse Under
KV
Cache
Quantization
: Diagnosis and Mitigation
🧠
LLMs
Content type:
Academic
arxiv.org
·
16h
16 hours ago
Actions for Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
Inferoa
AI harness claimed 90%
cache
savings. We ran it and measured 97.8%
🧠
LLMs
zozo123.github.io
·
9h
9 hours ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
Qwen 3.6 27B AutoRound GGUF, need your feedback
🧠
LLMs
huggingface.co
·
1d
1 day ago
·
r/LocalLLaMA
Actions for Qwen 3.6 27B AutoRound GGUF, need your feedback
Gemma 4 QAT
models
: Optimizing model compression for mobile and laptop efficiency
🧠
LLMs
Content type:
News
Content type:
Blog
blog.google
·
5d
5 days ago
·
Hacker News
Actions for Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency
Big Blue’s Redbook on Storage Scale
KV
Cache
management
🤖
AI Agents
Content type:
News
blocksandfiles.com
·
1d
1 day ago
Actions for Big Blue’s Redbook on Storage Scale KV Cache management
DiffusionGemma: The Developer Guide
🤖
AI
Content type:
Blog
developers.googleblog.com
·
20h
20 hours ago
Actions for DiffusionGemma: The Developer Guide
Speculators
v0.5.0: DFlash support and online training
🧠
LLMs
developers.redhat.com
·
6d
6 days ago
Actions for Speculators v0.5.0: DFlash support and online training
2x GH200 for
LLM
inference
, Part 2:
vLLM
, DeepSeek V4 Flash, and MTP
🧠
LLMs
Content type:
Blog
dnhkng.github.io
·
2d
2 days ago
Actions for 2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP
The
Inference
Alpha: Maximizing Frontier
Models
on AMD
🧠
LLMs
Content type:
Blog
digitalocean.com
·
6h
6 hours ago
Actions for The Inference Alpha: Maximizing Frontier Models on AMD
The economics of
speculative
decoding
🧠
LLMs
Content type:
Blog
fergusfinn.com
·
2d
2 days ago
·
Hacker News
Actions for The economics of speculative decoding
LLM
Inference
Engineering Room — Part 3: The Orchestration Layer
🧠
LLMs
Content type:
Blog
vimal-dwarampudi.medium.com
·
6d
6 days ago
Actions for LLM Inference Engineering Room — Part 3: The Orchestration Layer
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
🧠
LLMs
Content type:
News
newsletter.semianalysis.com
·
1d
1 day ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
How we fight GPU scarcity without compromise
🧠
LLMs
Content type:
Blog
equixly.com
·
5d
5 days ago
·
Hacker News
Actions for How we fight GPU scarcity without compromise
HNSW vs LSH: How Elasticsearch hits 0.99 recall@10 at 15,000 QPS — and what it costs
🔍
Vector DBs
Content type:
Blog
elastic.co
·
1d
1 day ago
Actions for HNSW vs LSH: How Elasticsearch hits 0.99 recall@10 at 15,000 QPS — and what it costs
AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support
🧠
LLMs
phoronix.com
·
4h
4 hours ago
Actions for AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support
PagedAttention vs Traditional
KV
Cache
: How
vLLM
Reinvented GPU Memory for LLM Inference
🧠
LLMs
Content type:
Blog
medium.com
·
2d
2 days ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
🤖
AI
Content type:
News
Content type:
Blog
kaitchup.substack.com
·
4d
4 days ago
·
r/LocalLLaMA
Actions for MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
🤖
AI
Content type:
Blog
blogs.nvidia.com
·
4h
4 hours ago
Actions for NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help