Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Inference
⚙️ Inference
model inference, serving, quantization, throughput, vLLM
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
347
posts in
9.7
ms
harshuljain13/llm-inference-at-scale
: A Practitioner handbook for production
llm
serving
.
🧠
AI
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
🧠
AI
Content type:
News
newsletter.semianalysis.com
·
1d
1 day ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
Alignment Collapse Under KV Cache
Quantization
: Diagnosis and Mitigation
📄
AI Papers
Content type:
Academic
arxiv.org
·
16h
16 hours ago
Actions for Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
AI
Serving
Platform That Adapts to Your
Model
🚀
MLOps
Content type:
Blog
databricks.com
·
4h
4 hours ago
Actions for AI Serving Platform That Adapts to Your Model
A system programmer’s guide to
LLM
inference
💬
LLMs
Content type:
Blog
blog.xiangpeng.systems
·
2d
2 days ago
·
Hacker News
Actions for A system programmer’s guide to LLM inference
Inferoa
AI harness claimed 90% cache savings. We ran it and measured 97.8%
📊
AI Evals
zozo123.github.io
·
9h
9 hours ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
Running
LLM
Inference
on Kubernetes: What It Actually Takes
🧠
AI
Content type:
Blog
fairwinds.com
·
5d
5 days ago
Actions for Running LLM Inference on Kubernetes: What It Actually Takes
Infrastructure Options for Scalable AI
Inference
📊
AI Evals
Content type:
Blog
mirantis.com
·
21h
21 hours ago
Actions for Infrastructure Options for Scalable AI Inference
2x GH200 for
LLM
inference
, Part 2:
vLLM
, DeepSeek V4 Flash, and MTP
🚀
MLOps
Content type:
Blog
dnhkng.github.io
·
2d
2 days ago
Actions for 2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP
The
Inference
Alpha: Maximizing Frontier
Models
on AMD
⚡
Transformers
Content type:
Blog
digitalocean.com
·
6h
6 hours ago
Actions for The Inference Alpha: Maximizing Frontier Models on AMD
LLM
Inference
Engineering
Room — Part 3: The Orchestration Layer
💬
LLMs
Content type:
Blog
vimal-dwarampudi.medium.com
·
6d
6 days ago
Actions for LLM Inference Engineering Room — Part 3: The Orchestration Layer
lightmetal: GPU
LLM
Inference
From a Single Java 25 JAR
🚀
MLOps
Content type:
Blog
adambien.blog
·
1d
1 day ago
Actions for lightmetal: GPU LLM Inference From a Single Java 25 JAR
NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
🧠
AI
Content type:
Blog
blogs.nvidia.com
·
4h
4 hours ago
Actions for NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
Anatomy of a high-performance EP kernel
🎯
Fine-Tuning
Content type:
Blog
fergusfinn.com
·
20h
20 hours ago
·
Hacker News
Actions for Anatomy of a high-performance EP kernel
From GPU to Token: The 8-Layer Observability Stack for AI Infrastructure
🔀
LoRA
Content type:
Blog
jimmysong.io
·
1d
1 day ago
Actions for From GPU to Token: The 8-Layer Observability Stack for AI Infrastructure
The hidden bottleneck in
LLM
inference
and the impact on MLPerf benchmarking
💬
LLMs
edn.com
·
6d
6 days ago
Actions for The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking
PagedAttention vs Traditional KV Cache: How
vLLM
Reinvented GPU Memory for
LLM
Inference
🔀
LoRA
Content type:
Blog
medium.com
·
2d
2 days ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
DiffusionGemma: The Developer Guide- Google Developers Blog
🧠
AI
Content type:
Blog
developers.googleblog.com
·
20h
20 hours ago
·
r/LocalLLaMA
Actions for DiffusionGemma: The Developer Guide- Google Developers Blog
MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
🧠
AI
Content type:
News
Content type:
Blog
kaitchup.substack.com
·
4d
4 days ago
·
r/LocalLLaMA
Actions for MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
Qwen 3.6 27B AutoRound GGUF, need your feedback
🎯
Fine-Tuning
huggingface.co
·
1d
1 day ago
·
r/LocalLLaMA
Actions for Qwen 3.6 27B AutoRound GGUF, need your feedback
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help