Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
LLM Inference
🤖 LLM Inference
Specific
Model Serving, Quantization, vLLM, ONNX Runtime
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
187
posts in
4.3
ms
Nvidia DGX Spark GB10 – AI
Models
and Guide with
vLLM
and Autonomous Script
⚙️
AI Infrastructure
Content type:
Code
github.com
·
5d
5 days ago
·
Hacker News
Actions for Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script
lightmetal: GPU
LLM
Inference
From a Single Java 25 JAR
⚡
Inference Optimization
Content type:
Blog
adambien.blog
·
1d
1 day ago
Actions for lightmetal: GPU LLM Inference From a Single Java 25 JAR
Quality Is Not a Safety Proxy Under
Quantization
⚡
Inference Optimization
Content type:
Academic
arxiv.org
·
18h
18 hours ago
Actions for Quality Is Not a Safety Proxy Under Quantization
PagedAttention
vs Traditional KV Cache: How
vLLM
Reinvented GPU Memory for
LLM
Inference
⚙️
AI Infrastructure
Content type:
Blog
medium.com
·
2d
2 days ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
Inferoa
AI harness claimed 90% cache savings. We ran it and measured 97.8%
⚙️
AI Infrastructure
zozo123.github.io
·
11h
11 hours ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
GGUF vs
GPTQ
vs
AWQ
: The Plain-English Guide to
LLM
Quantization (and Which One to Pick)
⚡
Inference Optimization
vettedconsumer.com
·
4d
4 days ago
·
Hacker News
Actions for GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)
AI
Serving
Platform That Adapts to Your
Model
⚙️
AI Infrastructure
Content type:
Blog
databricks.com
·
6h
6 hours ago
Actions for AI Serving Platform That Adapts to Your Model
A system programmer’s guide to
LLM
inference
⚡
Inference Optimization
Content type:
Blog
blog.xiangpeng.systems
·
2d
2 days ago
·
Hacker News
Actions for A system programmer’s guide to LLM inference
NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
⚙️
AI Infrastructure
Content type:
Blog
blogs.nvidia.com
·
5h
5 hours ago
Actions for NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
Speculators
v0.5.0: DFlash support and online training
⚡
Inference Optimization
developers.redhat.com
·
6d
6 days ago
Actions for Speculators v0.5.0: DFlash support and online training
Anatomy of a high-performance EP kernel
⚙️
AI Infrastructure
Content type:
Blog
fergusfinn.com
·
22h
22 hours ago
·
Hacker News
Actions for Anatomy of a high-performance EP kernel
Qwen 3.6 27B AutoRound GGUF, need your feedback
⚡
Inference Optimization
huggingface.co
·
1d
1 day ago
·
r/LocalLLaMA
Actions for Qwen 3.6 27B AutoRound GGUF, need your feedback
AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support
⚙️
AI Infrastructure
phoronix.com
·
5h
5 hours ago
Actions for AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
⚡
Inference Optimization
Content type:
News
newsletter.semianalysis.com
·
1d
1 day ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
Pruned YOLOv8
ONNX
INT8 Fails: 3 Fixes That Work
⚡
Inference Optimization
Content type:
Blog
Content type:
Discussion
tildalice.io
·
5d
5 days ago
Actions for Pruned YOLOv8 ONNX INT8 Fails: 3 Fixes That Work
MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
⚡
Inference Optimization
Content type:
News
Content type:
Blog
kaitchup.substack.com
·
5d
5 days ago
·
r/LocalLLaMA
Actions for MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
Here's a
llama.cpp
CLI Command builder.
🎯
Post-Training
llamabuilding.com
·
1d
1 day ago
·
r/LocalLLaMA
Actions for Here's a llama.cpp CLI Command builder.
China's Xiaomi MiMo Is Now 15X Faster Than ChatGPT and Claude (4 minute read)
⚡
Inference Optimization
Content type:
News
decrypt.co
·
2d
2 days ago
Actions for China's Xiaomi MiMo Is Now 15X Faster Than ChatGPT and Claude (4 minute read)
2x GH200 for
LLM
inference
, Part 2:
vLLM
, DeepSeek V4 Flash, and MTP
⚙️
AI Infrastructure
Content type:
Blog
dnhkng.github.io
·
3d
3 days ago
Actions for 2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP
Xiaomi MiMo-V2.5-Pro Just Hit 1,000 Tokens Per Second!
⚡
Inference Optimization
gizchina.com
·
1d
1 day ago
Actions for Xiaomi MiMo-V2.5-Pro Just Hit 1,000 Tokens Per Second!
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help