Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Inference
⚡ Inference
Specific
LLM inference, vLLM, model serving, inference optimization
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
347
posts in
8.7
ms
harshuljain13/llm-inference-at-scale
: A Practitioner handbook for production
llm
serving
.
🧠
LLMs
Content type:
Code
github.com
·
3d
3 days ago
·
Hacker News
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
🧠
LLMs
Content type:
News
newsletter.semianalysis.com
·
23h
23 hours ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
PagedAttention
vs Traditional
KV
Cache
: How vLLM Reinvented GPU Memory for LLM Inference
🧠
LLMs
Content type:
Blog
medium.com
·
1d
1 day ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
Alignment Collapse Under
KV
Cache
Quantization
: Diagnosis and Mitigation
🧠
LLMs
Content type:
Academic
arxiv.org
·
7h
7 hours ago
Actions for Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
2x GH200 for
LLM
inference
, Part 2:
vLLM
, DeepSeek V4 Flash, and MTP
🧠
LLMs
Content type:
Blog
dnhkng.github.io
·
2d
2 days ago
Actions for 2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP
Inferoa
AI harness claimed 90%
cache
savings. We ran it and measured 97.8%
🧠
LLMs
zozo123.github.io
·
1h
1 hour ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
LLM
Inference
Engineering Room — Part 3: The Orchestration Layer
🧠
LLMs
Content type:
Blog
vimal-dwarampudi.medium.com
·
6d
6 days ago
Actions for LLM Inference Engineering Room — Part 3: The Orchestration Layer
A system programmer’s guide to
LLM
inference
🧠
LLMs
Content type:
Blog
blog.xiangpeng.systems
·
2d
2 days ago
·
Hacker News
Actions for A system programmer’s guide to LLM inference
Big Blue’s Redbook on Storage Scale
KV
Cache
management
🧠
LLMs
Content type:
News
blocksandfiles.com
·
18h
18 hours ago
Actions for Big Blue’s Redbook on Storage Scale KV Cache management
The hidden bottleneck in
LLM
inference
and the impact on MLPerf benchmarking
🧠
LLMs
edn.com
·
6d
6 days ago
Actions for The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking
Train
Models
Faster with JAX and MaxText Using NVFP4 on NVIDIA Blackwell
🏆
SOTA Models
Content type:
News
Content type:
Blog
developer.nvidia.com
·
1d
1 day ago
Actions for Train Models Faster with JAX and MaxText Using NVFP4 on NVIDIA Blackwell
Anatomy of a high-performance EP kernel
🧠
LLMs
Content type:
Blog
fergusfinn.com
·
11h
11 hours ago
Actions for Anatomy of a high-performance EP kernel
lightmetal: GPU
LLM
Inference
From a Single Java 25 JAR
🧠
LLMs
Content type:
Blog
adambien.blog
·
1d
1 day ago
Actions for lightmetal: GPU LLM Inference From a Single Java 25 JAR
A visionary legacy: Andy Brown
🕸️
Distributed Systems
uk.themedialeader.com
·
7h
7 hours ago
Actions for A visionary legacy: Andy Brown
Nvidia DGX Spark GB10 – AI
Models
and Guide with
vLLM
and Autonomous Script
🧠
LLMs
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script
Two Leaps to 1000 Tokens/s on a 1T-Parameter
Model
: On
Inference
Systems, Execution Boundaries, and Co-Design
🧠
LLMs
Content type:
Blog
tilert.ai
·
1d
1 day ago
·
Hacker News
Actions for Two Leaps to 1000 Tokens/s on a 1T-Parameter Model: On Inference Systems, Execution Boundaries, and Co-Design
Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
🎛️
Fine-tuning
aarushgupta.io
·
16h
16 hours ago
·
Lobsters
,
Hacker News
Actions for Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
Running
LLM
Inference
on Kubernetes: What It Actually Takes
🧠
LLMs
Content type:
Blog
fairwinds.com
·
4d
4 days ago
Actions for Running LLM Inference on Kubernetes: What It Actually Takes
Report: GKE
Inference
Gateway delivers up to 92% faster AI responses
🧠
LLMs
Content type:
Blog
cloud.google.com
·
1d
1 day ago
·
Hacker News
Actions for Report: GKE Inference Gateway delivers up to 92% faster AI responses
How we fight GPU scarcity without compromise
🧠
LLMs
Content type:
Blog
equixly.com
·
5d
5 days ago
·
Hacker News
Actions for How we fight GPU scarcity without compromise
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help