Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Prefill Decoding
⏱️ Prefill Decoding
Specific
prefill phase, decode phase, chunked prefill, time-to-first-token
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
180
posts in
11.9
ms
harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
💾
KV Cache
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
Inferoa AI harness claimed 90%
cache
savings. We ran it and measured 97.8%
💾
KV Cache
zozo123.github.io
·
7h
7 hours ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
PagedAttention
vs Traditional
KV
Cache
: How vLLM Reinvented GPU Memory for LLM Inference
🧠
Inference Engineering
Content type:
Blog
medium.com
·
2d
2 days ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
Fairness-Aware and
Latency-Controllable
Scheduling for
Chunked-Prefill
LLM Serving
🟢
CUDA
Content type:
Academic
arxiv.org
·
1d
1 day ago
Actions for Fairness-Aware and Latency-Controllable Scheduling for Chunked-Prefill LLM Serving
The Inference Alpha: Maximizing Frontier Models on AMD
🧠
Inference Engineering
Content type:
Blog
digitalocean.com
·
3h
3 hours ago
Actions for The Inference Alpha: Maximizing Frontier Models on AMD
How we fight GPU scarcity without compromise
💾
KV Cache
Content type:
Blog
equixly.com
·
5d
5 days ago
·
Hacker News
Actions for How we fight GPU scarcity without compromise
Big Blue’s Redbook on Storage Scale
KV
Cache
management
🧠
Inference Engineering
Content type:
News
blocksandfiles.com
·
1d
1 day ago
Actions for Big Blue’s Redbook on Storage Scale KV Cache management
DiffusionGemma: The Developer Guide
🧠
Inference Engineering
Content type:
Blog
developers.googleblog.com
·
18h
18 hours ago
Actions for DiffusionGemma: The Developer Guide
Anatomy of a high-performance EP kernel
💰
Inference Cost
Content type:
Blog
fergusfinn.com
·
18h
18 hours ago
·
Hacker News
Actions for Anatomy of a high-performance EP kernel
The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking
💰
Inference Cost
edn.com
·
6d
6 days ago
Actions for The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over
Time
- Huawei, GB300 NVL72, MI355X, B200
🧠
Inference Engineering
Content type:
News
newsletter.semianalysis.com
·
1d
1 day ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
🎮
GPU Computing
Content type:
Blog
blogs.nvidia.com
·
1h
1 hour ago
Actions for NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
Making Local LLM Go Brrr
💾
KV Cache
seanpedersen.github.io
·
6d
6 days ago
Actions for Making Local LLM Go Brrr
Report: GKE Inference Gateway delivers up to 92% faster AI responses
🧠
Inference Engineering
Content type:
Blog
cloud.google.com
·
1d
1 day ago
·
Hacker News
Actions for Report: GKE Inference Gateway delivers up to 92% faster AI responses
1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM
🎮
GPU Computing
smolhub.com
·
2d
2 days ago
·
r/LocalLLaMA
Actions for 1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM
How to Measure
Time
To First
Token
(TTFT) in AI Systems
🧠
Inference Engineering
qainsights.com
·
3d
3 days ago
·
Hacker News
Actions for How to Measure Time To First Token (TTFT) in AI Systems
libertywing/FlashMemory-Deepseek-V4: FlashMemory DS-V4 Retriever: a lightweight retriever that sparsifies DeepSeek-V4 CSA
KV-cache
. Weights available on Hugging Face.
🧠
Inference Engineering
Content type:
Code
github.com
·
18h
18 hours ago
Actions for libertywing/FlashMemory-Deepseek-V4: FlashMemory DS-V4 Retriever: a lightweight retriever that sparsifies DeepSeek-V4 CSA KV-cache. Weights available on Hugging Face.
2x GH200 for LLM inference, Part 2:
vLLM
, DeepSeek V4 Flash, and MTP
🧠
Inference Engineering
Content type:
Blog
dnhkng.github.io
·
2d
2 days ago
Actions for 2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP
RKSC: Reasoning-Aware
KV
Cache
Sharing and Confident Early Exit for Multi-Step LLM Inference
💾
KV Cache
Content type:
Academic
arxiv.org
·
14h
14 hours ago
Actions for RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference
Speculators v0.5.0: DFlash support and online training
🚀
Speculative Decoding
developers.redhat.com
·
6d
6 days ago
Actions for Speculators v0.5.0: DFlash support and online training
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help