Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Inference Engineering
🖥️ Inference Engineering
Specific
LLM serving, latency, batching, throughput
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
190
posts in
6.3
ms
harshuljain13/llm-inference-at-scale
: A Practitioner handbook for production
llm
serving
.
🗄️
KV Cache
Content type:
Code
github.com
·
5d
5 days ago
·
Hacker News
,
r/LLM
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
Big Blue’s Redbook on Storage Scale
KV
Cache
management
🗄️
KV Cache
Content type:
News
blocksandfiles.com
·
1d
1 day ago
Actions for Big Blue’s Redbook on Storage Scale KV Cache management
AMD's Lemonade SDK For Local AI Adds NVIDIA
CUDA
Support
🗄️
KV Cache
phoronix.com
·
23h
23 hours ago
·
r/artificial
Actions for AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support
Alignment Collapse Under
KV
Cache
Quantization
: Diagnosis and Mitigation
🗄️
KV Cache
Content type:
Academic
arxiv.org
·
1d
1 day ago
Actions for Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
Google's new open
model
DiffusionGemma
generates
text
from noise instead of word by word
🎯
Fine-tuning
the-decoder.com
·
20h
20 hours ago
Actions for Google's new open model DiffusionGemma generates text from noise instead of word by word
How we fight
GPU
scarcity without compromise
🗄️
KV Cache
Content type:
Blog
equixly.com
·
6d
6 days ago
·
Hacker News
Actions for How we fight GPU scarcity without compromise
PagedAttention
vs Traditional
KV
Cache
: How vLLM Reinvented GPU Memory for LLM Inference
🗄️
KV Cache
Content type:
Blog
medium.com
·
2d
2 days ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
Intelligent
inference
scheduling with
llm-d
on Red Hat AI
🗄️
KV Cache
developers.redhat.com
·
16h
16 hours ago
Actions for Intelligent inference scheduling with llm-d on Red Hat AI
CommBench: Can LLMs Write Correct and Efficient
GPU
Communication Code?
🗄️
KV Cache
uccl-project.github.io
·
9h
9 hours ago
·
Hacker News
Actions for CommBench: Can LLMs Write Correct and Efficient GPU Communication Code?
Xiaomi MiMo-V2.5-Pro Just Hit 1,000
Tokens
Per
Second
!
💰
API Pricing
gizchina.com
·
2d
2 days ago
Actions for Xiaomi MiMo-V2.5-Pro Just Hit 1,000 Tokens Per Second!
2x GH200 for
LLM
inference
,
Part
2: vLLM, DeepSeek V4 Flash, and MTP
🗄️
KV Cache
Content type:
Blog
dnhkng.github.io
·
3d
3 days ago
Actions for 2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP
Google unveils DiffusionGemma, delivering up to 4x faster
inference
on dedicated GPUs
💰
Compute Costs
alternativeto.net
·
4h
4 hours ago
Actions for Google unveils DiffusionGemma, delivering up to 4x faster inference on dedicated GPUs
From
GPU
to
Token
: The 8-Layer Observability Stack for AI Infrastructure
🗄️
KV Cache
Content type:
Blog
jimmysong.io
·
2d
2 days ago
Actions for From GPU to Token: The 8-Layer Observability Stack for AI Infrastructure
NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
🎯
Fine-tuning
Content type:
Blog
blogs.nvidia.com
·
23h
23 hours ago
Actions for NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
GGUF vs GPTQ vs AWQ: The Plain-English Guide to
LLM
Quantization
(and Which One to Pick)
🎯
Fine-tuning
vettedconsumer.com
·
5d
5 days ago
·
Hacker News
Actions for GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)
DiffusionGemma 26B A4B results on my 5090
🗄️
KV Cache
huggingface.co
·
1d
1 day ago
·
r/LocalLLaMA
Actions for DiffusionGemma 26B A4B results on my 5090
Putting a datacenter
GPU
in a gaming PC for £200 ($268)
💰
Compute Costs
Content type:
Blog
blog.adafruit.com
·
21h
21 hours ago
Actions for Putting a datacenter GPU in a gaming PC for £200 ($268)
China's Xiaomi MiMo Is Now 15X Faster Than ChatGPT and Claude (4 minute read)
🗄️
KV Cache
Content type:
News
decrypt.co
·
2d
2 days ago
·
Hacker News
Actions for China's Xiaomi MiMo Is Now 15X Faster Than ChatGPT and Claude (4 minute read)
Running
LLM
Inference
on Kubernetes: What It Actually Takes
🗄️
KV Cache
Content type:
Blog
fairwinds.com
·
6d
6 days ago
Actions for Running LLM Inference on Kubernetes: What It Actually Takes
Inferoa
AI harness claimed 90%
cache
savings. We ran it and measured 97.8%
🗄️
KV Cache
zozo123.github.io
·
1d
1 day ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help