Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Inference Engineering
🖥️ Inference Engineering
Specific
LLM serving, latency, batching, throughput
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
190
posts in
7.1
ms
AMD Radeon RX 9070 GRE vs. Radeon RX 9070
💰
Compute Costs
club386.com
·
1d
1 day ago
Actions for AMD Radeon RX 9070 GRE vs. Radeon RX 9070
Apples to Apples: MLX vs.
Llama.cpp
for Gemma 4 12B on an M1 16GB
💰
Compute Costs
Content type:
Blog
ziraph.com
·
5d
5 days ago
·
Hacker News
Actions for Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB
Less-relevant results
Token4Token
—
pay-per-token
inference
on Gnosis + Swarm
🗄️
KV Cache
t4t.eth.link
·
2d
2 days ago
·
Hacker News
Actions for Token4Token — pay-per-token inference on Gnosis + Swarm
Why I care so much about energy
per
token
💰
AI Economics
Content type:
Blog
ziraph.com
·
3d
3 days ago
·
Hacker News
Actions for Why I care so much about energy per token
The Memory Problem is Solved: How Google’s Memory
Caching
Makes RNNs Smart Again
🎯
Fine-tuning
Content type:
Blog
medium.com
·
2d
2 days ago
Actions for The Memory Problem is Solved: How Google’s Memory Caching Makes RNNs Smart Again
1-bit and 1.58 bit
LLM
Benchmarking on Jetson Orin Nano Super | Bonsai LM
🗄️
KV Cache
smolhub.com
·
3d
3 days ago
·
r/LocalLLaMA
Actions for 1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM
BeeLlama.cpp DFlash on Strix Halo: 2.7x Gemma 31B, But MTP Is Still Faster
🗄️
KV Cache
sleepingrobots.com
·
4d
4 days ago
Actions for BeeLlama.cpp DFlash on Strix Halo: 2.7x Gemma 31B, But MTP Is Still Faster
NetX-lab/Frontier: Frontier: A Discrete-Event Simulator for
Modern
LLM
Serving
🗄️
KV Cache
Content type:
Code
github.com
·
9h
9 hours ago
·
Hacker News
Actions for NetX-lab/Frontier: Frontier: A Discrete-Event Simulator for Modern LLM Serving
Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and 4-Bit Shouldn’t Be This Good
🗄️
KV Cache
Content type:
Blog
towardsai.net
·
3d
3 days ago
Actions for Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and 4-Bit Shouldn’t Be This Good
Two Leaps to 1000
Tokens/s
on a 1T-Parameter
Model
: On
Inference
Systems, Execution Boundaries, and Co-Design
🗄️
KV Cache
Content type:
Blog
tilert.ai
·
2d
2 days ago
·
Hacker News
Actions for Two Leaps to 1000 Tokens/s on a 1T-Parameter Model: On Inference Systems, Execution Boundaries, and Co-Design
Benchmarking dots.tts on Strix Halo
♟️
Game Theory
sleepingrobots.com
·
3d
3 days ago
Actions for Benchmarking dots.tts on Strix Halo
STAR-KV
: Low-Rank
KV
Cache
Compression via Soft Thresholding for Adaptive Rank Control
🗄️
KV Cache
Content type:
Academic
arxiv.org
·
2d
2 days ago
Actions for STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control
A system programmer’s guide to
LLM
inference
💰
Compute Costs
Content type:
Blog
blog.xiangpeng.systems
·
3d
3 days ago
·
Hacker News
Actions for A system programmer’s guide to LLM inference
Report: GKE
Inference
Gateway delivers up to 92% faster AI responses
🔍
RAG
Content type:
Blog
cloud.google.com
·
2d
2 days ago
·
Hacker News
Actions for Report: GKE Inference Gateway delivers up to 92% faster AI responses
MLPerf and the rise of
latency-aware
LLM
benchmarking
🤖
AI
edn.com
·
6d
6 days ago
Actions for MLPerf and the rise of latency-aware LLM benchmarking
A UK startup says it can cut data centre network power by 81% by replacing every electrical switch with light
💰
AI Economics
Content type:
News
thenextweb.com
·
3d
3 days ago
Actions for A UK startup says it can cut data centre network power by 81% by replacing every electrical switch with light
AI
Serving
Platform That Adapts to Your
Model
🗄️
KV Cache
Content type:
Blog
databricks.com
·
1d
1 day ago
Actions for AI Serving Platform That Adapts to Your Model
Build a local voice agent with Red Hat OpenShift AI
🗄️
KV Cache
developers.redhat.com
·
3d
3 days ago
Actions for Build a local voice agent with Red Hat OpenShift AI
KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4
GPU
(gfx1201): TurboQuant
KV
cache
+ HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.
🗄️
KV Cache
Content type:
Code
github.com
·
1d
1 day ago
·
Hacker News
Actions for KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.
RKSC: Reasoning-Aware
KV
Cache
Sharing and Confident Early Exit for Multi-Step
LLM
Inference
🗄️
KV Cache
Content type:
Academic
arxiv.org
·
1d
1 day ago
Actions for RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference
Sign up or log in to see more results
Sign Up
Login
« Page 2
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help