Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Inference Engineering
🖥️ Inference Engineering
Specific
LLM serving, latency, batching, throughput
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
194
posts in
8.8
ms
DiffusionGemma: The Developer Guide
🗄️
KV Cache
Content type:
Blog
developers.googleblog.com
·
1d
1 day ago
Actions for DiffusionGemma: The Developer Guide
Running
LLM
Inference
on Kubernetes: What It Actually Takes
🗄️
KV Cache
Content type:
Blog
fairwinds.com
·
5d
5 days ago
Actions for Running LLM Inference on Kubernetes: What It Actually Takes
Less-relevant results
Introducing Piper: A Programmable Distributed Training System
💰
Compute Costs
Content type:
Academic
Content type:
Blog
syfi.cs.washington.edu
·
11h
11 hours ago
·
Hacker News
Actions for Introducing Piper: A Programmable Distributed Training System
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
🗄️
KV Cache
Content type:
News
newsletter.semianalysis.com
·
2d
2 days ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
Running Qwen 35B MoE at 450k Context on a Single 32GB
GPU
🗄️
KV Cache
local-llm.utop.workers.dev
·
4d
4 days ago
·
Hacker News
Actions for Running Qwen 35B MoE at 450k Context on a Single 32GB GPU
Google’s DiffusionGemma is 4x faster than its other Gemma
models
🗄️
KV Cache
thenewstack.io
·
21h
21 hours ago
Actions for Google’s DiffusionGemma is 4x faster than its other Gemma models
Anatomy of a high-performance EP kernel
🗄️
KV Cache
Content type:
Blog
fergusfinn.com
·
1d
1 day ago
·
Hacker News
Actions for Anatomy of a high-performance EP kernel
google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation
🗄️
KV Cache
huggingface.co
·
3d
3 days ago
·
r/LocalLLaMA
Actions for google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation
KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4
GPU
(gfx1201): TurboQuant
KV
cache
+ HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.
🗄️
KV Cache
Content type:
Code
github.com
·
22h
22 hours ago
·
Hacker News
Actions for KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.
Building & Benchmarking: LLMs on a 16GB Jetson Orin NX for Hermes Agent
🗄️
KV Cache
Content type:
Blog
dnhkng.github.io
·
2d
2 days ago
Actions for Building & Benchmarking: LLMs on a 16GB Jetson Orin NX for Hermes Agent
Gemma 4 QAT
models
: Optimizing model compression for mobile and laptop efficiency
🗄️
KV Cache
Content type:
News
Content type:
Blog
blog.google
·
5d
5 days ago
·
Hacker News
Actions for Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency
WEKA software speeds long context AI
inferencing
on Oracle’s public cloud
🗄️
KV Cache
Content type:
News
blocksandfiles.com
·
1d
1 day ago
Actions for WEKA software speeds long context AI inferencing on Oracle’s public cloud
Google open-sources speedy DiffusionGemma
text
diffusion
model
🔍
RAG
siliconangle.com
·
13h
13 hours ago
Actions for Google open-sources speedy DiffusionGemma text diffusion model
Luce Spark: a 35B MoE on a 16 GB
GPU
, without the offload tax
💰
Compute Costs
Content type:
Blog
lucebox.com
·
6d
6 days ago
·
Hacker News
Actions for Luce Spark: a 35B MoE on a 16 GB GPU, without the offload tax
The
Inference
Alpha: Maximizing Frontier
Models
on AMD
🗄️
KV Cache
Content type:
Blog
digitalocean.com
·
1d
1 day ago
Actions for The Inference Alpha: Maximizing Frontier Models on AMD
DiffusionGemma is Google’s fastest AI yet, but it comes with a big trade-off
🗄️
KV Cache
androidauthority.com
·
8h
8 hours ago
Actions for DiffusionGemma is Google’s fastest AI yet, but it comes with a big trade-off
STAR-KV
: Low-Rank
KV
Cache
Compression via Soft Thresholding for Adaptive Rank Control
🗄️
KV Cache
Content type:
Academic
arxiv.org
·
2d
2 days ago
Actions for STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control
AMD Radeon RX 9070 GRE vs. Radeon RX 9070
💰
Compute Costs
club386.com
·
1d
1 day ago
Actions for AMD Radeon RX 9070 GRE vs. Radeon RX 9070
MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
🎯
Fine-tuning
Content type:
News
Content type:
Blog
kaitchup.substack.com
·
5d
5 days ago
·
r/LocalLLaMA
Actions for MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
Google's latest DiffusionGemma open AI
model
comes with a 4x speed boost
🗄️
KV Cache
Content type:
News
arstechnica.com
·
19h
19 hours ago
Actions for Google's latest DiffusionGemma open AI model comes with a 4x speed boost
« Page 1
·
Page 3 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help