Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Inference Engineering
🧠 Inference Engineering
model serving, inference optimization, LLM inference, throughput
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
342
posts in
11.8
ms
harshuljain13/llm-inference-at-scale
: A Practitioner handbook for production
llm
serving
.
💾
KV Cache
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
Inferoa
AI harness claimed 90%
cache
savings. We ran it and measured 97.8%
💾
KV Cache
zozo123.github.io
·
9h
9 hours ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
Clairvoyant: Predictive SJF Scheduling to Mitigate Head-of-Line Blocking in Serial
LLM
Backends
💾
KV Cache
Content type:
Academic
arxiv.org
·
2d
2 days ago
Actions for Clairvoyant: Predictive SJF Scheduling to Mitigate Head-of-Line Blocking in Serial LLM Backends
The
Inference
Alpha: Maximizing Frontier
Models
on AMD
💰
Inference Cost
Content type:
Blog
digitalocean.com
·
5h
5 hours ago
Actions for The Inference Alpha: Maximizing Frontier Models on AMD
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
💾
KV Cache
Content type:
News
newsletter.semianalysis.com
·
1d
1 day ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
Infrastructure Options for Scalable AI
Inference
☁️
Cloud Infrastructure
Content type:
Blog
mirantis.com
·
20h
20 hours ago
Actions for Infrastructure Options for Scalable AI Inference
Running
LLM
Inference
on Kubernetes: What It Actually Takes
💰
Inference Cost
Content type:
Blog
fairwinds.com
·
5d
5 days ago
Actions for Running LLM Inference on Kubernetes: What It Actually Takes
Big Blue’s Redbook on Storage Scale
KV
Cache
management
⏱️
Prefill Decoding
Content type:
News
blocksandfiles.com
·
1d
1 day ago
Actions for Big Blue’s Redbook on Storage Scale KV Cache management
DiffusionGemma: The Developer Guide- Google Developers Blog
💾
KV Cache
Content type:
Blog
developers.googleblog.com
·
19h
19 hours ago
·
r/LocalLLaMA
Actions for DiffusionGemma: The Developer Guide- Google Developers Blog
I've tested so many desktop AI tools, but Hermes with
Ollama
is my new favorite - here's why
🚀
Model Serving
Content type:
News
Content type:
Tutorial
zdnet.com
·
5h
5 hours ago
Actions for I've tested so many desktop AI tools, but Hermes with Ollama is my new favorite - here's why
Token4Token — pay-per-token
inference
on Gnosis + Swarm
💾
KV Cache
t4t.eth.link
·
1d
1 day ago
·
Hacker News
Actions for Token4Token — pay-per-token inference on Gnosis + Swarm
The hidden bottleneck in
LLM
inference
and the impact on MLPerf benchmarking
💰
Inference Cost
edn.com
·
6d
6 days ago
Actions for The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking
Massive AI Storage Demand Creates a New Memory Wall
🧠
HBM Bandwidth
Content type:
News
eetimes.com
·
5h
5 hours ago
Actions for Massive AI Storage Demand Creates a New Memory Wall
NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet
🧠
HBM Bandwidth
huggingface.co
·
2d
2 days ago
·
Hacker News
Actions for NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet
Improved performance and
model
support with GGUF
🗜️
Quantization
Content type:
Blog
ollama.com
·
5d
5 days ago
Actions for Improved performance and model support with GGUF
Anatomy of a high-performance EP kernel
💰
Inference Cost
Content type:
Blog
fergusfinn.com
·
19h
19 hours ago
·
Hacker News
Actions for Anatomy of a high-performance EP kernel
PagedAttention vs Traditional
KV
Cache
: How
vLLM
Reinvented GPU Memory for LLM Inference
⏱️
Prefill Decoding
Content type:
Blog
medium.com
·
2d
2 days ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
🎮
GPU Computing
Content type:
Blog
blogs.nvidia.com
·
3h
3 hours ago
Actions for NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
Report: GKE
Inference
Gateway delivers up to 92% faster AI responses
💾
KV Cache
Content type:
Blog
cloud.google.com
·
1d
1 day ago
·
Hacker News
Actions for Report: GKE Inference Gateway delivers up to 92% faster AI responses
LLM
Inference
Engineering
Room — Part 3: The Orchestration Layer
💾
KV Cache
Content type:
Blog
vimal-dwarampudi.medium.com
·
6d
6 days ago
Actions for LLM Inference Engineering Room — Part 3: The Orchestration Layer
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help