Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
ML Inference
⚡ ML Inference
inference engine, model serving, inference optimization, runtime
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
148
posts in
9.1
ms
harshuljain13/llm-inference-at-scale
: A Practitioner handbook for production llm
serving
.
🤖
Machine Learning
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
🤖
Machine Learning
Content type:
News
newsletter.semianalysis.com
·
1d
1 day ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
AI
Serving
Platform That Adapts to Your
Model
🔄
MLOps
Content type:
Blog
databricks.com
·
14h
14 hours ago
Actions for AI Serving Platform That Adapts to Your Model
Inferoa
AI harness claimed 90% cache savings. We ran it and measured 97.8%
🔄
MLOps
zozo123.github.io
·
20h
20 hours ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
Infrastructure Options for Scalable AI
Inference
⚙️
Systems Programming
Content type:
Blog
mirantis.com
·
1d
1 day ago
Actions for Infrastructure Options for Scalable AI Inference
Breaking the Ice: Analyzing Cold Start
Latency
in
vLLM
🔧
MLIR
Content type:
Academic
arxiv.org
·
3d
3 days ago
·
Hacker News
Actions for Breaking the Ice: Analyzing Cold Start Latency in vLLM
The
Inference
Alpha: Maximizing Frontier
Models
on AMD
📐
Model Architecture
Content type:
Blog
digitalocean.com
·
16h
16 hours ago
Actions for The Inference Alpha: Maximizing Frontier Models on AMD
OpenCV Introduces New DNN
Inference
Engine
🤖
Machine Learning
i-programmer.info
·
2d
2 days ago
Actions for OpenCV Introduces New DNN Inference Engine
NVIDIA
Accelerates Google DeepMind’s DiffusionGemma for Local AI
🎮
GPU Programming
Content type:
Blog
blogs.nvidia.com
·
14h
14 hours ago
Actions for NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
Running LLM
Inference
on Kubernetes: What It Actually Takes
🔗
Distributed Training
Content type:
Blog
fairwinds.com
·
5d
5 days ago
Actions for Running LLM Inference on Kubernetes: What It Actually Takes
Mobile AI Compute
Engine
(MACE)
inference
framework — Vision SDK
🛠️
ML Frameworks
Content type:
Blog
mapbox.com
·
2d
2 days ago
Actions for Mobile AI Compute Engine (MACE) inference framework — Vision SDK
OpenCV 5.0 Computer Vision Library Released with Rewritten DNN
Engine
🤖
Machine Learning
linuxiac.com
·
2d
2 days ago
Actions for OpenCV 5.0 Computer Vision Library Released with Rewritten DNN Engine
Pruned YOLOv8
ONNX
INT8 Fails: 3 Fixes That Work
🗜️
Quantization
Content type:
Blog
Content type:
Discussion
tildalice.io
·
5d
5 days ago
Actions for Pruned YOLOv8 ONNX INT8 Fails: 3 Fixes That Work
DiffusionGemma: 4x Faster Text Generation
🧠
Deep Learning
Content type:
News
Content type:
Blog
blog.google
·
14h
14 hours ago
·
Hacker News
,
r/LocalLLaMA
,
r/singularity
Actions for DiffusionGemma: 4x Faster Text Generation
Token4Token — pay-per-token
inference
on Gnosis + Swarm
🔄
MLOps
t4t.eth.link
·
1d
1 day ago
·
Hacker News
Actions for Token4Token — pay-per-token inference on Gnosis + Swarm
From GPU to Token: The 8-Layer Observability Stack for AI Infrastructure
🎮
GPU Programming
Content type:
Blog
jimmysong.io
·
2d
2 days ago
Actions for From GPU to Token: The 8-Layer Observability Stack for AI Infrastructure
Magenta RealTime 2: Open and Local Live Music
Models
🧠
Deep Learning
magenta.withgoogle.com
·
6d
6 days ago
·
Hacker News
,
Hacker News
,
r/LocalLLaMA
Actions for Magenta RealTime 2: Open and Local Live Music Models
RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM
Inference
🧠
Deep Learning
Content type:
Academic
arxiv.org
·
1d
1 day ago
Actions for RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference
🇳🇱 Go/Golang job: Senior Backend
Engineer
(Go) | Studio AI at Creative Fabrica (Amsterdam, Netherlands)
⚙️
Model Training
golangprojects.com
·
15h
15 hours ago
Actions for 🇳🇱 Go/Golang job: Senior Backend Engineer (Go) | Studio AI at Creative Fabrica (Amsterdam, Netherlands)
MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
🖥️
Systems ML
Content type:
News
Content type:
Blog
kaitchup.substack.com
·
5d
5 days ago
·
r/LocalLLaMA
Actions for MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help