Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Inference
⚡ Inference
Specific
LLM inference, vLLM, speculative decoding, latency, throughput
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
339
posts in
4.8
ms
harshuljain13/llm-inference-at-scale
: A Practitioner handbook for production
llm
serving.
🧠
LLMs
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
PagedAttention
vs Traditional
KV
Cache
: How vLLM Reinvented GPU Memory for LLM Inference
🧠
LLMs
Content type:
Blog
medium.com
·
2d
2 days ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
Ollama
0.30 GPU Boost: Faster local Qwen
inference
on NVIDIA
🗄️
Vector Databases
everylocalai.com
·
6h
6 hours ago
·
DEV
Actions for Ollama 0.30 GPU Boost: Faster local Qwen inference on NVIDIA
RKSC: Reasoning-Aware
KV
Cache
Sharing and Confident Early Exit for Multi-Step
LLM
Inference
📐
Embeddings
Content type:
Academic
arxiv.org
·
22h
22 hours ago
Actions for RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference
Xiaomi MiMo-V2.5-Pro Just Hit 1,000
Tokens
Per
Second
!
🤖
AI Agents
gizchina.com
·
1d
1 day ago
Actions for Xiaomi MiMo-V2.5-Pro Just Hit 1,000 Tokens Per Second!
What
Ollama
Reveals About Local AI, Agents, and Open
Models
🌐
World Models
Content type:
Blog
odsc.medium.com
·
4h
4 hours ago
Actions for What Ollama Reveals About Local AI, Agents, and Open Models
Improved performance and
model
support with GGUF
🧠
LLMs
Content type:
Blog
ollama.com
·
6d
6 days ago
Actions for Improved performance and model support with GGUF
Inferoa
AI harness claimed 90%
cache
savings. We ran it and measured 97.8%
🔌
MCP
zozo123.github.io
·
16h
16 hours ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
Big Blue’s Redbook on Storage Scale
KV
Cache
management
🤖
AI Agents
Content type:
News
blocksandfiles.com
·
1d
1 day ago
Actions for Big Blue’s Redbook on Storage Scale KV Cache management
China's Xiaomi MiMo Is Now 15X Faster Than ChatGPT and Claude (4 minute read)
🎛️
Fine-tuning
Content type:
News
decrypt.co
·
2d
2 days ago
Actions for China's Xiaomi MiMo Is Now 15X Faster Than ChatGPT and Claude (4 minute read)
I've tested so many desktop AI tools, but Hermes with
Ollama
is my new favorite - here's why
🤖
AI Agents
Content type:
News
Content type:
Tutorial
zdnet.com
·
12h
12 hours ago
Actions for I've tested so many desktop AI tools, but Hermes with Ollama is my new favorite - here's why
Running
LLM
Inference
on Kubernetes: What It Actually Takes
🧠
LLMs
Content type:
Blog
fairwinds.com
·
5d
5 days ago
Actions for Running LLM Inference on Kubernetes: What It Actually Takes
Ollama
0.30 delivers faster NVIDIA GPU performance and wider hardware support
🧠
LLMs
alternativeto.net
·
2d
2 days ago
Actions for Ollama 0.30 delivers faster NVIDIA GPU performance and wider hardware support
Google's new open
model
DiffusionGemma generates text from noise instead of word by word
🧠
LLMs
the-decoder.com
·
7h
7 hours ago
Actions for Google's new open model DiffusionGemma generates text from noise instead of word by word
Using
Scikit-LLM
with Open-Source LLMs
🧠
LLMs
machinelearningmastery.com
·
6d
6 days ago
Actions for Using Scikit-LLM with Open-Source LLMs
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
🎛️
Fine-tuning
Content type:
News
newsletter.semianalysis.com
·
1d
1 day ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
👁️
Multimodal AI
Content type:
Blog
blogs.nvidia.com
·
10h
10 hours ago
Actions for NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet
🎛️
Fine-tuning
huggingface.co
·
2d
2 days ago
·
Hacker News
Actions for NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet
How we fight GPU scarcity without compromise
🧠
Reasoning Models
Content type:
Blog
equixly.com
·
5d
5 days ago
·
Hacker News
Actions for How we fight GPU scarcity without compromise
Less-relevant results
The
Inference
Alpha: Maximizing Frontier
Models
on AMD
🧠
LLMs
Content type:
Blog
digitalocean.com
·
12h
12 hours ago
Actions for The Inference Alpha: Maximizing Frontier Models on AMD
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help