Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Inference Engineering
🧠 Inference Engineering
model serving, inference optimization, LLM inference, throughput
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
344
posts in
18.2
ms
RKSC: Reasoning-Aware
KV
Cache
Sharing and Confident Early Exit for Multi-Step
LLM
Inference
💾
KV Cache
Content type:
Academic
arxiv.org
·
14h
14 hours ago
Actions for RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference
libertywing/FlashMemory-Deepseek-V4: FlashMemory DS-V4 Retriever: a lightweight retriever that sparsifies DeepSeek-V4 CSA
KV-cache
. Weights available on Hugging Face.
⏱️
Prefill Decoding
Content type:
Code
github.com
·
18h
18 hours ago
Actions for libertywing/FlashMemory-Deepseek-V4: FlashMemory DS-V4 Retriever: a lightweight retriever that sparsifies DeepSeek-V4 CSA KV-cache. Weights available on Hugging Face.
Fixing a stuck
Ollama
runner and building a GPU watchdog
🧵
Warp Scheduling
patrickmccanna.net
·
1d
1 day ago
·
Hacker News
Actions for Fixing a stuck Ollama runner and building a GPU watchdog
How we fight GPU scarcity without compromise
💾
KV Cache
Content type:
Blog
equixly.com
·
5d
5 days ago
·
Hacker News
Actions for How we fight GPU scarcity without compromise
2x GH200 for
LLM
inference
, Part 2:
vLLM
, DeepSeek V4 Flash, and MTP
💾
KV Cache
Content type:
Blog
dnhkng.github.io
·
2d
2 days ago
Actions for 2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP
DiffusionGemma: 4x Faster Text Generation
🎮
GPU Computing
Content type:
News
Content type:
Blog
blog.google
·
2h
2 hours ago
·
Hacker News
,
r/LocalLLaMA
,
r/singularity
Actions for DiffusionGemma: 4x Faster Text Generation
Using
Scikit-LLM
with Open-Source LLMs
⚙️
ML Compilers
machinelearningmastery.com
·
6d
6 days ago
Actions for Using Scikit-LLM with Open-Source LLMs
Ollama
0.30 delivers faster NVIDIA GPU performance and wider hardware support
🗜️
Quantization
alternativeto.net
·
2d
2 days ago
Actions for Ollama 0.30 delivers faster NVIDIA GPU performance and wider hardware support
AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support
🎮
GPU Computing
phoronix.com
·
1h
1 hour ago
Actions for AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support
A system programmer’s guide to
LLM
inference
💰
Inference Cost
Content type:
Blog
blog.xiangpeng.systems
·
2d
2 days ago
·
Hacker News
Actions for A system programmer’s guide to LLM inference
Speculators v0.5.0: DFlash support and online training
🚀
Speculative Decoding
developers.redhat.com
·
6d
6 days ago
Actions for Speculators v0.5.0: DFlash support and online training
High Bandwidth Flash | A New Memory for AI Data Centers and Edge Computing | Sandisk
🧠
HBM Bandwidth
ncnonline.net
·
1d
1 day ago
Actions for High Bandwidth Flash | A New Memory for AI Data Centers and Edge Computing | Sandisk
Tales of an
Ollama
Honeypot (Part 3): More Traffic, More Findings
🔭
Observability
posts.inthecyber.com
·
2d
2 days ago
Actions for Tales of an Ollama Honeypot (Part 3): More Traffic, More Findings
NVIDIA Nemotron 3 Ultra
⚗️
Kernel Fusion
Content type:
Blog
ollama.com
·
6d
6 days ago
Actions for NVIDIA Nemotron 3 Ultra
Neo-X7/Neo-AI: A fully offline AI assistant powered by
Ollama
. Stores and retrieves conversations using SQLite + LanceDB vector search. No cloud. No API keys. Runs entirely on your machine.
⚙️
MLOps
Content type:
Code
github.com
·
5h
5 hours ago
·
DEV
Actions for Neo-X7/Neo-AI: A fully offline AI assistant powered by Ollama. Stores and retrieves conversations using SQLite + LanceDB vector search. No cloud. No API keys. Runs entirely on your machine.
From GPU to Token: The 8-Layer Observability Stack for AI Infrastructure
💰
Inference Cost
Content type:
Blog
jimmysong.io
·
1d
1 day ago
Actions for From GPU to Token: The 8-Layer Observability Stack for AI Infrastructure
Alignment Collapse Under
KV
Cache
Quantization
: Diagnosis and Mitigation
💰
Inference Cost
Content type:
Academic
arxiv.org
·
14h
14 hours ago
Actions for Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
Running Qwen 35B MoE at 450k Context on a Single 32GB GPU
💰
Inference Cost
local-llm.utop.workers.dev
·
3d
3 days ago
·
Hacker News
Actions for Running Qwen 35B MoE at 450k Context on a Single 32GB GPU
Two Leaps to 1000 Tokens/s on a 1T-Parameter
Model
: On
Inference
Systems, Execution Boundaries, and
Co-Design
💰
Inference Cost
Content type:
Blog
tilert.ai
·
2d
2 days ago
·
Hacker News
Actions for Two Leaps to 1000 Tokens/s on a 1T-Parameter Model: On Inference Systems, Execution Boundaries, and Co-Design
MLPerf and the rise of latency-aware
LLM
benchmarking
⏱️
Prefill Decoding
edn.com
·
5d
5 days ago
Actions for MLPerf and the rise of latency-aware LLM benchmarking
« Page 1
·
Page 3 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help