Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
LLM serving frameworks
🚀 LLM serving frameworks
Specific
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
236
posts in
9.3
ms
harshuljain13/llm-inference-at-scale
: A Practitioner handbook for production
llm
serving
.
🔧
Systems-level optimizations for LLM serving
Content type:
Code
github.com
·
5d
5 days ago
·
Hacker News
,
r/LLM
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
Ollama
0.30 GPU Boost: Faster local Qwen
inference
on NVIDIA
🔢
Quantization of LLMs
everylocalai.com
·
1d
1 day ago
·
DEV
Actions for Ollama 0.30 GPU Boost: Faster local Qwen inference on NVIDIA
6. Air-Gapped Claude Code - The Claude Code SRE Handbook
🤖
Agents using LLMs
har-ki.github.io
·
8h
8 hours ago
·
Hacker News
Actions for 6. Air-Gapped Claude Code - The Claude Code SRE Handbook
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
🔧
Systems-level optimizations for LLM serving
Content type:
News
newsletter.semianalysis.com
·
2d
2 days ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
Google's DiffusionGemma
generates
256 tokens in parallel and self-corrects as it goes
🔧
Systems-level optimizations for LLM serving
venturebeat.com
·
9h
9 hours ago
Actions for Google's DiffusionGemma generates 256 tokens in parallel and self-corrects as it goes
Breaking the Ice: Analyzing Cold Start
Latency
in
vLLM
🌐
Distributed LLM Systems
Content type:
Academic
arxiv.org
·
3d
3 days ago
·
Hacker News
Actions for Breaking the Ice: Analyzing Cold Start Latency in vLLM
Inferoa
AI harness claimed 90%
cache
savings. We ran it and measured 97.8%
🌐
Distributed LLM Systems
zozo123.github.io
·
1d
1 day ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
Mi50 32GB / GFX906 -
vLLM
Qwen 3.5 Configuration for Qwen 3.5:9B AWQ-4bit
🔢
Quantization of LLMs
huggingface.co
·
2h
2 hours ago
·
r/LocalLLaMA
Actions for Mi50 32GB / GFX906 - vLLM Qwen 3.5 Configuration for Qwen 3.5:9B AWQ-4bit
Running
LLM
Inference
on Kubernetes: What It Actually Takes
🧠
Large Language Models (LLMs)
Content type:
Blog
fairwinds.com
·
6d
6 days ago
Actions for Running LLM Inference on Kubernetes: What It Actually Takes
Token4Token — pay-per-token
inference
on Gnosis + Swarm
🧠
Large Language Models (LLMs)
t4t.eth.link
·
2d
2 days ago
·
Hacker News
Actions for Token4Token — pay-per-token inference on Gnosis + Swarm
Intelligent
inference
scheduling with
llm-d
on Red Hat AI
🔧
Systems-level optimizations for LLM serving
developers.redhat.com
·
1d
1 day ago
Actions for Intelligent inference scheduling with llm-d on Red Hat AI
PagedAttention
vs Traditional
KV
Cache
: How vLLM Reinvented GPU Memory for LLM Inference
🔧
Systems-level optimizations for LLM serving
Content type:
Blog
medium.com
·
3d
3 days ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
Less-relevant results
vLLM
Transformers Backend: Bridging Hugging Face Compatibility and High-Performance
Inference
🌐
Distributed LLM Systems
Content type:
Blog
odsc.medium.com
·
2h
2 hours ago
Actions for vLLM Transformers Backend: Bridging Hugging Face Compatibility and High-Performance Inference
A visionary legacy: Andy Brown
🔧
Systems-level optimizations for LLM serving
uk.themedialeader.com
·
1d
1 day ago
Actions for A visionary legacy: Andy Brown
Ollama
0.30 delivers faster NVIDIA GPU performance and wider hardware support
🔢
Quantization of LLMs
alternativeto.net
·
3d
3 days ago
Actions for Ollama 0.30 delivers faster NVIDIA GPU performance and wider hardware support
NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
🧠
Large Language Models (LLMs)
Content type:
Blog
blogs.nvidia.com
·
1d
1 day ago
Actions for NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
Unsloth Gemma 4 QAT
✨
Model optimizations in LLMs
unsloth.ai
·
6d
6 days ago
Actions for Unsloth Gemma 4 QAT
I've tested so many desktop AI tools, but Hermes with
Ollama
is my new favorite - here's why
🤖
Agents using LLMs
Content type:
News
Content type:
Tutorial
zdnet.com
·
1d
1 day ago
Actions for I've tested so many desktop AI tools, but Hermes with Ollama is my new favorite - here's why
massimo92/spark: CLI tool for
serving
LLMs with
vLLM
on NVIDIA DGX Spark. One file, zero friction.
🔧
Systems-level optimizations for LLM serving
Content type:
Code
github.com
·
5h
5 hours ago
·
Hacker News
Actions for massimo92/spark: CLI tool for serving LLMs with vLLM on NVIDIA DGX Spark. One file, zero friction.
DiffusionGemma: The Developer Guide- Google Developers Blog
🔧
Systems-level optimizations for LLM serving
Content type:
Blog
developers.googleblog.com
·
2d
2 days ago
·
r/LocalLLaMA
Actions for DiffusionGemma: The Developer Guide- Google Developers Blog
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help