Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
LLM Inference
🤖 LLM Inference
Specific
Model Serving, Quantization, vLLM, ONNX Runtime
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
145
posts in
20.0
ms
harshuljain13/llm-inference-at-scale
: A Practitioner handbook for production
llm
serving
.
⚡
Inference
Content type:
Code
github.com
·
3d
3 days ago
·
Hacker News
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
lightmetal: GPU
LLM
Inference
From a Single Java 25 JAR
⚡
Quantization
Content type:
Blog
adambien.blog
·
1d
1 day ago
Actions for lightmetal: GPU LLM Inference From a Single Java 25 JAR
PagedAttention
vs Traditional KV Cache: How
vLLM
Reinvented GPU Memory for
LLM
Inference
⚡
Inference
Content type:
Blog
medium.com
·
1d
1 day ago
Actions for PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference
MoQ
GGUFs
and GSQ: Low-Bit
GGUFs
Are About to Get Much Better
⚡
Quantization
Content type:
News
Content type:
Blog
kaitchup.substack.com
·
4d
4 days ago
·
r/LocalLLaMA
Actions for MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
2x GH200 for
LLM
inference
, Part 2:
vLLM
, DeepSeek V4 Flash, and MTP
⚡
Quantization
Content type:
Blog
dnhkng.github.io
·
2d
2 days ago
Actions for 2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP
Improved performance and
model
support with
GGUF
🖥️
Local AI
Content type:
Blog
ollama.com
·
5d
5 days ago
Actions for Improved performance and model support with GGUF
GGUF
vs GPTQ vs AWQ: The Plain-English Guide to
LLM
Quantization
(and Which One to Pick)
⚡
Quantization
vettedconsumer.com
·
3d
3 days ago
·
Hacker News
Actions for GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)
Unsloth Gemma 4 QAT
🖥️
Local AI
unsloth.ai
·
4d
4 days ago
Actions for Unsloth Gemma 4 QAT
Part 3 — Implementation/Engine-Level: Choosing the
Runtime
That Gives You These for Free
⚡
Inference
towardsai.net
·
6d
6 days ago
Actions for Part 3 — Implementation/Engine-Level: Choosing the Runtime That Gives You These for Free
Pruned YOLOv8
ONNX
INT8 Fails: 3 Fixes That Work
⚡
Quantization
Content type:
Blog
Content type:
Discussion
tildalice.io
·
4d
4 days ago
Actions for Pruned YOLOv8 ONNX INT8 Fails: 3 Fixes That Work
LLM
Inference
Engineering Room — Part 3: The Orchestration Layer
🧠
LLM Tooling
Content type:
Blog
vimal-dwarampudi.medium.com
·
6d
6 days ago
Actions for LLM Inference Engineering Room — Part 3: The Orchestration Layer
Nvidia DGX Spark GB10 – AI
Models
and Guide with
vLLM
and Autonomous Script
⚡
Inference
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script
local
llm
on laptop 780M GPU using
llama
+ gemma 4 qat
🧠
LLM
Content type:
Blog
alper.bearblog.dev
·
4d
4 days ago
Actions for local llm on laptop 780M GPU using llama + gemma 4 qat
Ideogram4
GGUF
is out!
⚡
Quantization
huggingface.co
·
3d
3 days ago
·
r/StableDiffusion
Actions for Ideogram4 GGUF is out!
Gemma 4 QAT
models
: Optimizing model compression for mobile and laptop efficiency
🔓
Open Source AI
Content type:
News
Content type:
Blog
blog.google
·
4d
4 days ago
·
Hacker News
Actions for Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency
huawei-csl/KVarN: KVarN is a native
vLLM
KV-cache
quantization
backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.
⚡
Inference
Content type:
Code
github.com
·
5d
5 days ago
·
Hacker News
Actions for huawei-csl/KVarN: KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.
fix(gateway): fail closed for unknown
model
auth · openclaw/openclaw@85343ea
🧠
LLM
Content type:
Code
github.com
·
4d
4 days ago
Actions for fix(gateway): fail closed for unknown model auth · openclaw/openclaw@85343ea
heterodoxin/graphkv: Graph-guided KV cache compression for memory-efficient
LLM
inference
.
💾
KV Cache
Content type:
Code
github.com
·
3d
3 days ago
·
r/LocalLLaMA
Actions for heterodoxin/graphkv: Graph-guided KV cache compression for memory-efficient LLM inference.
stable-diffusion.cpp/docs/quantization
_and_
gguf.md
at master ·
leejet/stable-diffusion.cpp
⚡
Quantization
Content type:
Code
github.com
·
3d
3 days ago
·
r/StableDiffusion
Actions for stable-diffusion.cpp/docs/quantization_and_gguf.md at master · leejet/stable-diffusion.cpp
zhongkaifu/TensorSharp
: A C#
inference
engine for running large language
models
(LLMs) locally using GGUF model files.
TensorSharp
provides a console application, a web-based chatbot interface, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/Linux with full GPU capability
🖥️
Local AI
Content type:
Code
github.com
·
6d
6 days ago
·
Hacker News
Actions for zhongkaifu/TensorSharp: A C# inference engine for running large language models (LLMs) locally using GGUF model files. TensorSharp provides a console application, a web-based chatbot interface, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/Linux with full GPU capability
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help