Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Inference
🤖 Inference
Model Serving, Quantization, vLLM, TensorRT
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
260
posts in
7.0
ms
GGUF
vs
GPTQ
vs
AWQ
: The Plain-English Guide to LLM Quantization (and Which One to Pick)
🧠
LLMs
vettedconsumer.com
·
3d
3 days ago
·
Hacker News
Actions for GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)
lightmetal: GPU
LLM
Inference
From a Single Java 25 JAR
🧠
LLMs
Content type:
Blog
adambien.blog
·
1d
1 day ago
Actions for lightmetal: GPU LLM Inference From a Single Java 25 JAR
Inferoa
AI harness claimed 90% cache savings. We ran it and measured 97.8%
🧠
LLMs
zozo123.github.io
·
2h
2 hours ago
·
Hacker News
Actions for Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%
Alignment Collapse Under KV Cache
Quantization
: Diagnosis and Mitigation
🧠
LLMs
Content type:
Academic
arxiv.org
·
9h
9 hours ago
Actions for Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
2x GH200 for
LLM
inference
, Part 2:
vLLM
, DeepSeek V4 Flash, and MTP
🧠
LLMs
Content type:
Blog
dnhkng.github.io
·
2d
2 days ago
Actions for 2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP
MoQ
GGUFs
and GSQ: Low-Bit
GGUFs
Are About to Get Much Better
🧠
LLMs
Content type:
News
Content type:
Blog
kaitchup.substack.com
·
4d
4 days ago
·
r/LocalLLaMA
Actions for MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better
A visionary legacy: Andy Brown
🌐
Distributed Systems
uk.themedialeader.com
·
8h
8 hours ago
Actions for A visionary legacy: Andy Brown
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
🧠
LLMs
Content type:
News
newsletter.semianalysis.com
·
1d
1 day ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
Nvidia DGX Spark GB10 – AI
Models
and Guide with
vLLM
and Autonomous Script
🧠
LLMs
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script
Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
🎮
GPUs
aarushgupta.io
·
17h
17 hours ago
·
Lobsters
,
Hacker News
Actions for Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
Ollama 0.30 delivers faster NVIDIA GPU performance and wider hardware support
🏗️
MLSys
alternativeto.net
·
2d
2 days ago
Actions for Ollama 0.30 delivers faster NVIDIA GPU performance and wider hardware support
Improved performance and
model
support with
GGUF
🏗️
MLSys
Content type:
Blog
ollama.com
·
5d
5 days ago
Actions for Improved performance and model support with GGUF
google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation
🧠
LLMs
huggingface.co
·
2d
2 days ago
·
r/LocalLLaMA
Actions for google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation
Unsloth Gemma 4 QAT
🎮
GPUs
unsloth.ai
·
4d
4 days ago
Actions for Unsloth Gemma 4 QAT
HNSW vs LSH: How Elasticsearch hits 0.99 recall@10 at 15,000 QPS — and what it costs
🏗️
MLSys
Content type:
Blog
elastic.co
·
1d
1 day ago
Actions for HNSW vs LSH: How Elasticsearch hits 0.99 recall@10 at 15,000 QPS — and what it costs
Gemma 4 QAT
models
: Optimizing model compression for mobile and laptop efficiency
🧠
LLMs
Content type:
News
Content type:
Blog
blog.google
·
4d
4 days ago
·
Hacker News
Actions for Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency
A system programmer’s guide to
LLM
inference
🎮
GPUs
Content type:
Blog
blog.xiangpeng.systems
·
2d
2 days ago
·
Hacker News
Actions for A system programmer’s guide to LLM inference
LLM
Inference
Engineering Room — Part 3: The Orchestration Layer
🧠
LLMs
Content type:
Blog
vimal-dwarampudi.medium.com
·
6d
6 days ago
Actions for LLM Inference Engineering Room — Part 3: The Orchestration Layer
What's in the Box? A Field Guide to AI
Models
🏗️
MLSys
Content type:
Blog
iankduncan.com
·
1d
1 day ago
Actions for What's in the Box? A Field Guide to AI Models
local
llm
on laptop 780M GPU using
llama
+ gemma 4 qat
🧠
LLMs
Content type:
Blog
alper.bearblog.dev
·
4d
4 days ago
Actions for local llm on laptop 780M GPU using llama + gemma 4 qat
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help