LLM Inference

Feeds to Scour
SubscribedAll
Scoured 417 posts in 7.3 ms

massimo92/spark: CLI tool for serving LLMs with vLLM on NVIDIA DGX Spark. One file, zero friction.

 🧠LLMs  Content type: Code
github.com··Hacker News

SPEAR: A System for Post-Quantization Error-Adaptive Recovery Enabling Efficient Low-Bit LLM Serving

 🎯Fine-tuning  Content type: Academic
arxiv.org·

DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200

 🧠LLMs  Content type: News

PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference

 🧠LLMs  Content type: Blog
medium.com
·

I've tested so many desktop AI tools, but Hermes with Ollama is my new favorite - here's why

 🤖AI Agents  Content type: News  Content type: Tutorial
zdnet.com·

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

 🧠LLMs  Content type: News  Content type: Blog
blog.google··Hacker News

Anatomy of a high-performance EP kernel

 CUDA  Content type: Blog

2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP

 🧠LLMs  Content type: Blog
dnhkng.github.io·

Gemma 4 QAT on 10GB Laptop: Local AI with 6.7GB VRAM

 🎯Fine-tuning
everylocalai.com··DEV

Stop Treating Your Models Like Microservices

 🧩Cognitive Science
cloudnativenow.com·

Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit

 🪟Context Windows
venturebeat.com·

Ollama 0.30 delivers faster NVIDIA GPU performance and wider hardware support

 CUDA
alternativeto.net·

Qwen 3.6 27B AutoRound GGUF, need your feedback

 🧠LLMs

WEKA software speeds long context AI inferencing on Oracle’s public cloud

 🪟Context Windows  Content type: News
blocksandfiles.com·

vLLM Transformers Backend: Bridging Hugging Face Compatibility and High-Performance Inference

 🤖Data science  Content type: Blog
odsc.medium.com·

NetX-lab/Frontier: Frontier: A Discrete-Event Simulator for Modern LLM Serving

 🧠LLMs  Content type: Code
github.com··Hacker News

The economics of speculative decoding

 🤖LLM  Content type: Blog

Characterizing Software Aging in GPU-Based LLM Serving Systems

 🔬Deep Learning  Content type: Academic
arxiv.org·

The Inference Alpha: Maximizing Frontier Models on AMD

 🧠Transformers  Content type: Blog
digitalocean.com·

Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB

 🎯Fine-tuning  Content type: Blog
ziraph.com··Hacker News

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help