Model Efficiency

⚡ Model Efficiency

Inference Optimization, VRAM Calculation, Performance Tuning, Resource Management

Feeds to Scour
SubscribedAll
Scoured 272 posts in 6.5 ms

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

 🤖AI  Content type: Code
github.com··Hacker News, r/LLM

Big Blue’s Redbook on Storage Scale KV Cache management

 ⚡LLM Optimization  Content type: News
blocksandfiles.com·

Making FlashAttention-4 faster for inference

 ⚡LLM Optimization  Content type: Blog
modal.com·

Ask HN: What's the best LLM model that on a 24 GB VRAM GPU?

 ⚡LLM Optimization  Content type: Discussion
news.ycombinator.com··Hacker News

STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control

 ⚡LLM Optimization  Content type: Academic
arxiv.org·

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

 ⚡LLM Optimization
vettedconsumer.com··Hacker News

Google's DiffusionGemma generates 256 tokens in parallel and self-corrects as it goes

 🤖AI
venturebeat.com·

MTP Isn't Always a Win: 1.95x on My 3090, but Speculative Decoding Is Hardware-Dependent

 ⚡LLM Optimization  Content type: Blog
bric.pe.kr··DEV

Intelligent inference scheduling with llm-d on Red Hat AI

 ✍️Prompt Engineering
developers.redhat.com·

The economics of speculative decoding

 ⚡LLM Optimization  Content type: Blog
fergusfinn.com··Hacker News

Gemma 4 QAT on 10GB Laptop: Local AI with 6.7GB VRAM

 ⚡LLM Optimization
everylocalai.com··DEV

Microsoft is killing the Copilot+ PC advantage, brings Windows 11’s local AI to RTX 30+ PCs with 6GB vRAM

 🔍AI Interpretability
windowslatest.com·

Apple WWDC On-Device AI Deep Dive - Google Docs

 🤖AI
gist.is··Hacker News

The Inference Alpha: Maximizing Frontier Models on AMD

 ⚡LLM Optimization  Content type: Blog
digitalocean.com·

Train Models Faster with JAX and MaxText Using NVFP4 on NVIDIA Blackwell

 ⚡LLM Optimization  Content type: News  Content type: Blog
developer.nvidia.com·

DiffusionGemma

 🔍AI Interpretability
deepmind.google··Hacker News

Report: GKE Inference Gateway delivers up to 92% faster AI responses

 ✍️Prompt Engineering  Content type: Blog
cloud.google.com··Hacker News

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

 ⚡LLM Optimization
local-llm.utop.workers.dev··Hacker News

Google's new open-weights model brings image-generation tricks to AI text generation

 🤖AI  Content type: News
theregister.com·

DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200

 🤖AI  Content type: News
newsletter.semianalysis.com
··Hacker News

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help