Model Efficiency

⚡ Model Efficiency

Inference Optimization, VRAM Calculation, Performance Tuning, Resource Management

Feeds to Scour
SubscribedAll
Scoured 261 posts in 7.9 ms

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

 🤖AI  Content type: Code
github.com··Hacker News, r/LLM

Big Blue’s Redbook on Storage Scale KV Cache management

 ⚡LLM Optimization  Content type: News
blocksandfiles.com·

Holding the FP8 Quality Ceiling at 8-Bit Weights and Activations: INT8 and GGUF Post-Training Quantization of Ideogram 4.0 for Consumer GPUs

 ⚡LLM Optimization  Content type: Academic
arxiv.org·

Gemma 4 QAT on 10GB Laptop: Local AI with 6.7GB VRAM

 ⚡LLM Optimization
everylocalai.com··DEV

Train Models Faster with JAX and MaxText Using NVFP4 on NVIDIA Blackwell

 ⚡LLM Optimization  Content type: News  Content type: Blog
developer.nvidia.com·

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

 ⚡LLM Optimization
vettedconsumer.com··Hacker News

MTP Isn't Always a Win: 1.95x on My 3090, but Speculative Decoding Is Hardware-Dependent

 ⚡LLM Optimization  Content type: Blog
bric.pe.kr··DEV

Intelligent inference scheduling with llm-d on Red Hat AI

 ✍️Prompt Engineering
developers.redhat.com·

The economics of speculative decoding

 ⚡LLM Optimization  Content type: Blog
fergusfinn.com··Hacker News

Putting a datacenter GPU in a gaming PC for £200 ($268)

 ⚡LLM Optimization  Content type: Blog
blog.adafruit.com·

The Inference Alpha: Maximizing Frontier Models on AMD

 ⚡LLM Optimization  Content type: Blog
digitalocean.com·

Report: GKE Inference Gateway delivers up to 92% faster AI responses

 ✍️Prompt Engineering  Content type: Blog
cloud.google.com··Hacker News

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

 ⚡LLM Optimization
local-llm.utop.workers.dev··Hacker News

Microsoft is killing the Copilot+ PC advantage, brings Windows 11’s local AI to RTX 30+ PCs with 6GB vRAM

 🔍AI Interpretability
windowslatest.com·

Apple WWDC On-Device AI Deep Dive - Google Docs

 🤖AI
gist.is··Hacker News

DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200

 🤖AI  Content type: News
newsletter.semianalysis.com
··Hacker News

PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference

 ⚡LLM Optimization  Content type: Blog
medium.com
·

Two old GPUs I salvaged are doing more AI work than a brand new $2000 card, and I won't be upgrading anytime soon

 🛠️Developer Tools
xda-developers.com·

Building & Benchmarking: LLMs on a 16GB Jetson Orin NX for Hermes Agent

 ⚡LLM Optimization  Content type: Blog
dnhkng.github.io·

MLPerf and the rise of latency-aware LLM benchmarking

 ⚡LLM Optimization
edn.com·

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help