LLM Inference

Feeds to Scour
SubscribedAll
Scoured 297 posts in 7.0 ms

Two Leaps to 1000 Tokens/s on a 1T-Parameter Model: On Inference Systems, Execution Boundaries, and Co-Design

 🧠KV Cache  Content type: Blog
tilert.ai··Hacker News

WEKA software speeds long context AI inferencing on Oracle’s public cloud

 🧠KV Cache  Content type: News
blocksandfiles.com·

MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better

 🧠KV Cache  Content type: News  Content type: Blog
kaitchup.substack.com··r/LocalLLaMA

Qwen 3.6 27B AutoRound GGUF, need your feedback

 📦Parquet
huggingface.co··r/LocalLLaMA

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

 🧠KV Cache  Content type: News  Content type: Blog
blog.google··Hacker News

Report: GKE Inference Gateway delivers up to 92% faster AI responses

 🧠KV Cache  Content type: Blog
cloud.google.com··Hacker News
Less-relevant results

Massive AI Storage Demand Creates a New Memory Wall

 🧠KV Cache  Content type: News
eetimes.com·

Magenta RealTime 2: Open and Local Live Music Models

 ⚡vLLM
magenta.withgoogle.com··Hacker News, Hacker News, r/LocalLLaMA

DiffusionGemma: The Developer Guide

 🧠KV Cache  Content type: Blog
developers.googleblog.com·

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

 🧠KV Cache
vettedconsumer.com··Hacker News

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

 🧠KV Cache  Content type: Code
github.com··Hacker News

RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference

 🧠KV Cache  Content type: Academic
arxiv.org·

google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation

 🧠KV Cache
huggingface.co··r/LocalLLaMA

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

 🧠KV Cache
local-llm.utop.workers.dev··Hacker News

DiffusionGemma: 4x Faster Text Generation

 🧠KV Cache  Content type: News  Content type: Blog
blog.google··Hacker News, r/LocalLLaMA, r/singularity

Here's a llama.cpp CLI Command builder.

 🦀Rust
llamabuilding.com··r/LocalLLaMA

The economics of speculative decoding

 🧠KV Cache  Content type: Blog
fergusfinn.com··Hacker News

AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support

 🧠KV Cache
phoronix.com·

OpenCV Introduces New DNN Inference Engine

 🧠KV Cache
i-programmer.info·

Unsloth Gemma 4 QAT

 📦Parquet
unsloth.ai·

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help