🗄️ KV Cache - ghosh.debasish · Scour

DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inference 🧠LLMs

Gemma 4 and Qwen 3.6 with q8_0 and q4_0 KV cache: KL divergence results 💾Storage Engines

localbench.substack.com·6d·r/LocalLLaMA

AmSach/kvquant: Drop-in KV cache compressor for local LLM inference - Run 70B models on 8GB RAM 🧠LLMs

github.com·5h·DEV

Qwen 3.6-35B-A3B KV cache bench: f16 vs q8_0 vs turbo3 vs turbo4 from 0 to 1M context on M5 Max 💾Storage Engines

llmkube.com·2d·r/LocalLLaMA

Intel prioritizes Xeon; CPU shortage opens door for AMD and MediaTek 🖥️Systems Programming

digitimes.com·3h

DeepSeek V4 Cuts KV Cache by 90% at 1M Tokens, But Aggressive Compression Could Risk ‘Needle in a Haystack’ Failures 🧠Reasoning Models

wccftech.com·5d

Speculative Decoding vs MoE: 3.2x Cost Gap on Llama 3 🧠LLMs

tildalice.io·2d

Legare Kerrison and Cedric Clyburn on LLM Performance and Evaluations 📊LLM Evaluation

Skymizer Taiwan Inc. Unveils Breakthrough Architecture Enabling Ultra-Large LLM Inference on a Single Card 🧠Reasoning Models

en.prnasia.com·3d·r/LocalLLaMA

not much happened today 🌊Streaming Algorithms

news.smol.ai·2d

Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective 🧮Cache-Oblivious Algorithms

What 2x GH200 delivers: memory paths for LLM inference 💾Storage Engines

dnhkng.github.io·5d

Human Memory Management And Obsidian V2 🦎Zig Allocators

grahamhelton.com·2d

Lumai Launches the World’s First Optical Computing System for Real-Time, Billion-Parameter LLM Inference 🧠Reasoning Models

globenewswire.com·2d

New comment by zigzag312 in "Rust Memory Management: Ownership vs. Reference Counting" 🖥️Systems Programming

news.ycombinator.com·3d·Hacker News

When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding? 🧮Cache-Oblivious Algorithms

I got a $134 Cloudflare D1 bill. Here's how I cut it 95% 💾Storage Engines

fullstacksveltekit.com·3d·Hacker News

libtalloc_debugging (3) Linux Manual Page 🖥️Systems Programming

systutorials.com·5d

DAK: Direct-Access-Enabled GPU Memory Offloading with Optimal Efficiency for LLM Inference 🧠Reasoning Models

SecureDrop 2.15.1 released 🦀Rust

securedrop.org·6d

Log in to enable infinite scrolling