⚡ FlashAttention - nayyara.airlangga

libertywing/FlashMemory-Deepseek-V4: FlashMemory DS-V4 Retriever: a lightweight retriever that sparsifies DeepSeek-V4 CSA KV-cache. Weights available on Hugging Face.

🧠Inference Engineering Code

github.com·

Less-relevant results

The Inference Alpha: Maximizing Frontier Models on AMD

🧠Inference Engineering Blog

digitalocean.com·

High Bandwidth Flash | A New Memory for AI Data Centers and Edge Computing | Sandisk

🧠Inference Engineering

ncnonline.net·

How we fight GPU scarcity without compromise

💾KV Cache Blog

equixly.com··Hacker News

Massive AI Storage Demand Creates a New Memory Wall

🧠Inference Engineering News

eetimes.com·

The Memory Problem is Solved: How Google’s Memory Caching Makes RNNs Smart Again

🧠Inference Engineering Blog

medium.com·

Hugging Face Transformers RCE flaw enables stealthy compromise via AI model configs

🔢FP8 Training

csoonline.com·

DiffusionGemma: The Developer Guide

🧠Inference Engineering Blog

developers.googleblog.com·

Making Local LLM Go Brrr

⏱️Prefill Decoding

seanpedersen.github.io·

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

⏱️Prefill Decoding Code

github.com··Hacker News

PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference

⏱️Prefill Decoding Blog

medium.com

Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation

💰Inference Cost Academic

arxiv.org·

The economics of speculative decoding

🚀Speculative Decoding Blog

fergusfinn.com··Hacker News

Stateful Swarms: How Persistent Memory Beats Traditional Agent Architectures

🧠Inference Engineering News

artificialintelligencemadesimple.com·

The Edge LLM Offload Story

🧠Inference Engineering

semiengineering.com·

A system programmer’s guide to LLM inference

💰Inference Cost Blog

blog.xiangpeng.systems··Hacker News

RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference

💾KV Cache Academic

arxiv.org·

Building & Benchmarking: LLMs on a 16GB Jetson Orin NX for Hermes Agent

🎮GPU Computing Blog

dnhkng.github.io·

RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention

Big Blue’s Redbook on Storage Scale KV Cache management

libertywing/FlashMemory-Deepseek-V4: FlashMemory DS-V4 Retriever: a lightweight retriever that sparsifies DeepSeek-V4 CSA KV-cache. Weights available on Hugging Face.

The Inference Alpha: Maximizing Frontier Models on AMD

High Bandwidth Flash | A New Memory for AI Data Centers and Edge Computing | Sandisk

How we fight GPU scarcity without compromise

Massive AI Storage Demand Creates a New Memory Wall

The Memory Problem is Solved: How Google’s Memory Caching Makes RNNs Smart Again

Hugging Face Transformers RCE flaw enables stealthy compromise via AI model configs

DiffusionGemma: The Developer Guide

Making Local LLM Go Brrr

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference

Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation

The economics of speculative decoding

Stateful Swarms: How Persistent Memory Beats Traditional Agent Architectures

The Edge LLM Offload Story

A system programmer’s guide to LLM inference

RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference

Building & Benchmarking: LLMs on a 16GB Jetson Orin NX for Hermes Agent