💾 KV Cache - moyutianzun

🔄Transformers News Blog

blog.google··Hacker News, r/LocalLLaMA, r/singularity

From GPU to Token: The 8-Layer Observability Stack for AI Infrastructure

⚡CUDA Blog

jimmysong.io·

Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and 4-Bit Shouldn’t Be This Good

⚡Inference Optimization Blog

towardsai.net·

libertywing/FlashMemory-Deepseek-V4: FlashMemory DS-V4 Retriever: a lightweight retriever that sparsifies DeepSeek-V4 CSA KV-cache. Weights available on Hugging Face.

⚡Inference Optimization Code

github.com·

google/gemma-4-12B-it-qat-q4_0-gguf

🎭Mixture of Experts

huggingface.co·

Breaking the Ice: Analyzing Cold Start Latency in vLLM

⚡Inference Optimization Academic

arxiv.org··Hacker News

OpenCV 5.0 Computer Vision Library Released with Rewritten DNN Engine

⚡CUDA

linuxiac.com·

DeepSeek V4, LeCun's Bet Against LLMs, and Lovable's Self-Improving Agent - The Tokenizer Edition #30

⚡Inference Optimization

newsletter.artofsaience.com·

fix(gateway): fail closed for unknown model auth · openclaw/openclaw@85343ea

⚡Inference Optimization Code

github.com·

Beyond Per-Token Pricing: A Concurrency-Aware Methodology for LLM Infrastructure Cost Estimation

⚡Inference Optimization Academic

arxiv.org·

#065 - Claude writes 80% of Anthropic's own code, Cloudflare buys Vite, ChatGPT ships Dreaming memory

⚡Inference Optimization

indiehacker.news·

Florian Brand, Prime Intellect research engineer, adopts Gemma 4 E4B 6-bit quantized as his primary local Mac LLM

⚡Inference Optimization News

digg.com··Hacker News

End-to-End Context Compression at Scale

⚡Inference Optimization Academic

arxiv.org·

High Bandwidth Flash | A New Memory for AI Data Centers and Edge Computing | Sandisk

⚡Inference Optimization

ncnonline.net·

Build a local voice agent with Red Hat OpenShift AI

🤖LLM Agents

developers.redhat.com·

[eCHO News] Episode #104: mTLS for Cilium. Lisp for eBPF

⚡Inference Optimization

isovalent-9197153.hs-sites.com·

Integrate OpenShift AI and PG Airman MCP Server

heterodoxin/graphkv: Graph-guided KV cache compression for memory-efficient LLM inference.

Claude Fable 5 🚀, Gemini 3.5 Live Translate 📱, scaling test time compute 📈

Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models

DiffusionGemma: 4x Faster Text Generation

From GPU to Token: The 8-Layer Observability Stack for AI Infrastructure

Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and 4-Bit Shouldn’t Be This Good

libertywing/FlashMemory-Deepseek-V4: FlashMemory DS-V4 Retriever: a lightweight retriever that sparsifies DeepSeek-V4 CSA KV-cache. Weights available on Hugging Face.

google/gemma-4-12B-it-qat-q4_0-gguf

Breaking the Ice: Analyzing Cold Start Latency in vLLM

OpenCV 5.0 Computer Vision Library Released with Rewritten DNN Engine

DeepSeek V4, LeCun's Bet Against LLMs, and Lovable's Self-Improving Agent - The Tokenizer Edition #30

fix(gateway): fail closed for unknown model auth · openclaw/openclaw@85343ea

Beyond Per-Token Pricing: A Concurrency-Aware Methodology for LLM Infrastructure Cost Estimation

#065 - Claude writes 80% of Anthropic's own code, Cloudflare buys Vite, ChatGPT ships Dreaming memory

Florian Brand, Prime Intellect research engineer, adopts Gemma 4 E4B 6-bit quantized as his primary local Mac LLM

End-to-End Context Compression at Scale

High Bandwidth Flash | A New Memory for AI Data Centers and Edge Computing | Sandisk

Build a local voice agent with Red Hat OpenShift AI

[eCHO News] Episode #104: mTLS for Cilium. Lisp for eBPF