⚡ LLM Optimization - jimman

Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit

🤖AI

venturebeat.com·

TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUs

⚡Model Efficiency Academic

arxiv.org·

How we fight GPU scarcity without compromise

✍️Prompt Engineering Blog

equixly.com··Hacker News

DiffusionGemma: The Developer Guide

🤖AI Blog

developers.googleblog.com··Hacker News

WEKA software speeds long context AI inferencing on Oracle’s public cloud

⚡Model Efficiency News

blocksandfiles.com·

DeskDash - a free Windows tool to easily manage your GGUF files

🛠️Developer Tools

gerry7.itch.io··r/LocalLLaMA

Anatomy of a high-performance EP kernel

⚡Model Efficiency Blog

fergusfinn.com··Hacker News

2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP

⚡Model Efficiency Blog

dnhkng.github.io·

Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB

⚡Model Efficiency Blog

ziraph.com··Hacker News

146th airhacks tv: Rust, Java 25, AI Agents, BCE, Web Components, zunit, zb

🔓Hacking Blog

adambien.blog·

Building & Benchmarking: LLMs on a 16GB Jetson Orin NX for Hermes Agent

⚡Model Efficiency Blog

dnhkng.github.io·

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

🤖AI

deemwar-products.github.io··Hacker News

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

⚡Model Efficiency Code

github.com··Hacker News

Stop Treating Your Models Like Microservices

⚡Model Efficiency

cloudnativenow.com·

The latest Gemma 4 models use a training trick to slash their on-device memory footprint

🤖AI

androidauthority.com·

The Inference Alpha: Maximizing Frontier Models on AMD

local llm on laptop 780M GPU using llama + gemma 4 qat

Anthropic apologizes for invisible Claude Fable guardrails

PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference

DiffusionGemma 26B A4B results on my 5090

Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit

TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUs

How we fight GPU scarcity without compromise

DiffusionGemma: The Developer Guide

WEKA software speeds long context AI inferencing on Oracle’s public cloud

DeskDash - a free Windows tool to easily manage your GGUF files

Anatomy of a high-performance EP kernel

2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP

Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB

146th airhacks tv: Rust, Java 25, AI Agents, BCE, Web Components, zunit, zb

Building & Benchmarking: LLMs on a 16GB Jetson Orin NX for Hermes Agent

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

Stop Treating Your Models Like Microservices

The latest Gemma 4 models use a training trick to slash their on-device memory footprint