🗜️ Quantization - nayyara.airlangga

Less-relevant results

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

⏱️Prefill Decoding Code

github.com··Hacker News

MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better

💰Inference Cost News Blog

kaitchup.substack.com··r/LocalLLaMA

A system programmer’s guide to LLM inference

💰Inference Cost Blog

blog.xiangpeng.systems··Hacker News

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

💰Inference Cost News Blog

blog.google··Hacker News

Here's a llama.cpp CLI Command builder.

🚀Speculative Decoding

llamabuilding.com··r/LocalLLaMA

Pruned YOLOv8 ONNX INT8 Fails: 3 Fixes That Work

💰Inference Cost Blog Discussion

tildalice.io·

local llm on laptop 780M GPU using llama + gemma 4 qat

⚗️Kernel Fusion Blog

alper.bearblog.dev·

DeskDash - a free Windows tool to easily manage your GGUF files

📡OpenTelemetry

gerry7.itch.io··r/LocalLLaMA

Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and 4-Bit Shouldn’t Be This Good

💰Inference Cost Blog

towardsai.net·

146th airhacks tv: Rust, Java 25, AI Agents, BCE, Web Components, zunit, zb

💰Inference Cost Blog

adambien.blog·

fix(memory): move local llama.cpp runtime to provider plugin · openclaw/openclaw@3137110

📡OpenTelemetry Code

github.com·

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

🧠Inference Engineering

deemwar-products.github.io··Hacker News

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

Quality Is Not a Safety Proxy Under Quantization

Qwen 3.6 27B AutoRound GGUF, need your feedback

Ollama 0.30 GPU Boost: Faster local Qwen inference on NVIDIA

lightmetal: GPU LLM Inference From a Single Java 25 JAR

Improved performance and model support with GGUF

Ollama 0.30 delivers faster NVIDIA GPU performance and wider hardware support

Unsloth Gemma 4 QAT

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better

A system programmer’s guide to LLM inference

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

Here's a llama.cpp CLI Command builder.

Pruned YOLOv8 ONNX INT8 Fails: 3 Fixes That Work

local llm on laptop 780M GPU using llama + gemma 4 qat

DeskDash - a free Windows tool to easily manage your GGUF files

Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and 4-Bit Shouldn’t Be This Good

146th airhacks tv: Rust, Java 25, AI Agents, BCE, Web Components, zunit, zb

fix(memory): move local llama.cpp runtime to provider plugin · openclaw/openclaw@3137110

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM