⚡ LLM Quantization - akapaka

Less-relevant results

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

🧠LLM Inference Code

github.com··Hacker News

MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better

🧠LLM Inference News Blog

kaitchup.substack.com··r/LocalLLaMA

A system programmer’s guide to LLM inference

🧠LLM Inference Blog

blog.xiangpeng.systems··Hacker News

DeskDash - a free Windows tool to easily manage your GGUF files

🧠Local llm

gerry7.itch.io··r/LocalLLaMA

Here's a llama.cpp CLI Command builder.

🧠Local llm

llamabuilding.com··r/LocalLLaMA

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

🧠Local llm News Blog

blog.google··Hacker News

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

🧠Local llm

deemwar-products.github.io··Hacker News

Evaluating bigaspv2-5, a Flow Matching Alternative to SDXL

🧠Local llm

hackernoon.com·

Remove padding and multiple D2D copies for MTP by gaugarg-nv · Pull Request #24086 · ggml-org/llama.cpp

🧠Local llm Code

github.com··r/LocalLLaMA

Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB

🧠Local llm Blog

ziraph.com··Hacker News

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

🧠LLM Inference

local-llm.utop.workers.dev··Hacker News

Apple WWDC On-Device AI Deep Dive - Google Docs

🧠LLM Inference

gist.is··Hacker News

mtmd : add video input support by ngxson · Pull Request #24269 · ggml-org/llama.cpp

🧠Local llm Code

github.com··r/LocalLLaMA

Launch HN: General Instinct (YC P26) – Frontier models on edge devices

🧠Local llm Discussion

news.ycombinator.com··Hacker News

mtp: support for gemma-4 E2B and E4B assistants by max-krasnyansky · Pull Request #24282 · ggml-org/llama.cpp

🧠Local llm Code

github.com··r/LocalLLaMA

zhongkaifu/TensorSharp: A C# inference engine for running large language models (LLMs) locally using GGUF model files. TensorSharp provides a console application, a web-based chatbot interface, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/Linux with full GPU capability

🧠LLM Inference Code

github.com··Hacker News

1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM

🤖Qwen

smolhub.com··r/LocalLLaMA

john-rocky/apple-silicon-llm-bench: Neutral, reproducible benchmark for local LLMs on Apple Silicon (Mac · iPhone · iPad) — MLX, llama.cpp, CoreML, Apple Foundation Models

🤖Qwen Code

github.com··Hacker News

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

Qwen 3.6 27B AutoRound GGUF, need your feedback

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better

A system programmer’s guide to LLM inference

DeskDash - a free Windows tool to easily manage your GGUF files

Here's a llama.cpp CLI Command builder.

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

Evaluating bigaspv2-5, a Flow Matching Alternative to SDXL

Remove padding and multiple D2D copies for MTP by gaugarg-nv · Pull Request #24086 · ggml-org/llama.cpp

Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

Apple WWDC On-Device AI Deep Dive - Google Docs

mtmd : add video input support by ngxson · Pull Request #24269 · ggml-org/llama.cpp

Launch HN: General Instinct (YC P26) – Frontier models on edge devices

mtp: support for gemma-4 E2B and E4B assistants by max-krasnyansky · Pull Request #24282 · ggml-org/llama.cpp

1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM

john-rocky/apple-silicon-llm-bench: Neutral, reproducible benchmark for local LLMs on Apple Silicon (Mac · iPhone · iPad) — MLX, llama.cpp, CoreML, Apple Foundation Models