🧠 LLM Inference - emschwartz

🤖AI Blog

ollama.com·

Less-relevant results

Token4Token — pay-per-token inference on Gnosis + Swarm

🤖AI

t4t.eth.link··Hacker News

Fixing a stuck Ollama runner and building a GPU watchdog

🤖AI

patrickmccanna.net··Hacker News

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

🤖AI

local-llm.utop.workers.dev··Hacker News

A system programmer’s guide to LLM inference

🤖AI Blog

blog.xiangpeng.systems··Hacker News

NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet

🤖AI

huggingface.co··Hacker News

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

🤖AI News Blog

blog.google··Hacker News

How we fight GPU scarcity without compromise

🏗️LLM Infrastructure Blog

equixly.com··Hacker News

google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation

⚡Fast AI Inference

huggingface.co··r/LocalLLaMA

huawei-csl/KVarN: KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.

🏗️LLM Infrastructure Code

github.com··Hacker News

Clairvoyant: Predictive SJF Scheduling to Mitigate Head-of-Line Blocking in Serial LLM Backends

🤖AI Academic

arxiv.org·

MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better

🆕New AI News Blog

kaitchup.substack.com··r/LocalLLaMA

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

🤖AI

vettedconsumer.com··Hacker News

Youssof Altoukhi (@Youssofal_)

🤖AI

xcancel.com··r/LocalLLaMA

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

🤖AI

deemwar-products.github.io··Hacker News

WWDC 2026: Foundation Models (& Anarlog)

🆕New AI

skushagra.com·

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

LLM Inference Handbook 2026

DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200

STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control

Improved performance and model support with GGUF

Token4Token — pay-per-token inference on Gnosis + Swarm

Fixing a stuck Ollama runner and building a GPU watchdog

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

A system programmer’s guide to LLM inference

NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

How we fight GPU scarcity without compromise

google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation

huawei-csl/KVarN: KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.

Clairvoyant: Predictive SJF Scheduling to Mitigate Head-of-Line Blocking in Serial LLM Backends

MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

Youssof Altoukhi (@Youssofal_)

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

WWDC 2026: Foundation Models (& Anarlog)