💻 Local LLMs - nmarshall

🔥Burn Blog

blog.xiangpeng.systems··Hacker News

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

🧠AI News Blog

blog.google··Hacker News

Token4Token — pay-per-token inference on Gnosis + Swarm

🧠AI

t4t.eth.link··Hacker News

zhongkaifu/TensorSharp: A C# inference engine for running large language models (LLMs) locally using GGUF model files. TensorSharp provides a console application, a web-based chatbot interface, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/Linux with full GPU capability

🧠AI Code

github.com··Hacker News

NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet

🧠AI

huggingface.co··Hacker News

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

🤖AI agents

deemwar-products.github.io··Hacker News

Fixing a stuck Ollama runner and building a GPU watchdog

🧠AI

patrickmccanna.net··Hacker News

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

🤖AI Inference Code

github.com··Hacker News

local llm on laptop 780M GPU using llama + gemma 4 qat

🏠Self-hosted AI Blog

alper.bearblog.dev·

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

🌟Ray Tracing

local-llm.utop.workers.dev··Hacker News

Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB

🤖AI agents Blog

ziraph.com··Hacker News

Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script

🏗️AI Infrastructure Code

github.com··Hacker News

Show HN: Ext-Infer

🤖LLMs

infer.displace.tech··Hacker News

KJLdefeated/RL.cu: RLVR training for LLM in CUDA/C++

🔥PyTorch Code

github.com··Hacker News

mirkolenz/llmhop: Tiny, stateless Go router that dispatches OpenAI-compatible requests to single-model vLLM and sglang backends with zero external dependencies

🧠AI Code

github.com··Hacker News

john-rocky/apple-silicon-llm-bench: Neutral, reproducible benchmark for local LLMs on Apple Silicon (Mac · iPhone · iPad) — MLX, llama.cpp, CoreML, Apple Foundation Models

🤖AI agents Code

github.com··Hacker News

zaydmulani09/mnemo: Local-first AI memory layer for any LLM. Persistent knowledge graph, entity extraction, semantic retrieval. Works with Ollama, OpenAI, Anthropic, or any OpenAI-compatible backend.

🧠AI Code

github.com··Hacker News

No more posts from nmarshall's subscribed feeds.

Scour all 25255 feeds Learn more about Feeds

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200

Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%

A system programmer’s guide to LLM inference

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

Token4Token — pay-per-token inference on Gnosis + Swarm

NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

Fixing a stuck Ollama runner and building a GPU watchdog

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

local llm on laptop 780M GPU using llama + gemma 4 qat

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB

Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script

Show HN: Ext-Infer

KJLdefeated/RL.cu: RLVR training for LLM in CUDA/C++

mirkolenz/llmhop: Tiny, stateless Go router that dispatches OpenAI-compatible requests to single-model vLLM and sglang backends with zero external dependencies

john-rocky/apple-silicon-llm-bench: Neutral, reproducible benchmark for local LLMs on Apple Silicon (Mac · iPhone · iPad) — MLX, llama.cpp, CoreML, Apple Foundation Models

zaydmulani09/mnemo: Local-first AI memory layer for any LLM. Persistent knowledge graph, entity extraction, semantic retrieval. Works with Ollama, OpenAI, Anthropic, or any OpenAI-compatible backend.