🏠 Self-hosted AI - nmarshall

martidu4/honey-ai: 🍯 All-in-one AI honeypot powered by local LLMs. SSH, HTTP, FTP, Telnet, SMTP, MySQL, Redis, Git, VNC, RDP — with canary tokens, tarpits, GZIP bombs, and threat intel reporting.

🧠AI Code

github.com··Hacker News

NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet

🧠AI

huggingface.co··Hacker News

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

💻Local LLMs

deemwar-products.github.io··Hacker News

I finally built the central AI hub I've been wanting, and Open WebUI made it stupidly simple

🧠AI

xda-developers.com·

local llm on laptop 780M GPU using llama + gemma 4 qat

💻Local LLMs Blog

alper.bearblog.dev·

Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script

🏗️AI Infrastructure Code

github.com··Hacker News

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

💻Local LLMs News Blog

blog.google··Hacker News

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

💻Local LLMs

local-llm.utop.workers.dev··Hacker News

Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB

🤖AI agents Blog

ziraph.com··Hacker News

zhongkaifu/TensorSharp: A C# inference engine for running large language models (LLMs) locally using GGUF model files. TensorSharp provides a console application, a web-based chatbot interface, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/Linux with full GPU capability

💻Local LLMs Code

github.com··Hacker News

Show HN: Ext-Infer

💻Local LLMs

infer.displace.tech··Hacker News

Large companies can add a local LLM filter layer to considerably reducing their AI costs

🧠AI

umrashrf.github.io··Hacker News

mirkolenz/llmhop: Tiny, stateless Go router that dispatches OpenAI-compatible requests to single-model vLLM and sglang backends with zero external dependencies

🧠AI Code

github.com··Hacker News

zaydmulani09/mnemo: Local-first AI memory layer for any LLM. Persistent knowledge graph, entity extraction, semantic retrieval. Works with Ollama, OpenAI, Anthropic, or any OpenAI-compatible backend.

🧠AI Code

github.com··Hacker News

john-rocky/apple-silicon-llm-bench: Neutral, reproducible benchmark for local LLMs on Apple Silicon (Mac · iPhone · iPad) — MLX, llama.cpp, CoreML, Apple Foundation Models

🤖AI agents Code

github.com··Hacker News

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

🤖AI Inference Code

github.com··Hacker News

KJLdefeated/RL.cu: RLVR training for LLM in CUDA/C++

🔥PyTorch Code

github.com··Hacker News

No more posts from nmarshall's subscribed feeds.

Scour all 25255 feeds Learn more about Feeds

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

Fixing a stuck Ollama runner and building a GPU watchdog

Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%

martidu4/honey-ai: 🍯 All-in-one AI honeypot powered by local LLMs. SSH, HTTP, FTP, Telnet, SMTP, MySQL, Redis, Git, VNC, RDP — with canary tokens, tarpits, GZIP bombs, and threat intel reporting.

NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

I finally built the central AI hub I've been wanting, and Open WebUI made it stupidly simple

local llm on laptop 780M GPU using llama + gemma 4 qat

Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

Running Qwen 35B MoE at 450k Context on a Single 32GB GPU

Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB

Show HN: Ext-Infer

Large companies can add a local LLM filter layer to considerably reducing their AI costs

mirkolenz/llmhop: Tiny, stateless Go router that dispatches OpenAI-compatible requests to single-model vLLM and sglang backends with zero external dependencies

zaydmulani09/mnemo: Local-first AI memory layer for any LLM. Persistent knowledge graph, entity extraction, semantic retrieval. Works with Ollama, OpenAI, Anthropic, or any OpenAI-compatible backend.

john-rocky/apple-silicon-llm-bench: Neutral, reproducible benchmark for local LLMs on Apple Silicon (Mac · iPhone · iPad) — MLX, llama.cpp, CoreML, Apple Foundation Models

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

KJLdefeated/RL.cu: RLVR training for LLM in CUDA/C++