🧠 LLM Inference - aaaaa · Scour

Improved performance and model support with GGUF

🧠LLMs Blog

lightmetal: GPU LLM Inference From a Single Java 25 JAR

🧠LLMs Blog

adambien.blog·

Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%

🤖AI Engineering

zozo123.github.io··Hacker News

Ollama 0.30 delivers faster NVIDIA GPU performance and wider hardware support

alternativeto.net·

Neo-X7/Neo-AI: A fully offline AI assistant powered by Ollama. Stores and retrieves conversations using SQLite + LanceDB vector search. No cloud. No API keys. Runs entirely on your machine.

🧠LLMs Code

github.com··DEV

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

vettedconsumer.com··Hacker News

I've tested so many desktop AI tools, but Hermes with Ollama is my new favorite - here's why

🧠LLMs News Tutorial

2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP

🧠LLMs Blog

dnhkng.github.io·

RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference

🤖ai Academic

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

deemwar-products.github.io··Hacker News

Fixing a stuck Ollama runner and building a GPU watchdog

patrickmccanna.net··Hacker News

Less-relevant results

AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support

A system programmer’s guide to LLM inference

💬NLP Blog

blog.xiangpeng.systems··Hacker News

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

🧠LLMs Code

github.com··Hacker News

NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet

huggingface.co··Hacker News

Running LLM Inference on Kubernetes: What It Actually Takes

🧠LLMs Blog

fairwinds.com·

DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200

🤖Machine Learning News

newsletter.semianalysis.com

··Hacker News

MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better

🤖ai News Blog

kaitchup.substack.com··r/LocalLLaMA

Tales of an Ollama Honeypot (Part 3): More Traffic, More Findings

posts.inthecyber.com·

Using Scikit-LLM with Open-Source LLMs

machinelearningmastery.com·

Log in to enable infinite scrolling