LLM Inference

Feeds to Scour
SubscribedAll
Scoured 327 posts in 8.0 ms

Improved performance and model support with GGUF

 🧠LLMs  Content type: Blog
ollama.com·

lightmetal: GPU LLM Inference From a Single Java 25 JAR

 🧠LLMs  Content type: Blog
adambien.blog·

DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200

 🤖Machine Learning  Content type: News
newsletter.semianalysis.com
··Hacker News

Ollama 0.30 delivers faster NVIDIA GPU performance and wider hardware support

 🧠LLMs
alternativeto.net·

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

 🧠LLMs  Content type: Code
github.com··Hacker News
Less-relevant results

Token4Token — pay-per-token inference on Gnosis + Swarm

 🧠LLMs
t4t.eth.link··Hacker News

2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP

 🧠LLMs  Content type: Blog
dnhkng.github.io·

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

 🤖ai
vettedconsumer.com··Hacker News

Clairvoyant: Predictive SJF Scheduling to Mitigate Head-of-Line Blocking in Serial LLM Backends

 🧠LLMs  Content type: Academic
arxiv.org·

DeskDash - a free Windows tool to easily manage your GGUF files

 🔍RAG
gerry7.itch.io··r/LocalLLaMA

Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

 🧠LLMs
deemwar-products.github.io··Hacker News

I built an open-source persistent memory layer for AI coding agents

 🧠LLMs  Content type: Code
github.com··r/GithubCopilot

Fixing a stuck Ollama runner and building a GPU watchdog

 🧠LLMs
patrickmccanna.net··Hacker News

Running LLM Inference on Kubernetes: What It Actually Takes

 🧠LLMs  Content type: Blog
fairwinds.com·

Self-hosted remote access for Ollama without complicated setup

 🧠LLMs
oab.arc-i.co.uk··r/selfhosted

MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better

 🤖ai  Content type: News  Content type: Blog
kaitchup.substack.com··r/LocalLLaMA

A system programmer’s guide to LLM inference

 💬NLP  Content type: Blog
blog.xiangpeng.systems··Hacker News

NexusOS v2.0 – A zero-dependency pipeline streaming server chaos to Parquet

 🧠LLMs
huggingface.co··Hacker News

Using Scikit-LLM with Open-Source LLMs

 🧠LLMs
machinelearningmastery.com·

147th airhacks tv: Local LLMs, LightMetal, ZSmith Agents, AI Rails, Saving Tokens

 🧠LLMs  Content type: Blog
adambien.blog·

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help