💻 Local AI - ibrahimsharaf · Scour

Inference Arena – new benchmark of local inference and training 🚀LLM Deployment

kvark.github.io·4d·Hacker News

Blink: CPU-Free LLM Inference by Delegating the Serving Stack to GPU and SmartNIC 🚀LLM Deployment

arxiv.org·9h

Article: Building a Voice-Controlled Local AI Agent 🔬Small LMs

medium.com·4h

RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference 🚀LLM Deployment

vldb.org·1d

Run Qwen3.5 on an Old Laptop: A Lightweight Local Agentic AI Setup Guide 🚀LLM Deployment

kdnuggets.com·1d

The Engine Behind Modern LLM Inference, Part 1: Continuous Batching, PagedAttention, and the End of… 🚀LLM Deployment

medium.com·20h

LLM inference engine from scratch in C++ 🚀LLM Deployment

anirudhsathiya.com·4d·Hacker News

Best Open Source Offline AI Agent 🔓Open Source AI

news.ycombinator.com·22h·Hacker News

Ollama is still the easiest way to start local LLMs, but it's the worst way to keep running them 🎯LLM Finetuning

xda-developers.com·1d

Building Local AI Video Generation Rig: A Hardware Breakdown 🚀LLM Deployment

hackster.io·5h

Google’s new AI app is a glimpse of the future 🔬Small LMs

computerworld.com·5h·Hacker News

ASUS UGen300 USB AI Accelerator 8GB for local inference 🚀LLM Deployment

asus.com·4h·r/StableDiffusion

EU's Exposed AI Infrastructure 🚀LLM Deployment

insecurestack.substack.com·2d·Substack

TurboQuant Explained: Extreme AI Compression for Faster, Cheaper LLM Inference and Vector Search 🚀LLM Deployment

medium.com

·5d

lunargate-ai/gateway: High-performance self-hosted AI gateway (OpenAI-compatible) with routing, retries, and streaming 🚀LLM Deployment

github.com·2h·Hacker News

Running Gemma 4 Locally with Ollama on Your PC 🚀LLM Deployment

analyticsvidhya.com·1d

We burned $200 learning that local AI wasn't about the money 🚀LLM Deployment

write.as·22h

F&S M.2 AI Accelerator Uses NXP Ara-240 for Edge Inference Workloads 🚀LLM Deployment

linuxgizmos.com·9h

Decentralized AI in 50 Lines of Python 🔓Open Source AI

iamtrask.github.io·3d

Inside LLM Inference: KV Cache, Prefill, and the Decode Bottleneck 🚀LLM Deployment

pub.towardsai.net

·1d

Loading more...