⚡ LLM Quantization - akapaka · Scour

I wired a fully offline voice loop to Ollama + LM Studio — 100% CPU, no GPU, nothing leaves your machine (Silero VAD + Parakeet STT + Supertonic TTS 3)

🧠Local llm Code

github.com··r/LocalLLaMA

OpenRTLSet: A Fully Open-Source Dataset for Large Language Model-based Verilog Module Design

🤖Qwen Academic

google/gemma-4-12B-it-qat-q4_0-gguf

huggingface.co·

Apple rebuilt its on-device AI stack at WWDC 2026

🤖Machine Learning Blog

ziraph.com··Hacker News

not much happened today | AINews

Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script

🧠LLM Inference Code

github.com··Hacker News

heterodoxin/graphkv: Graph-guided KV cache compression for memory-efficient LLM inference.

🧠LLM Inference Code

github.com··r/LocalLLaMA

Where to Host Your Open-Source Model (Under 10B Parameters)

🧠LLM Inference

digitalocean.com·

1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM

smolhub.com··r/LocalLLaMA

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

🧠LLM Inference Code

github.com··Hacker News

FADA: Accessible fetal ultrasound interpretation and annotation with a selectively distilled unified vision-language model

🧠Local llm Academic

Using local LLMs for agentic coding

🧠Local llm Blog

blog.alexewerlof.com·

bigattichouse/packed-twin-inference: PTI achieves ~2× throughput using a single quantized model (Q5_K_M or better) by running 4 generation streams in one batched decode call. The GPU loads model weights once per step and produces 4 predictions simultaneously. KV cache overhead is ~0.8 GiB total for all 4 streams. No draft model. No quality loss

🧠LLM Inference Code

github.com··r/LocalLLaMA

Show HN: Ext-Infer

infer.displace.tech··Hacker News

Florian Brand, Prime Intellect research engineer, adopts Gemma 4 E4B 6-bit quantized as his primary local Mac LLM

🤖Qwen News

digg.com··Hacker News

Qwen3.6 + MTP: Calculated context size is smaller when I use `--spec-draft-type-* q4_0`. is this normal? · ggml-org llama.cpp · Discussion #24102

🧠Local llm Discussion Code

github.com··r/LocalLLaMA

Does anyone know what PCIe mode was used for these benchmarks?

🧠LLM Inference Code

github.com··r/LocalLLaMA

2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP

🧠LLM Inference Blog

dnhkng.github.io·

defai-digital/ax-engine: Apple Silicon LLM runtime supporting Gemma 4 and Qwen 3.6 MTP modes

🤖Qwen Code

github.com··Hacker News

NeuroBait: I fine-tuned a model to spark dopamine for ADHD brain

🧠Local llm Blog

huggingface.co·

Sign up or log in to see more results

Log in to enable infinite scrolling