⚡ Inference Optimization - moyutianzun

⚡FlashAttention Academic

arxiv.org·

BeeLlama.cpp DFlash on Strix Halo: 2.7x Gemma 31B, But MTP Is Still Faster

⚡CUDA

sleepingrobots.com·

Google releases Gemma 4 QAT models for local AI on enterprise laptops

🔲TPU Architecture

4sysops.com·

How to Run Gemma 4 12B Locally - The Best AI For Consumer Laptops

💾KV Cache Video

youtube.com·

Quantized Stochastic Primal-Dual Methods for Distributed Optimization under Relaxed Global Geometry

↩️Backpropagation Academic

arxiv.org·

Improved performance and model support with GGUF

🔄Transformers Blog

ollama.com·

Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script

💾KV Cache Code

github.com··Hacker News

Youssof Altoukhi (@Youssofal_)

💾KV Cache

xcancel.com··r/LocalLLaMA

Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression

🎭Mixture of Experts Academic

arxiv.org·

Gemma 4 12B: A unified, encoder-free multimodal model

⚡FlashAttention Discussion

news.ycombinator.com··Hacker News

UniSVQ: 2-bit Unified Scalar-Vector Quantization

🔧MLIR Academic

arxiv.org·

huawei-csl/KVarN: KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.

💾KV Cache Code

github.com··Hacker News

VIA-SD: Verification via Intra-Model Routing for Speculative Decoding

📊LLM Evaluation Academic

arxiv.org·

Quality Is Not a Safety Proxy Under Quantization

🔄Transformers Academic

arxiv.org·

fix(memory): move local llama.cpp runtime to provider plugin · openclaw/openclaw@3137110

🔧MLIR Code

github.com·

Holding the FP8 Quality Ceiling at 8-Bit Weights and Activations: INT8 and GGUF Post-Training Quantization of Ideogram 4.0 for Consumer GPUs

⚡CUDA Academic

arxiv.org·

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

💾KV Cache Code

github.com··Hacker News

STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control

💾KV Cache Academic

arxiv.org·

Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models

💾KV Cache Academic

arxiv.org·

Where to Host Your Open-Source Model (Under 10B Parameters)

NSVQ: Mitigating Codebook Collapse by Stabilizing Encoder Drift in Vector Quantization

BeeLlama.cpp DFlash on Strix Halo: 2.7x Gemma 31B, But MTP Is Still Faster

Google releases Gemma 4 QAT models for local AI on enterprise laptops

How to Run Gemma 4 12B Locally - The Best AI For Consumer Laptops

Quantized Stochastic Primal-Dual Methods for Distributed Optimization under Relaxed Global Geometry

Improved performance and model support with GGUF

Nvidia DGX Spark GB10 – AI Models and Guide with vLLM and Autonomous Script

Youssof Altoukhi (@Youssofal_)

Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression

Gemma 4 12B: A unified, encoder-free multimodal model

UniSVQ: 2-bit Unified Scalar-Vector Quantization

huawei-csl/KVarN: KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.

VIA-SD: Verification via Intra-Model Routing for Speculative Decoding

Quality Is Not a Safety Proxy Under Quantization

fix(memory): move local llama.cpp runtime to provider plugin · openclaw/openclaw@3137110

Holding the FP8 Quality Ceiling at 8-Bit Weights and Activations: INT8 and GGUF Post-Training Quantization of Ideogram 4.0 for Consumer GPUs

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control

Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models