🗜️ Quantization - nayyara.airlangga · Scour

Zero-Shot Quantization via Weight-Space Arithmetic ⚡FlashAttention

arxiv.org·3d

I Ran My KYB Engine at Three Quantization Levels. Accuracy Didn't Move. Cost Dropped 6x. 🚀Speculative Decoding

walsenburgtech.com·1d·Hacker News

Cooperative Vectors Introduction ⚗️Kernel Fusion

evolvebenchmark.com·6h·Hacker News

Attn-QAT: Making 4-Bit Attention Actually Work ⚡FlashAttention

haoailab.com·2d

jagmarques/nexusquant: Training-free KV cache compression for LLMs. 10-33x compression via E8 lattice quantization + attention-aware token eviction. One line of code. 💾KV Cache

github.com·3d·DEV

Breaking the Memory Wall: TurboQuant KV Cache Quantization on Apple Silicon 💾KV Cache

pub.towardsai.net

·1d

GGUF vs AWQ vs EXL2: What's Inside Your LLM Model Files 🚀Speculative Decoding

femiadeniran.com·6d·r/LocalLLaMA

New course: Efficient Inference with SGLang: Text and Image Generation, built in partnership with LMSys @lmsysorg and RadixArk @radixark, and taught by Richard ... 🚀Speculative Decoding

twitter.macworks.dev

·1d

TurboQuant: The Compression Algorithm That Just Made Your Vector Database Obsolete ⚙️ML Compilers

danwichoudhary.medium.com·4d

Jackrong/Qwopus3.5-27B-v3-GGUF 🚀Speculative Decoding

huggingface.co·5d

Initialisation Determines the Basin: Efficient Codebook Optimisation for Extreme LLM Quantization 🧠Inference Engineering

arxiv.org·15h

How Do You Actually Scale High-Throughput LLM Serving in Production with vLLM? ⚗️Kernel Fusion

medium.com·5d

Evolving Side-Channel Attacks with Genetic Algorithms 🧠Inference Engineering

cong-or.xyz·4d

Spilling the Neural Tea: A Journey Down the Side-Channel ⚡FlashAttention

sigarch.org·4d·Hacker News

Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs 🚀Speculative Decoding

arxiv.org·15h

AI breakthrough cuts energy use by 100x while boosting accuracy 🚀Speculative Decoding

sciencedaily.com·4d·Hacker News, r/singularity

Intel QAT Driver With Linux 7.1 Adding Zstd Offload Support 🐝eBPF

phoronix.com·3d

Neural Networks Explained with a Biryani: How Models Actually Learn 🧠Inference Engineering

ganeshkumarm1.medium.com·3d

New method makes neural networks three times faster in wave propagation problems 🚀Model Serving

techxplore.com·3d

Weight Group-wise Post-Training Quantization for Medical Foundation Model 🚀Speculative Decoding

arxiv.org·15h

Loading more...