Inference Optimization

Feeds to Scour
SubscribedAll
Scoured 291 posts in 6.4 ms

Pruned YOLOv8 ONNX INT8 Fails: 3 Fixes That Work

 🤖AI  Content type: Blog  Content type: Discussion
tildalice.io·

Qwen 3.6 27B AutoRound GGUF, need your feedback

 🧠OpenAI
huggingface.co··r/LocalLLaMA

Ollama 0.30 GPU Boost: Faster local Qwen inference on NVIDIA

 🦙Llama
everylocalai.com··DEV

UniSVQ: 2-bit Unified Scalar-Vector Quantization

 🤖LLM  Content type: Academic
arxiv.org·

Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%

 🤖LLM
zozo123.github.io··Hacker News

stable-diffusion.cpp/docs/quantization_and_gguf.md at master · leejet/stable-diffusion.cpp

 🧠OpenAI  Content type: Code
github.com··r/StableDiffusion

OpenCV 5 Debuts with Improved ONNX Support and Native AI Upgrades

 👁️Computer Vision  Content type: News
hackster.io·

lightmetal: GPU LLM Inference From a Single Java 25 JAR

 🤖LLM  Content type: Blog
adambien.blog·

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

 🤖LLM
vettedconsumer.com··Hacker News

HNSW vs LSH: How Elasticsearch hits 0.99 recall@10 at 15,000 QPS — and what it costs

 🔍RAG  Content type: Blog
elastic.co·

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

 🎯Fine-tuning  Content type: News  Content type: Blog
blog.google··Hacker News

PagedAttention vs Traditional KV Cache: How vLLM Reinvented GPU Memory for LLM Inference

 🪟Context Windows  Content type: Blog
medium.com
·

OpenCV 5.0 Computer Vision Library Released with Rewritten DNN Engine

 👁️Computer Vision
linuxiac.com·

Improved performance and model support with GGUF

 🦙Llama  Content type: Blog
ollama.com·

Train Models Faster with JAX and MaxText Using NVFP4 on NVIDIA Blackwell

 🦙Llama  Content type: News  Content type: Blog
developer.nvidia.com·

MoQ GGUFs and GSQ: Low-Bit GGUFs Are About to Get Much Better

 🎯Fine-tuning  Content type: News  Content type: Blog
kaitchup.substack.com··r/LocalLLaMA
Less-relevant results

NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI

 🤖LLM  Content type: Blog
blogs.nvidia.com·

DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200

 🤖AI  Content type: News
newsletter.semianalysis.com
··Hacker News

The latest Gemma 4 models use a training trick to slash their on-device memory footprint

 🎯Fine-tuning
androidauthority.com·

OpenCV 5 release - New DNN engine with enhanced ONNX and LLM/VLM support, Intel, Arm, and RISC-V hardware optimizations - CNX Software

 👁️Computer Vision  Content type: News
cnx-software.com·

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help