Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
GEMM Optimization
🔢 GEMM Optimization
Specific
matrix multiply, cuBLAS, GEMM kernel, tiling, compute-bound
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
51
posts in
6.7
ms
DiffusionGemma: 4x Faster Text Generation
🧠
Inference Engineering
Content type:
News
Content type:
Blog
blog.google
·
7h
7 hours ago
·
Hacker News
,
r/LocalLLaMA
,
r/singularity
Actions for DiffusionGemma: 4x Faster Text Generation
Two Leaps to 1000 Tokens/s on a 1T-Parameter Model: On Inference Systems, Execution
Boundaries
, and
Co-Design
🧠
Inference Engineering
Content type:
Blog
tilert.ai
·
2d
2 days ago
·
Hacker News
Actions for Two Leaps to 1000 Tokens/s on a 1T-Parameter Model: On Inference Systems, Execution Boundaries, and Co-Design
Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a
GEMM-Centric
Taxonomy
💰
Inference Cost
Content type:
Academic
arxiv.org
·
1d
1 day ago
Actions for Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy
MLPerf and the rise of latency-aware LLM benchmarking
⏱️
Prefill Decoding
edn.com
·
5d
5 days ago
Actions for MLPerf and the rise of latency-aware LLM benchmarking
DiffusionGemma: The Developer Guide
🧠
Inference Engineering
Content type:
Blog
developers.googleblog.com
·
23h
23 hours ago
Actions for DiffusionGemma: The Developer Guide
KaiFelixBennett/gemma4-turboquant-rdna4
: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4
GPU
(gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.
⏱️
Prefill Decoding
Content type:
Code
github.com
·
7h
7 hours ago
·
Hacker News
Actions for KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.
The Inference Alpha: Maximizing Frontier Models on AMD
🧠
Inference Engineering
Content type:
Blog
digitalocean.com
·
9h
9 hours ago
Actions for The Inference Alpha: Maximizing Frontier Models on AMD
AutoMegaKernel: A Statically-Checked Agent Harness for Self-Retargeting Megakernel Synthesis
🎮
GPU Computing
Content type:
Academic
arxiv.org
·
1d
1 day ago
·
Hacker News
Actions for AutoMegaKernel: A Statically-Checked Agent Harness for Self-Retargeting Megakernel Synthesis
harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
💾
KV Cache
Content type:
Code
github.com
·
4d
4 days ago
·
Hacker News
Actions for harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.
Discrete Diffusion Modelling by Estimating the Ratios of the Data Distribution
🚀
Speculative Decoding
Content type:
News
Content type:
Blog
leetarxiv.substack.com
·
1d
1 day ago
·
Substack
,
r/programming
Actions for Discrete Diffusion Modelling by Estimating the Ratios of the Data Distribution
Density Field State Space Models: 1-Bit Distillation, Efficient Inference, and Knowledge Organization in Mamba-2
🎮
GPU Computing
Content type:
Academic
arxiv.org
·
19h
19 hours ago
Actions for Density Field State Space Models: 1-Bit Distillation, Efficient Inference, and Knowledge Organization in Mamba-2
bigattichouse/packed-twin-inference: PTI achieves ~2× throughput using a single quantized model (Q5_K_M or better) by running 4 generation streams in one batched decode call. The
GPU
loads model weights once per step and produces 4 predictions simultaneously. KV cache overhead is ~0.8 GiB total for all 4 streams. No draft model. No quality loss
🧠
Inference Engineering
Content type:
Code
github.com
·
1d
1 day ago
·
r/LocalLLaMA
Actions for bigattichouse/packed-twin-inference: PTI achieves ~2× throughput using a single quantized model (Q5_K_M or better) by running 4 generation streams in one batched decode call. The GPU loads model weights once per step and produces 4 predictions simultaneously. KV cache overhead is ~0.8 GiB total for all 4 streams. No draft model. No quality loss
What Arm-based innovations happened in May 2026?
🧠
Inference Engineering
Content type:
Blog
newsroom.arm.com
·
5d
5 days ago
Actions for What Arm-based innovations happened in May 2026?
K-Forcing: Joint Next-K-Token Decoding via Push-Forward Language Modeling
🚀
Speculative Decoding
Content type:
Academic
arxiv.org
·
19h
19 hours ago
Actions for K-Forcing: Joint Next-K-Token Decoding via Push-Forward Language Modeling
Apples to Apples: MLX vs. Llama.cpp for
Gemma
4 12B on an M1 16GB
💰
Inference Cost
Content type:
Blog
ziraph.com
·
5d
5 days ago
·
Hacker News
Actions for Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB
PALUTE:
Processing-In-Memory
Acceleration via Lookup Table for Edge LLM Inference
💰
Inference Cost
Content type:
Academic
arxiv.org
·
1d
1 day ago
Actions for PALUTE: Processing-In-Memory Acceleration via Lookup Table for Edge LLM Inference
DeepSeek V4, LeCun's Bet Against LLMs, and Lovable's Self-Improving Agent - The Tokenizer Edition #30
🔢
FP8 Training
newsletter.artofsaience.com
·
6d
6 days ago
Actions for DeepSeek V4, LeCun's Bet Against LLMs, and Lovable's Self-Improving Agent - The Tokenizer Edition #30
Benchmarking dots.tts on Strix Halo
🎮
GPU Computing
sleepingrobots.com
·
2d
2 days ago
Actions for Benchmarking dots.tts on Strix Halo
From Database and Virtualized Workloads to Backup: Dell PowerEdge R4715 and R5715 for SMB Realities
🕸️
Network Fabrics
storagereview.com
·
5d
5 days ago
Actions for From Database and Virtualized Workloads to Backup: Dell PowerEdge R4715 and R5715 for SMB Realities
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
🧠
Inference Engineering
Content type:
News
newsletter.semianalysis.com
·
1d
1 day ago
·
Hacker News
Actions for DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
« Page 1
·
Page 3 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help