🏗️ LLM Infrastructure - emschwartz

google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation

⚡Fast AI Inference

huggingface.co··r/LocalLLaMA

NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI

🤖AI Blog

blogs.nvidia.com·

How we fight GPU scarcity without compromise

🧠LLM Inference Blog

equixly.com··Hacker News

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

🤖AI Code

github.com··Hacker News

1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM

🤖AI

smolhub.com··r/LocalLLaMA

RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference

💾Prompt Caching Academic

arxiv.org·

AI Serving Platform That Adapts to Your Model

🤖AI Blog

databricks.com·

CUDA-Oxide 0.2 Brings Early Improvements To Pure Rust CUDA Kernels

🔓Open Source Software

phoronix.com·

A system programmer’s guide to LLM inference

🤖AI Blog

blog.xiangpeng.systems··Hacker News

Core Automation co-founder Jerry Tworek jokes that Nvidia's CUDA translates to miracles in Polish

⚡Hardware Acceleration

digg.com·

Mobile AI Compute Engine (MACE) inference framework — Vision SDK

🔍FAISS Blog

mapbox.com·

Release TorchCodec 0.14: HDR Video Decoding for CPU & CUDA, and Fast Wav Decoder · meta-pytorch/torchcodec

🕯️Candle Code

github.com··Hacker News

KJLdefeated/RL.cu: RLVR training for LLM in CUDA/C++

Breaking the Ice: Analyzing Cold Start Latency in vLLM

AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support

Inferoa AI harness claimed 90% cache savings. We ran it and measured 97.8%

DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200

Optimizing Local LLM Inference on Constrained Hardware

NVIDIA Nsight Compute

DiffusionGemma: The Developer Guide

google/gemma-4-31B-it · fix: chat template — null handling, reasoning preservation, turn-tag balance, input validation

NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI

How we fight GPU scarcity without compromise

KaiFelixBennett/gemma4-turboquant-rdna4: Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM

RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference

AI Serving Platform That Adapts to Your Model

CUDA-Oxide 0.2 Brings Early Improvements To Pure Rust CUDA Kernels

A system programmer’s guide to LLM inference

Core Automation co-founder Jerry Tworek jokes that Nvidia's CUDA translates to miracles in Polish

Mobile AI Compute Engine (MACE) inference framework — Vision SDK

Release TorchCodec 0.14: HDR Video Decoding for CPU & CUDA, and Fast Wav Decoder · meta-pytorch/torchcodec