🔢 Quantization of LLMs - pleto

Holding the FP8 Quality Ceiling at 8-Bit Weights and Activations: INT8 and GGUF Post-Training Quantization of Ideogram 4.0 for Consumer GPUs

✨Model optimizations in LLMs Academic

arxiv.org·

Show HN: Ext-Infer

🧠Large Language Models (LLMs)

infer.displace.tech··Hacker News

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

✨Model optimizations in LLMs News Blog

blog.google··Hacker News

The Order Matters: Sequential Fine-Tuning of LLaMA for Coherent Automated Essay Scoring

🧠Large Language Models (LLMs) Academic

arxiv.org·

Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and 4-Bit Shouldn’t Be This Good

🧠Large Language Models (LLMs) Blog

towardsai.net·

NeuroBait: I fine-tuned a model to spark dopamine for ADHD brain

✨Model optimizations in LLMs Blog

huggingface.co·

Florian Brand, Prime Intellect research engineer, adopts Gemma 4 E4B 6-bit quantized as his primary local Mac LLM

🧠Large Language Models (LLMs) News

digg.com··Hacker News

I wired a fully offline voice loop to Ollama + LM Studio — 100% CPU, no GPU, nothing leaves your machine (Silero VAD + Parakeet STT + Supertonic TTS 3)

🚀LLM serving frameworks Code

github.com··r/LocalLLaMA

2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP

🔧Systems-level optimizations for LLM serving Blog

dnhkng.github.io·

not much happened today | AINews

✨Model optimizations in LLMs

news.smol.ai·

Launch HN: General Instinct (YC P26) – Frontier models on edge devices

🧠Large Language Models (LLMs) Discussion

news.ycombinator.com··Hacker News

1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM

🧠Large Language Models (LLMs)

smolhub.com··r/LocalLLaMA

Google Colab CLI opens runtimes to Claude Code and Codex

🤖Agents using LLMs

helpnetsecurity.com··r/ClaudeAI

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

🔧Systems-level optimizations for LLM serving Code

github.com··Hacker News, r/LLM

google/gemma-4-12B-it-qat-q4_0-gguf

🧠Large Language Models (LLMs)

huggingface.co·

146th airhacks tv: Rust, Java 25, AI Agents, BCE, Web Components, zunit, zb

🧠Large Language Models (LLMs) Blog

adambien.blog·

Semantic Grading of Written Answers in Low-Resource Language Bangla Using a Fine-Tuned Lightweight Language Model

✨Model optimizations in LLMs Academic

arxiv.org·

techjarves/Portable-AI-USB: A 100% offline, fully portable, zero-trace AI (Ollama + Llama 3 + AnythingLLM) that runs natively from a USB drive on Windows and Mac.

🧠Large Language Models (LLMs) Code

github.com·

LLM Research Papers: The 2026 List (January to May)

stable-diffusion.cpp/docs/quantization_and_gguf.md at master · leejet/stable-diffusion.cpp

Holding the FP8 Quality Ceiling at 8-Bit Weights and Activations: INT8 and GGUF Post-Training Quantization of Ideogram 4.0 for Consumer GPUs

Show HN: Ext-Infer

Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

The Order Matters: Sequential Fine-Tuning of LLaMA for Coherent Automated Essay Scoring

Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and 4-Bit Shouldn’t Be This Good

NeuroBait: I fine-tuned a model to spark dopamine for ADHD brain

Florian Brand, Prime Intellect research engineer, adopts Gemma 4 E4B 6-bit quantized as his primary local Mac LLM

I wired a fully offline voice loop to Ollama + LM Studio — 100% CPU, no GPU, nothing leaves your machine (Silero VAD + Parakeet STT + Supertonic TTS 3)

2x GH200 for LLM inference, Part 2: vLLM, DeepSeek V4 Flash, and MTP

not much happened today | AINews

Launch HN: General Instinct (YC P26) – Frontier models on edge devices

1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM

Google Colab CLI opens runtimes to Claude Code and Codex

harshuljain13/llm-inference-at-scale: A Practitioner handbook for production llm serving.

google/gemma-4-12B-it-qat-q4_0-gguf

146th airhacks tv: Rust, Java 25, AI Agents, BCE, Web Components, zunit, zb

Semantic Grading of Written Answers in Low-Resource Language Bangla Using a Fine-Tuned Lightweight Language Model

techjarves/Portable-AI-USB: A 100% offline, fully portable, zero-trace AI (Ollama + Llama 3 + AnythingLLM) that runs natively from a USB drive on Windows and Mac.