📊 LLM Evaluation - ghosh.debasish · Scour

The LLM Evaluation Playbook Every AI Engineer Needs 🤖AI Agents

medium.com

·4d

Benchmarking LLMs with Marimo Pair ⚙️Compilers

ericmjl.github.io·15h·Hacker News

Does Pass Rate Tell the Whole Story? Evaluating Design Constraint Compliance in LLM-based Issue Resolution 🧠LLMs

arxiv.org·2d

A Deep Dive into LLM Evaluation Metrics: From Perplexity to Production 🧠LLMs

medium.com·20h

Better Harness: A Recipe for Harness Hill-Climbing with Evals 🧠Reasoning Models

blog.langchain.com·1d

A Fast and Loose Clustering of LLM Benchmarks 🧠Reasoning Models

lesswrong.com·12h

Give an LLM an API and It'll Thrive. Give It a Touchscreen and It Struggles 🧪Property-based Testing

blog.allada.com·4d·Hacker News, Hacker News

I benchmarked my own product, published everything, and 0.2.0 is basically the list of things I had to fix. 🖥️Systems Programming

blog.routerly.ai·2d·r/SideProject

How We Evaluate Search Quality at Scale With LLM Judging and IR Metrics 🧠Reasoning Models

medium.com·19h

Are Local LLMs Good Enough for Vibe Coding? Gemma4-26B-A4B vs. Qwen3.5-35B-A3B 🧠LLMs

grigio.org·2d·Hacker News

Show HN: Pre-training, fine-tuning, and evals platform 🧠Reasoning Models

oumi.ai·6d·Hacker News

Show HN: Proposal for a real long-term AI memory benchmark 🧠Reasoning Models

penfieldlabs.substack.com·20h·Substack

Overshoot-ai/vlm-benchmarks: 2500+ VLM benchmarks, auto-updated daily from arXiv 🌊Streaming Algorithms

github.com·1d·Hacker News, r/LocalLLaMA

Benchmarks are the new stars 🧠Reasoning Models

mercurialsolo.github.io·19h

Why Measuring AI Features Is Nothing Like Measuring Regular Software 🧠Reasoning Models

medium.com·4d

You Fine-Tuned Your Model. Now It’s Worse. Here’s the Concept You Were Never Taught. 🧠LLMs

pub.towardsai.net

·1d

Meta Muse Spark: What the Benchmarks Actually Mean 🧠Reasoning Models

medium.com·20h

Our AI Hallucinated in Production: How We Fixed It With Evals — Yicheng Guo at AI Engineer Melbourne 2026 🧪Property-based Testing

webdirections.org·2d

Semantic Layer vs. Text-to-SQL: 2026 Benchmark Update (11 minute read) 🛢️Database Internals

docs.getdbt.com·3d

CivBench: Progress-Based Evaluation for LLMs' Strategic Decision-Making in Civilization V 🤖AI Agents

arxiv.org·9h

Loading more...