📊 LLM Evaluation - alvin.kuruvilla · Scour

The LLM Evaluation Playbook Every AI Engineer Needs ✍️Prompt Engineering

medium.com

·4d

A Deep Dive into LLM Evaluation Metrics: From Perplexity to Production ✍️Prompt Engineering

medium.com·22h

Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios ⚙️MLOps

arxiv.org·1d

Benchmarking LLMs with Marimo Pair ⚙️MLOps

ericmjl.github.io·17h·Hacker News

Overshoot-ai/vlm-benchmarks: 2500+ VLM benchmarks, auto-updated daily from arXiv ⚙️MLOps

github.com·1d·Hacker News, r/LocalLLaMA

A Fast and Loose Clustering of LLM Benchmarks 🚀Performance Engineering

lesswrong.com·13h

Semantic Layer vs. Text-to-SQL: 2026 Benchmark Update (11 minute read) 🗄️Database Internals

docs.getdbt.com·3d

Show HN: Proposal for a real long-term AI memory benchmark 📱Edge AI

penfieldlabs.substack.com·22h·Substack

Give an LLM an API and It'll Thrive. Give It a Touchscreen and It Struggles ⚙️MLOps

blog.allada.com·4d·Hacker News, Hacker News

How We Evaluate Search Quality at Scale With LLM Judging and IR Metrics ⚡Query Optimization

medium.com·21h

Show HN: Pre-training, fine-tuning, and evals platform ⚙️MLOps

oumi.ai·6d·Hacker News

April 7, 2026 (#4641) 🤝AI Agents

alvinashcraft.com·3d

The LLM Effect on IR Benchmarks: A Meta-Analysis of Effectiveness, Baselines, and Contamination ✍️Prompt Engineering

arxiv.org·2d

EVGeoQA: Benchmarking LLMs on Dynamic, Multi-Objective Geo-Spatial Exploration ⚡Query Optimization

arxiv.org·1d

smoothyy3/willitrun: CLI to tell you if an ML model will fit and run on your device, using real benchmarks + lightweight estimation. ⚙️MLOps

github.com·3d·Hacker News

Does Pass Rate Tell the Whole Story? Evaluating Design Constraint Compliance in LLM-based Issue Resolution ✍️Prompt Engineering

arxiv.org·2d

Prediction Arena: Benchmarking AI Models on Real-World Prediction Markets 📱Edge AI

arxiv.org·11h

Beyond Accuracy: Diagnosing Algebraic Reasoning Failures in LLMs Across Nine Complexity Dimensions ✍️Prompt Engineering

arxiv.org·1d

Sell More, Play Less: Benchmarking LLM Realistic Selling Skill ✍️Prompt Engineering

arxiv.org·1d

SQLStructEval: Structural Evaluation of LLM Text-to-SQL Generation 🗄️Database Internals

arxiv.org·1d

Loading more...