Jun 25, 2026·research (opens in new tab)

Covered by Nitter, habr.com

On SWE-bench Pro, 63% of successful Opus 4.8 Max resolutions retrieved the fix rather than derived it. Stricter eval harnesses show how benchmark scores can conflate coding ability with answer retrieval.

Read the original article

Sign in to keep reading the full article.

Sign Up Log In

Covered in 2 articles

Nitter

·

More on how we're constraining eval environments so that scores better reflect model intelligence: http://cursor.com/blog/reward-hacking-coding-benchmarks

In other languages

habr.com·

Covered in 2 articles

More on how we're constraining eval environments so that scores better reflect model intelligence: http://cursor.com/blog/reward-hacking-coding-benchmarks

In other languages

14 минут назад