Through the looking glass of benchmark hacking (opens in new tab)

Covered by 3 sources including giovannigatti.github.io, InterconnectsDiscussed on Hacker News

Outlining some of the reward hacks we’ve encountered and what strategies we are exploring to resolve them.

Read the original article

Sign in to keep reading the full article.

Covered in 3 articles

giovannigatti.github.io·

I Tested Whether AI Can Fix Security Vulnerabilities. Well, It's Complicated.

Discussed on Hacker News, Hacker News, and r/netsec

·

Latest open artifacts (#21): Open model bonanza! Gemma 4, DeepSeek V4, Kimi K2.6, MiMo 2.5, GLM-5.1 & others. On CAISI's V4 assessment.

AIエージェントが試験で一生懸命「カンニング」していることが発覚