https://www.anthropic.com/research/exploit-evals (opens in new tab)

We've developed two new, challenging academic benchmarks measuring AI models’ ability to develop exploits, and an updated version of the benchmark measuring smart contract exploitation.