AI tests expose limits in math and attention (opens in new tab)

Recent science coverage described new tests of large language models that probe whether chatbot performance reflects durable reasoning or narrower benchmark skill. Nautilus reported that a new AI test suggests some newer LLMs perform less intelligently than older models, while Scientific American reported that the second batch of “First Proof” problems was designed to evaluate AI usefulness for research-level mathematics. On the math test, the strongest model solved six or seven of 10 questio...