Logits as a new monitor for evaluation awareness (opens in new tab)

TL;DR:We build a logit monitor for eval awareness: throughout the CoT, we estimate an LLM's probability of producing an eval-aware sentence.The logit monitor outperforms LLM judge monitoring of verbalized eval awareness, using 10× to 100× fewer rollouts, on Kimi K2.5 and Qwen 3 32B, across two tasks:separating evaluation prompts (Fortress & Petri) from deployment prompts (WildChat)predicting which evaluation prompts will elicit rollouts with verbalized eval awareness.The logit monitor works f...