Why AI Agent Policies Must Be Deterministic, Not Probabilistic (opens in new tab)

There's a philosophical split in how the AI industry thinks about agent safety. One camp says the model should govern itself — better prompts, better training, better alignment. The other says external enforcement is necessary because models are inherently probabilistic and shouldn't be trusted to enforce their own constraints. Both camps are partially right — but in practice, almost every production MCP agent today relies entirely on the first approach. Safety rules live in the system prompt...