We Should Train Frontier AIs on a Synthetic World, Not Ours (opens in new tab)

Epistemic status: I think the core idea could actually be built. My real doubt is whether anyone with the compute will ever bother to try it. I pitched a version of this on a recent Doom Debates livestream with Liron Shapira, got pushback in real time, and want to lay the argument out more carefully than a janky phone connection allowed.The single most dangerous thing the labs do, and they do it on purpose, is train the model on the real world. The entire human corpus goes in, so the model le...