The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning (opens in new tab)

The White House Executive Order on Artificial Intelligence highlights the risks of large language models (LLMs) empowering malicious actors in developing biological, cyber, and chemical weapons.1 In collaboration with a consortium of experts, we release the Weapons of Mass Destruction Proxy (WMDP) benchmark, an extensive dataset of questions that serve as a proxy measurement of hazardous knowledge in biology, chemistry, and cybersecurity. Using this benchmark, we develop ‘CUT’—a state-of-the-...