posted Feb 3 @ 01:29 PM by Jason Kottke · gift link
A couple of weeks ago, AI company Anthropic published the constitution that they use to train their Claude LLM (“under a Creative Commons CC0 1.0 Deed, meaning it can be freely used by anyone for any purpose without asking for permission”). From the company’s news release:
We’re publishing a new constitution for our AI model, Claude. It’s a detailed description of Anthropic’s vision for Claude’s values and behavior; a holistic document that explains the context in which Claude operates and the kind of entity we would like Claude to be.
The constitut…
posted Feb 3 @ 01:29 PM by Jason Kottke · gift link
A couple of weeks ago, AI company Anthropic published the constitution that they use to train their Claude LLM (“under a Creative Commons CC0 1.0 Deed, meaning it can be freely used by anyone for any purpose without asking for permission”). From the company’s news release:
We’re publishing a new constitution for our AI model, Claude. It’s a detailed description of Anthropic’s vision for Claude’s values and behavior; a holistic document that explains the context in which Claude operates and the kind of entity we would like Claude to be.
The constitution is a crucial part of our model training process, and its content directly shapes Claude’s behavior. Training models is a difficult task, and Claude’s outputs might not always adhere to the constitution’s ideals. But we think that the way the new constitution is written — with a thorough explanation of our intentions and the reasons behind them — makes it more likely to cultivate good values during training.
The full document is 80+ pages, but the news release does a decent job in summarizing what’s in it.
Claude’s constitution is the foundational document that both expresses and shapes who Claude is. It contains detailed explanations of the values we would like Claude to embody and the reasons why. In it, we explain what we think it means for Claude to be helpful while remaining broadly safe, ethical, and compliant with our guidelines. The constitution gives Claude information about its situation and offers advice for how to deal with difficult situations and tradeoffs, like balancing honesty with compassion and the protection of sensitive information. Although it might sound surprising, the constitution is written primarily for Claude. It is intended to give Claude the knowledge and understanding it needs to act well in the world.
We treat the constitution as the final authority on how we want Claude to be and to behave — that is, any other training or instruction given to Claude should be consistent with both its letter and its underlying spirit. This makes publishing the constitution particularly important from a transparency perspective: it lets people understand which of Claude’s behaviors are intended versus unintended, to make informed choices, and to provide useful feedback. We think transparency of this kind will become ever more important as AIs start to exert more influence in society.
Casey Newton and Kevin Roose recently interviewed the primary author of the constitution, philosopher Amanda Askell, for the Hard Fork podcast (the segment starts at ~25min).
Newton says the document reads like “a letter from a parent to a child maybe who’s leaving for college”:
And it’s like, we hope that you take with you the values that you grew up with. And we know we’re not going to be there to help you through every little thing, but we trust you. And good luck.
Both the constitution and the conversation with Askell are fascinating, no matter where you lie on the AI debate continuum. You might also be interested in this video of Askell answering questions from Claude users about her work: