Anthropic Taps Theologen Claude Moral Framework zu gestalten

Anthropic has been consulting theologians and ethicists on Claude's behavior, raising questions about who gets to shape a chatbot's values. In late March, around 15 religious thinkers met with the artificial intelligence company to discuss how to teach a chatbot to be good. The aim wasn't to make the chatbot Bible-thumping or pious, but to acknowledge that centuries-old traditions of moral reasoning might offer insights to a five-year-old frontier AI lab whose systems are becoming more capable, more persuasive, and harder to govern by simple rules.

The Pope's Warning On May 25, Pope Leo XIV presented his first encyclical, Magnifica Humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence, an about 40,000-word treatise calling for AI to be disarmed—not rejected but freed from the assumption that technical power automatically confers the right to govern.

Shaping Claude's Constitution The goal of the meetings in late March was to help refine what Anthropic calls Claude's constitution, a written set of principles the company uses to shape how the model responds, including by training Claude to critique and revise its own answers against those principles. Anthropic is looking for what works and may try religiously informed ideas or techniques to see whether they improve model behavior. Not everyone is convinced that religious consultation solves the accountability problem. Carissa Véliz, an AI ethicist at the University of Oxford, notes that the incentives of Anthropic's business model may push the company toward unethical decisions, regardless of its intentions. The fact that a leading company is now consulting theologians is either a rare sign of humility or of an industry improvising its ethics in real time—possibly both. As the development of AI chatbots like Claude continues to accelerate, the question of who gets to shape their values and how they are held accountable will only become more pressing. With hundreds of millions of people now talking to AI chatbots every week, the values their developers bake in via guardrails and corrective tuning shape what those models say about everything from end-of-life care to abortion to managing grief.

Source: Anthropic asks religious thinkers to help shape Claude as pope warns about AI
Domain: scientificamerican.com