Anthropic's Paradox: AI Safety Through Leadership

Anthropic has spent the last five years warning the world about the potential dangers of advanced artificial intelligence, from enabling mass destruction to destabilizing society. Simultaneously, the company has become a major force in advancing AI capabilities, developing and distributing cutting-edge models, and courting high-profile customers like the US military. Recently valued at almost $1 trillion, Anthropic's dual approach may seem contradictory.

Anthropic operates based on two core beliefs. The first is that artificial intelligence is the most transformative technology in human history, and its arrival is inevitable. The only real question is whether it leads to catastrophe or extraordinary prosperity.

The second belief is that the world will be better off if Anthropic remains at the frontier of the AI race, according to former employees. Internally, leaders and employees see themselves as the "good guys," responsible stewards of AI technology, accumulating power not as an end in itself but as the price of fulfilling its mission: "to ensure the world safely makes the transition through transformative AI." Helen Toner, executive director of Georgetown's Center for Security and Emerging Technology and a former OpenAI board member, uses an analogy to describe Anthropic's worldview. She compares powerful AI to a forest filled with both magical treasures and dangerous monsters.

All the villagers are rushing in, lured by the treasure. In her telling, Anthropic wants to venture farther into the forest than anyone else while investing heavily in taming the monsters—that is, capturing AI's benefits while containing its catastrophic risks. "What's distinctive about Anthropic is they're like, 'People are going in the forest anyway, we have to do it first.' This is very explicitly their strategy: build cutting-edge AI in order to be a serious player at the table who can talk about what cutting-edge AI systems look like, what risks they pose, and pushing for reasonable safeguards," Toner says.

"They're very straightforward about this. It's just a weird enough strategy that people have a hard time hearing it." Anthropic CEO Dario Amodei outlined this approach plainly in a conversation with his cofounders posted on the company's career page: "You have to find a way to actually be competitive, to actually lead the industry in some cases, and yet manage to do things safely," he says. "If you can do that, the gravitational pull you exert is so great." Anthropic was founded in 2021 by a group of former OpenAI employees who defected after losing faith in the ability of the company's leadership—particularly CEO Sam Altman—to safely bring transformational AI into the world.