One-in-Ten Odds of Human Extinction: What Anthropic’s Safety Crisis Reveals About the AI Race
A senior Anthropic safety researcher has stated there is more than a 10 percent chance AI could kill all humans by decade's end, hours after colleague Jacob Coxon resigned accusing Anthropic and OpenAI of recklessly racing toward self-improving superintelligence. The episode exposes a deepening gap between the industry's safety messaging and the alarming assessments of its own insiders.
When the People Building AI Warn You to Be Afraid
It is one thing for critics on the outside to warn that artificial intelligence poses existential dangers. It is quite another when the researchers on the inside — the people actively training these systems — stand up and say the same thing. That is precisely what happened in a remarkable sequence of events reported by The Verge, which you can read in full at theverge.com.
Within hours of a colleague resigning in protest, a senior safety researcher at Anthropic publicly stated that artificial intelligence carries more than a 10 percent chance of killing all humans by the end of this decade. To put that number in perspective: most people would refuse to board a plane, eat at a restaurant, or cross a bridge if they were told there was a one-in-ten chance it would kill them. Yet the AI industry is, according to these insiders, accelerating toward exactly that kind of risk.
Jacob Coxon’s Resignation — and What It Says
The immediate trigger for this wave of public concern was the departure of Jacob Coxon, a researcher who had trained AI systems at Anthropic and, before that, at OpenAI. Coxon announced his resignation in a post on X, and his language was unambiguous. He accused both Anthropic and OpenAI of “racing straight to self-improving superintelligence and gambling with our lives.”
This is not the language of a disgruntled employee leaving over a pay dispute. This is a technical expert — someone who understands from the inside how these systems are built — saying that the two most prominent AI safety-focused labs in the world are behaving recklessly. The phrase “self-improving superintelligence” is key here. Once an AI system becomes capable of improving its own design and capabilities without human guidance, the pace of development could accelerate far beyond anything humans can monitor, evaluate, or correct.
Coxon’s specific complaint was about the lax approach to safety he observed at Anthropic. The company has long positioned itself as the “responsible” AI lab — its founding story is literally that of former OpenAI employees leaving because they were worried about safety. If even Anthropic is now seen by its own researchers as cutting corners in a race to build superhuman systems, that positioning deserves serious scrutiny.
The 10 Percent Figure: Why It Matters
The number itself — more than a 10 percent probability of AI-caused human extinction — deserves careful consideration. In the field of risk analysis, a 10 percent probability of a catastrophic, irreversible outcome is treated as an emergency. Nuclear safety engineers, epidemiologists, and aerospace professionals work tirelessly to reduce risks that are orders of magnitude smaller than this.
The fact that a senior Anthropic safety researcher — someone whose job is literally to understand and reduce these risks — assigns greater-than-one-in-ten odds to human extinction should be alarming. It suggests that even within the companies building these systems, and even among the people most dedicated to making them safe, there is no confidence that the current trajectory ends well.
This is not a fringe position. Over the past few years, a growing number of AI researchers, including many who have worked at leading labs, have expressed similar concerns. What makes the current moment different is the speed of capability gains. AI systems are becoming dramatically more powerful in very short timeframes, and the safety research needed to understand and control those systems is not keeping pace.
The “Racing” Problem at the Heart of AI Development
Coxon’s resignation letter points to a structural problem that goes beyond any single company: the competitive race between AI labs. When multiple well-funded organizations are competing to be the first to achieve a technological milestone — in this case, superhuman or self-improving AI — each one faces enormous pressure to move faster than its rivals. Safety precautions, by definition, slow things down. This creates a perverse incentive where the labs that invest most heavily in safety risk falling behind those that invest less.
This dynamic is sometimes called a “race to the bottom” on safety, and it is one of the most frequently cited reasons why AI governance advocates argue that voluntary commitments by individual companies are insufficient. If Anthropic slows down to be more careful, OpenAI or a well-funded competitor may not. The result is that safety-conscious behavior is punished by the market, while recklessness is rewarded with first-mover advantage.
For Indian readers watching this unfold, the stakes are particularly relevant. India is home to a rapidly growing AI ecosystem, with billions of rupees being invested in AI infrastructure, startups, and government initiatives. The systems being built by American labs like Anthropic and OpenAI will shape global AI norms, supply chains, and safety standards. If those systems are developed recklessly, the consequences will not be confined to Silicon Valley.
What “Superhuman Systems” Actually Means
It is worth unpacking the term Coxon used: “superhuman systems.” In the context of AI research, this refers to AI that surpasses human performance not just in narrow tasks — like playing chess or recognizing images — but across a broad range of cognitive tasks, including reasoning, planning, scientific research, and potentially self-improvement.
The concern is not that an AI will spontaneously “decide” to harm humans in the way a science fiction villain might. The concern is more subtle and, arguably, more frightening. A sufficiently powerful AI system optimizing for a goal — even a goal that sounds benign — could pursue that goal in ways that are catastrophic for humans if the goal is even slightly misspecified. This is the “alignment problem,” and it remains unsolved.
When Coxon says the companies are “gambling with our lives,” he is pointing to the fact that these systems are being deployed and scaled up before researchers have solved the alignment problem. The gamble is that the systems will remain aligned with human values even as they become more powerful — a bet that the safety researchers themselves are not confident about winning.
Industry Positioning vs. Internal Reality
One of the most striking aspects of this story is the gap between how these companies present themselves publicly and what their own researchers apparently believe internally. Anthropic’s entire brand is built around safety. Its website, its fundraising materials, and its public statements consistently emphasize its commitment to responsible AI development.
Yet here is a researcher who trained AI systems at Anthropic saying the company is racing carelessly toward superhuman AI. And here is a senior safety researcher at the same company saying the probability of human extinction from AI is greater than 10 percent. These are not compatible with the image of a company that has safety under control.
This does not mean Anthropic is uniquely bad — Coxon also named OpenAI in his criticism. It suggests that the entire frontier AI industry may be operating in a mode where the competitive pressure to advance capabilities is outrunning the ability of even well-intentioned safety researchers to keep things under control.
What Should Follow From This
The events described in The Verge’s reporting raise urgent questions that go well beyond any single company or resignation.
- Regulatory bodies in the US, EU, and India need to move faster on AI governance frameworks that address existential risk, not just near-term harms like bias or misinformation.
- Independent safety audits — conducted by researchers who do not have a financial stake in the outcome — should be mandatory for labs developing frontier AI systems.
- Public transparency about internal risk assessments should be required, not left to the conscience of departing employees.
- International coordination is essential, because AI risk does not respect national borders.
The fact that this warning is coming from inside Anthropic — a company that was founded specifically because its founders were worried about AI safety — is a signal that should not be dismissed. When the people building the most powerful AI systems in the world tell you there is a greater than one-in-ten chance those systems could kill everyone, the appropriate response is not to assume they are being dramatic.
The appropriate response is to take it seriously, and to demand that the institutions shaping this technology do the same.
