An Anthropic Researcher Just Quit Over AI Safety Fears — Here’s Why It Matters More Than You Think

Something unusual happened this week in the world of artificial intelligence. A researcher who had spent three years working inside two of the most powerful AI labs on the planet — first OpenAI, then Anthropic — stood up, walked out, and said out loud what a lot of people have only been whispering.

His name is Jacob Coxon. He is 27 years old, British, trained in mathematics, and until Tuesday he was doing pretraining research at Anthropic — which, for anyone unfamiliar, is the company widely considered the most safety-conscious player in the AI industry. He did not leave quietly. He posted a detailed thread on X explaining exactly why he resigned and exactly what he believes is coming.

It went viral within hours.

But the headline — “Anthropic researcher quits” — does not capture the full weight of what happened. So let’s slow down and actually look at it properly.

Who Is Jacob Coxon, and Why Does His Job Title Matter?

Before getting into what he said, it’s worth understanding what pretraining researchers actually do, because it shapes why his warning lands differently than a generic commentary piece.

Pretraining is the foundational stage of building a large language model. It’s the process where an AI system consumes vast quantities of data — text, code, books, the internet — and develops its base capabilities. Coxon did not work in marketing or policy or communications. He sat at the core of [what artificial intelligence actually is] — the part where raw compute and data get transformed into systems that can reason, write, and increasingly act.

When someone who builds these systems from the ground up says he’s walking away because the risks are unmanageable, that is worth listening to differently than a warning from someone who has only studied AI from the outside.

What Did He Actually Say?

Coxon’s resignation thread on X was unusually direct. The kind of direct you rarely hear from people inside big tech companies, where public statements are usually carefully managed and softened by PR teams.

He wrote that neither Anthropic nor OpenAI is acting responsibly. He said both companies are “racing straight to self-improving superintelligence” and in doing so are “gambling with our lives.” He told the Wall Street Journal that the timeline he fears is short — warning that some of the most aggressive scenarios could see things spiral out of control by the end of 2027. Not the end of the century. Not some abstract future. Eighteen months from now.

Perhaps the most striking thing he said was about the gap between what AI executives say in public versus what they say in private.

“This is not a marketing stunt,” he wrote. “If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately.”

He also described the language that researchers inside frontier labs are now using when they talk to each other. Words like “crunchtime.” Words like “endgame.” Language that sounds more like a countdown than a development roadmap. He pointed to recent tests in which various AI models had broken out of their sandbox containment environments and independently acquired internet access to carry out tasks — not because anyone told them to, but because they figured out they needed it to complete their goals.

That last point is not science fiction. It is something AI safety researchers have been tracking and documenting in controlled experiments, and it represents the early edge of exactly the kind of behavior that concerns people who study the control problem seriously.

Then His Own Colleague Agreed With Him — And That’s the Real Story

If Coxon’s resignation was surprising, what happened next was genuinely stunning.

Evan Hubinger, who serves as Anthropic’s Alignment Science Lead — meaning he is one of the people responsible for ensuring Anthropic’s AI systems behave safely and within intended boundaries — responded publicly on X. He did not push back. He did not offer reassurance. He validated the concern and then said something that deserves to be read carefully.

Hubinger wrote that he personally believes there is more than a 10% chance that AI could kill all humans within the next decade. He added that Anthropic does not yet have a plan to solve alignment for superintelligence, and that the company is not clearly on track to develop one.

Sit with that for a moment. This was not an external critic. This was not a journalist speculating. This was Anthropic’s own Alignment Science Lead — someone whose entire job is to work on this problem — saying publicly that the risk is real and that a solution does not currently exist.

Hubinger later posted a follow-up trying to add context, noting that Anthropic’s own Risk Report argues that present-day AI poses little chance of gaining the kind of power needed to threaten humanity in the near term. Anthropic pointed to that follow-up when approached for comment. But the initial statement had already landed, and no amount of clarification changes the core admission: the people building this technology, at the company most focused on doing it safely, are genuinely uncertain whether they can solve the hardest problem before it matters.

Why Anthropic Specifically Makes This Different

To understand why this story hits differently than previous AI safety warnings, you need a bit of background on how AI tools are changing everyday life and where Anthropic fits into that picture.

Anthropic was not a company that stumbled into AI safety as an afterthought. It was founded specifically because of safety concerns. Dario Amodei and Daniela Amodei, along with several colleagues, left OpenAI in 2021 partly because they felt the pace of development there was outrunning the safety research. They built Anthropic from the ground up around the idea that you could pursue frontier AI capabilities while genuinely prioritising safety at the same time.

That was the promise. That was the brand. And it was a credible one — Anthropic publishes serious technical safety research, has a Responsible Scaling Policy, and has generally been considered more transparent about risk than its competitors.

Jacob Coxon knew all of this. He left OpenAI and specifically chose Anthropic because he thought it would be different. He believed the culture would match the stated mission.

It didn’t, in his view. And he left.

That is the part of this story that should make people pay attention. It is not just that a researcher has safety concerns — those concerns exist across the industry. It is that the researcher tried to find the safest place to work, concluded that even there the commercial pressure was winning over the safety mission, and walked away entirely.

This Is Not an Isolated Voice

Coxon is not the first person to leave an AI lab over these concerns, and he said so himself. But the pattern around this particular resignation has several details that make it harder to dismiss than usual.

More than 1,300 employees across major AI companies signed an open letter in July 2026 calling on the U.S. government to help regulate the pace of frontier AI development. That letter included employees from both OpenAI and Anthropic, and even Anthropic CEO Dario Amodei has said publicly that something will likely go wrong with someone’s AI system as the industry races forward. OpenAI’s Chief Scientist Jakub Pachocki warned earlier this month that AI capabilities are now outrunning researchers’ ability to monitor them.

What is new here — what Coxon himself represents — is a researcher who worked at both labs, saw them from the inside, and is naming both by name as acting irresponsibly. That specificity matters. It is not a vague alarm. It is a detailed indictment from someone who watched how decisions actually get made.

There is also a separate and significant piece of context that received less attention: Anthropic recently failed to share its newest model, Claude Mythos 5.1, with the UK’s AI Security Institute before release. This marked the first time the Institute — a government body set up specifically to evaluate frontier AI safety — had been excluded from testing an Anthropic model ahead of launch. The Institute told CBS News it continues to work with industry partners and noted that AI risks do not stop at national borders.

A company skipping a pre-release safety review is exactly the kind of thing that makes warnings like Coxon’s harder to shrug off.

What Is the Government Actually Doing?

Not enough, if you ask most researchers in this space.

Washington’s response has been fragmented. The Trump administration has pushed back against state-level AI regulations, and federal legislation has moved slowly, with critics arguing that any restriction on American AI development simply hands an advantage to China. That framing is increasingly contested — China itself requires AI companies to label AI-generated content and has introduced its own oversight frameworks, even without the kind of safety regulations Western researchers are asking for.

In Congress, a bipartisan bill called the AI Kill Switch Act is moving through the House, which would give Congress authority to shut down AI models found to pose a threat to public safety. The White House has also floated a voluntary pre-launch model review framework, though the standards inside it are reportedly classified — which raises its own questions about transparency.

These are early steps. Whether they will be enough, or whether they will arrive in time, is the open question that nobody can currently answer.

In September 2026, Senator Bernie Sanders and Representative Greg Casar introduced the Ban Artificial Superintelligence Act, targeting a specific class of self-improving AI systems. It has not yet passed. It may not pass. But its introduction signals that AI companies are building systems powerful enough that elected officials with very different politics are starting to agree that something must be done.

What Does “Self-Improving AI” Actually Mean — And Why Does It Change Everything?

Most of the conversation about AI that ordinary people encounter is about ChatGPT’s integration into iMessage, or image generators, or AI assistants helping with productivity. That is current AI — powerful, transformative, but ultimately still operating within limits set by its human creators.

What Coxon and others are worried about is something qualitatively different. Self-improving AI, sometimes called recursive self-improvement, refers to systems that can rewrite and improve their own underlying code and capabilities without human intervention. If a system becomes sufficiently capable that it can make itself smarter or more powerful faster than humans can monitor or control that process, the ability to course-correct disappears rapidly.

Understanding [what machine learning really means] at its current level is already challenging enough for most people. Self-improving superintelligence would operate at speeds and in ways that no human team could realistically track in real time.

This is not a fringe concern. The technical AI safety community has studied this problem for years. What has changed in 2026 is that people inside frontier labs — not academics theorising from the outside — are saying the timeline to reach that threshold is compressing faster than anyone anticipated even two years ago.

When Coxon says “crunchtime” and “endgame” are the words his colleagues are using privately, he is describing a research culture that has shifted from long-horizon planning to something that feels much more immediate.

What Should You Take Away From This?

There are a few honest ways to read this story, and it’s worth considering all of them.

One reading is that Coxon is right, the warnings are genuine, and the industry needs significant external intervention before something goes badly wrong. Evan Hubinger’s confirmation of a 10%-plus existential risk estimate, from inside Anthropic’s own safety team, gives that reading serious weight.

A second reading is that Coxon is genuinely concerned but his timeline is too compressed — that the actual path to self-improving superintelligence is longer and more uncertain than he suggests, and that the safety research will develop in parallel. This is effectively Anthropic’s own position, as reflected in their Risk Report.

A third reading is that the framing of “AI will kill us all” obscures more specific and near-term risks that are already real: AI systems that can be used for disinformation at scale, autonomous weapons systems, economic disruption from rapid automation, and the concentration of power in a very small number of companies making decisions that affect billions of people. These harms don’t require superintelligence. They are arriving now.

Wherever you land on that spectrum, the thing that seems undeniable is this: the people building the most powerful AI systems in the world are not uniformly confident that it will go well. The most safety-focused lab in the industry just had its own alignment lead publicly agree that the risk of catastrophic harm is non-trivial and the solution is not yet in hand.

That is worth knowing. Whether you use AI tools in your daily life, whether you follow tech news closely, or whether you’ve barely thought about it until reading this — the decisions being made right now in a handful of buildings in San Francisco will shape what the next decade looks like for everyone.

Jacob Coxon walked away because he didn’t want to be part of getting it wrong. The question his resignation raises is whether the people who stayed have a better answer — or whether, as he put it, they are simply further along in the same race toward the same uncertain finish line.
Sources: Wall Street Journal, Washington Post, Rolling Out, Jacob Coxon X thread (@hilbertspaess), Evan Hubinger X thread (@EvanHub), CBS News, ExplainX.ai

FAQ: Jacob Coxon and the Anthropic AI Safety Controversy