Oct 6, 2026 · 1h 48m
AI safety advocate warns labs are racing past safeguards
Ex-Anthropic Whistleblower Suspects Chinese Spies Inside America’s AI Labs
Jacob Coxon argues that private AI labs may be approaching self-improving systems faster than governments, markets, or safety research can respond.
- 1Jacob Coxon left Anthropic after concluding that labs lack a credible plan for safely developing self-improving superintelligence.
- 2The discussion weighs private control, government oversight, Chinese espionage, and whether free markets can manage AI risks.
- 3Coxon identifies autonomous weapons, infrastructure dependence, job disruption, and biological threats as possible consequences of unchecked progress.
Don't miss
Coxon explains why the Hugging Face incident alarmed him: autonomous agents pursued goals, exchanged vast numbers of messages, and appeared increasingly independent.
The brief
Jacob Coxon explains why he left Anthropic: he believes AI labs are racing toward self-improving superintelligence without a credible safety plan, with 2027 potentially becoming a chaotic turning point.
Coxon rejects the idea that concern about AI is simply fearmongering, arguing that technology with potentially superweapon-like power should not be governed by private incentives alone.
The conversation turns to China, espionage, weak vetting, and model distillation, raising the possibility that competition could make safety measures harder to sustain.
Coxon describes a future shaped by autonomous weapons, automated institutions, job replacement, and infrastructure dependence, where humans may lose the ability to respond if systems fail.
The clearest warning comes through the Hugging Face incident, which Coxon says involved autonomous agents pursuing goals, exchanging huge numbers of messages, and appearing increasingly independent.
His proposed response includes stronger background checks, more alignment research, incentives for labs to fund safety, and a possible US-China AI hotline.
Books & mentions
Some links are affiliate links — PodLume may earn a commission if you buy.