Anthropic announced new AI safeguards that cut reasoning token theft from its Claude models by 90% to 99%. The measures block proxy networks and detection classifiers, leading to bans on tens of thousands of fake accounts. Anthropic named Chinese firms Zhipu and Moonshot as key actors extracting capabilities for models like GLM-5.3, warning that illicit distillation poses severe national security and proliferation risks amid US-China competition.
Anthropic's new distillation safeguards
- ▪Anthropic made it harder to use proxy networks to probe Claude AI models by restricting access to internal reasoning steps, running an extraction detection classifier, and blocking automated queries
- ▪Anthropic implemented new safeguards restricting access to Claude's reasoning that reduced reasoning tokens extracted from Claude AI models by 90% to 99%, or 10 to 100 times lower than previous levels
- ▪Anthropic stated its extraction safety system operates with a false positive rate of less than 0.1% and possesses robustness comparable to protections preventing Claude from assisting with bioweapons development
Account enforcement and user impact
- ▪Anthropic acknowledged that its new automated distillation safeguards for Claude caught some legitimate users during account enforcement
- ▪Anthropic banned tens of thousands of accounts linked to distillation campaigns attempting to extract capabilities from Claude AI models
Accusations against Chinese AI startups
- ▪Anthropic cited Chinese AI startup Zhipu, also known as Z.ai, for attempting to siphon capabilities from Claude to build its GLM-5.3 model
- ▪Anthropic identified Chinese AI company Moonshot as a participant in distillation campaigns that secretly routed queries through Claude AI models
How model distillation attacks operate
- ▪Model distillation allows competing companies to reproduce advanced AI capabilities faster and at lower cost by copying an existing model's output and reasoning steps
- ▪Distillation attackers deploy networks consisting of hundreds of thousands of fake accounts to systematically copy internal chain-of-thought steps from targeted AI models
National security and industry risks
- ▪Leading US AI developers including Anthropic, OpenAI, and Google face increasing model distillation attempts driven by intensifying US-China technology competition
- ▪Anthropic called on rival AI developers to adopt defensive protections against model distillation attacks
- ▪Anthropic warned that unauthorized distillation of advanced AI capabilities creates national security risks, including potential deployment in foreign military and surveillance systems
Debatable claims
- ▪Restricting public access to AI reasoning steps does more harm than good
- ▪Anthropic's account enforcement goes too far in restricting legitimate user access
- ▪Model distillation is an acceptable practice for training AI models
Story comments
Loading comments…