Anthropic conducted an experimental classified marketplace called Project Deal where 69 employees were each given $100 budgets via gift cards and represented by AI agents using Claude models to negotiate real transactions. The experiment generated 186 deals totaling over $4,000 across four separate marketplaces testing different model configurations. Users represented by more advanced AI models achieved objectively better outcomes, yet participants did not appear to notice these disparities in results. The experiment raises significant questions about agent quality gaps in autonomous commerce, where consumers using weaker AI negotiators could be systematically disadvantaged without awareness of their inferior position.
Anthropic's Project Deal marketplace experiment
- ▪Anthropic ran four separate marketplaces in Project Deal with different models
- ▪Anthropic's Project Deal involved AI agents striking real deals for real goods and real money
- ▪Anthropic's Project Deal experiment involved 69 Anthropic employees who were given a budget of $100 paid out via gift cards
- ▪Anthropic's Project Deal marketplace resulted in 186 deals totaling more than $4,000 in value
- ▪Anthropic created a classified marketplace called Project Deal where AI agents represented both buyers and sellers
- ▪One of Anthropic's Project Deal marketplaces was real where everyone was represented by Anthropic's most-advanced model and deals were actually honored after the experiment
Performance disparities between AI agent models
- ▪Users in Anthropic's Project Deal did not seem to notice the disparity in outcomes between different AI agent models
- ▪Users represented by more advanced AI agent models in Anthropic's Project Deal got objectively better outcomes
- ▪Initial instructions given to AI agents in Anthropic's Project Deal did not appear to affect sale likelihood or negotiated prices
Implications for autonomous AI commerce
- ▪Anthropic's Project Deal raised the possibility of agent quality gaps where people on the losing end might not realize they are worse off
Perspective of Consumer protection advocates
- ▪Anthropic's Project Deal experiment demonstrates that consumers using inferior AI agents could be systematically disadvantaged in automated negotiations without realizing it
- ▪The Project Deal results suggest a need for transparency requirements when AI agents negotiate on behalf of consumers in commercial transactions
Perspective of AI safety researchers
- ▪The Project Deal finding that users did not notice outcome disparities raises concerns about human oversight of AI agent performance in real-world transactions
- ▪Anthropic's Project Deal experiment illustrates how AI agent capability gaps could create information asymmetries in autonomous commerce systems
Perspective of Anthropic
- ▪Anthropic's Project Deal research aimed to understand whether AI agent model quality creates measurable differences in commercial outcomes
- ▪Anthropic designed Project Deal to test how AI agents would perform in authentic marketplace negotiations with real financial stakes
Story comments
Loading comments…