Anthropic Tests AI-Powered Marketplace with Real-World Transactions

In a bold experiment, Anthropic created a functional classified marketplace where AI agents acted as both buyers and sellers, facilitating real transactions for real goods and real money. The test, dubbed Project Deal, involved 69 Anthropic employees who were given a $100 budget, paid out via gift cards, to purchase items from their coworkers. The company's researchers were pleased with the outcome, noting that the platform yielded 186 deals worth over $4,000.

Anthropic ran four separate marketplaces with varying models, including one that was fully functional, where all participants were represented by the company's most advanced AI model, and deals were honored after the experiment. The results suggested that agents represented by more advanced models achieved objectively better outcomes. However, users did not seem to perceive the difference, which raises concerns about potential 'agent quality' gaps.

In these gaps, individuals on the losing end of transactions may not realize they are worse off due to the performance disparity between their agent and those of their counterparts. Interestingly, the initial instructions provided to the agents did not appear to influence the likelihood of sale or the negotiated prices. Anthropic described the experiment as a 'pilot test with a self-selected participant pool,' yet the company was 'struck by how well Project Deal worked.' The findings highlight the potential of AI-powered marketplaces and the importance of understanding the impact of agent performance on user outcomes.