Introduction

AI agents are becoming part of daily life, but a new Visa study reveals they're still far from reliable shopping assistants. When tasked with purchasing everyday items, most agents missed the mark more than 40% of the time.

What Happened

Visa instructed 22 different AI agents to purchase three everyday items—a television, a laptop, and a hair appliance. Across 15 shopping expeditions per agent, the overall success rate hovered around 57%. Top performers like Google's Gemini Aragon 4 and Meta's Wonder model succeeded 82% of the time, while Claude scored 76% and OpenAI's agent managed 64%. A Visa diagram plotted success rates against the cost to run each inquiry, showing that the five most expensive models succeeded on average 67% of the time, while the cheapest five scored only 50%. The study highlights how AI agents often struggle to gather complete shopper preferences, sometimes buying the wrong product or failing to suggest suitable alternatives.

Why This Matters

The study underscores a core challenge: AI agents frequently lack complete information about what shoppers actually want. In the TV test case, agents had to navigate attributes like gaming console compatibility, a 30-day return window, and Dolby Vision support. Many agents failed to elicit missing details or recommended unsuitable products. Beyond the test results, the findings raise important questions about trust, liability, and how AI-driven commerce will integrate with existing payment systems. Visa's Rubail Birwadker notes the technology is still in early days but advancing rapidly, and that product discovery remains a key element of this new type of commerce retailers and AI companies are working to figure out.

Key Takeaways

  • AI shopping agents succeed less than 60% of the time in controlled tests
  • Top-tier models from Google and Meta lead the pack, but even they miss the mark nearly one in five times
  • Cost correlates with performance, but price isn't a guarantee
  • Product discovery helps agents understand shopper needs and remains a major hurdle
  • As AI commerce evolves, questions around liability, payment security, and consumer trust will shape mainstream adoption

Conclusion

The Visa study serves as a reality check for the hype around AI-powered shopping. While the technology shows promise, current agents struggle with the complexity of real-world purchasing decisions. For consumers and retailers alike, the path to reliable AI shopping assistants depends on better product discovery, smoother payment integration, and clearer accountability frameworks. As the technology matures, delegating routine purchases to AI may become viable—but for now, human oversight remains essential.