Amazon AGI director: AI agent reliability, not capability, blocks enterprise deployment

The enterprise AI industry faces a significant challenge in deploying AI agents, with only 5% of enterprises successfully shipping them to production despite 85% piloting them. At VB Transform 2026, Bryan Silverthorn, Director of AGI Autonomy at Amazon, explained that the gap persists due to reliability issues, not capability. Silverthorn, who joined Amazon through its acquisition of Adept AI, argued that reliability must be broken into four distinct dimensions: consistency, robustness, predictability, and safety.

Silverthorn's framework, based on Princeton research, aims to unpack different factors tangled together in evaluations. He noted that AI agents often pass internal evaluations but fail in real-world scenarios. For instance, a customer deployed an agent for software QA that worked flawlessly for two months before intermittently reading wrong numbers due to a software change that triggered a failure.

The lesson, Silverthorn said, is about measurement, not just models. The models have to be better. Obviously, we're working hard on making the models better, he said.

But the deeper takeaway is that teams need to identify their dimensions of variability and match measurement rigor to the stakes of the application. VentureBeat's research supports this point, finding that half of surveyed companies shipped agents that passed internal evals but failed real customers, and enterprises overwhelmingly track uptime while ignoring accuracy. Silverthorn's most memorable prescription was cultural, not technical.

Inside Amazon's AGI lab, researchers call their agents 'interns,' managing them like employees. The joke carries a serious operational philosophy. Agents, like interns, are powerful but occasionally clueless, capable of amazing work and spectacular derailment.

Managing them requires management skills rather than software skills: asking what could go wrong, adding backups and undo capabilities, and consciously deciding what risk you can accept. For enterprises stuck in pilot purgatory, the path forward starts with a mindset shift: stop asking whether your agent can do something impressive once, and start asking whether it can do it correctly a thousand times in a row. The enterprises that escape the 85% ceiling won't be the ones with the smartest agents.

They'll be the ones with the best managers. Why this matters: The challenge of deploying reliable AI agents has significant implications for the enterprise AI industry. As companies increasingly look to AI to drive business value, the ability to manage and deploy reliable agents will become a key differentiator.