Legacy Infrastructure Slows AI Agents, Not Models

Legacy infrastructure, not AI models, is the main obstacle to deploying agents at scale. That's according to infrastructure leaders from LinkedIn, Walmart, and Zendesk, who shared their experiences at VB Transform 2026. Animesh Singh, senior director of AI platform and infrastructure at LinkedIn, Desiree Gosby, SVP of corporate technology services and technology strategy at Walmart, and Sami Ghoche, VP of applied AI at Zendesk, each described the bottlenecks they encountered when moving agents from pilot to production.

The common conclusion among the three leaders was that enterprise infrastructure was designed for human workflows, not agent workflows. The gap between these two speeds is where the real engineering challenge lies. Gosby put it plainly: the goal is to ensure that engineering doesn't become a bottleneck for AI adoption.

Each company faced a different version of the same challenge. At LinkedIn, the bottleneck was Kubernetes, which assumes containers spin up on demand, a process that takes seconds. Singh said that's too slow for agents.

The fix was moving to pre-provisioned pools of containers that swap agentic workloads in and out in real-time. Another problem arose when LinkedIn let agents control their own orchestration, which led to hallucination issues. Singh said the issue was structural and required building a custom harness and control flow.

Walmart's bottleneck came from success. An agent harness went viral internally, and employees began building their own agents, leading to duplication and overlap. The fix was building governance to spot duplication and promote the best version of an agent.

Zendesk's bottleneck was on the data side, with 20 billion customer conversations in its repository. Ghoche said that handing this history to a large language model doesn't work and that investing in underlying data pipelines is necessary. The leaders agreed on the importance of open-source models, suggesting that enterprises should own their models and infrastructure where possible and lean on frontier labs only where they have a clear edge.

They also shared advice for modernizing infrastructure, including investing in evaluations, owning the agent harness from day one, and building for model and context independence. The role of open source is crucial here. Ghoche said most enterprises would prefer to own their models and infrastructure, and that drives Zendesk's approach.

The exception is frontier reasoning work, where labs still lead, though that slice of use cases is shrinking. LinkedIn built two subsystems for independence: an AI gateway and a memory subsystem. The AI gateway provides a single interface for outbound calls to models, while the memory subsystem holds context independent of any model provider.