The Internet Is Being Rebuilt for Machines

The cloud infrastructure that has long been designed around human internet usage is being revamped to accommodate the growing presence of AI agents. For years, cloud systems have been built to handle the steady and predictable traffic generated by humans searching, clicking, scrolling, and streaming. However, AI agents behave differently.

They can suddenly unleash a swell of activity, spinning up multiple sub-agents that query hundreds of databases, search documents, and call APIs in seconds, only to disappear as quickly as they arrived. Amazon is at the forefront of this shift, redesigning a core piece of its cloud infrastructure to better handle the unpredictable traffic patterns of AI agents. On Thursday, AWS launched its next-generation OpenSearch Serverless, a fully managed search and vector database designed specifically for agentic workloads.

This new system can instantly scale up when agents trigger tasks and scale back down to zero when idle, allowing customers to pay only for the resources they use. The launch reflects a growing realization across the tech industry: infrastructure originally designed for a human-driven internet doesn’t work as well in a world increasingly populated by agents. While AI agents still represent a relatively small portion of internet activity, machine-generated traffic is already significant and poised to grow.

Cloudflare says bots accounted for 31% of overall HTTP traffic over the last six months, with AI crawlers, search engines, and assistants making up roughly a quarter of all bot requests during that period. According to Lai Yi Ohlsen, senior product manager at Cloudflare, "non-human traffic will exceed human traffic sometime in the first half of 2027." As AI agents become more prevalent, cloud providers and infrastructure companies are reckoning with how to adapt systems built for humans to a world of agents that are constantly and autonomously retrieving information, invoking tools, and generating machine-to-machine traffic. Enterprises are increasingly deploying agents internally and for their customers, creating new kinds of machine-generated traffic behind the scenes.

Tia White, general manager for Amazon OpenSearch Service, noted that "the timing is straightforward. Agents are moving from experimentation into production, and they create traffic patterns that previous infrastructure simply wasn’t designed for." The key technical change with this new generation of OpenSearch Serverless is that it decouples compute from storage, allowing compute to scale up in seconds to accommodate agent traffic bursts and to scale down to zero, so customers pay $0 when agents are idle. This is a significant improvement over previous versions, where storage and compute were coupled, requiring at least one instance to be operational and running, even when not in use.