AI Agents Need Oversight Like Human Interns to Avoid Security Risks

AI agents are evolving from simple chatbots to full-fledged digital workers authorized to take action on applications and data. And with those capabilities come a raft of security and governance concerns. Treat your AI agents as eager but misguided interns, requiring the same oversight and guidance as human interns, suggested experts in a panel held at the recent Snowflake Summit in San Francisco.

AI agents require specific instructions and careful monitoring by human managers. An agent without restraints can be extremely problematic, the panelists, representing AI security providers, agreed. "You may tell the agent to buy you shoes, and before you know it, it has bought you a car," said Mayank Agarwal, founder and CTO of Resolve AI.

"You have to think very hard about what permissions you're giving the agent. You can't just expect an agent to stay on the straight and narrow. You have to put these ironclad constraints around it to limit what it's able to do." Along with restraint, context and intent are the key watchwords for spinning up and managing agents.

"It's not just enough to know what this agent was created to do. You also have to know things like whose authority it is acting under and what it's going to do, for example, with data it's accessing," said Nancy Wang, chief technology officer for 1Password. Professionals should throw out the old software development rulebook, as building and deploying agents today is very different from software practices of the recent past, Agarwal pointed out.

"If you go back just two years, an engineer knew exactly how they were going to connect APIs across different systems," he said. "The whole thing was very predictable: A is going to call API B, B is going to do this with that data, and call C, and do this with that data. In the agentic world, it's completely unpredictable.

The agent wires the stuff on the fly. Give it a goal, solve this problem, and it goes out and tries all the paths that it has access to." This approach can lead to new types of issues for which professionals and managers are not prepared. The agent is "talking to tools which are capable of doing things on your behalf, so you don't know if these tools are exfiltrating data," Agarwal said.

"The agent may read from a tool and use another tool to write it to someplace it shouldn't be." This concern raises the specter of shadow AI, operating out of view. "We had a client that had 12 OpenClaw instances within their framework, with access to API feeds, source code, and a contractor using Telegram to communicate," said Jason Merrick, senior vice president of product at Tenable. "What could go wrong, right?" As a result of these issues, understanding what agents do behind the scenes can be a challenge.