Identity and permissions aren’t enough to govern AI agent behavior

Presented by Box Identity and permissions are no longer enough to secure enterprise AI agents. They govern what an agent can reach, not how it behaves once it starts working on its own, and an autonomous agent can turn legitimate access of enterprise data into unintended action in seconds. That gap is pushing enterprise AI security from just governing access toward a layered approach that includes governing execution, says Heather Ceylan, chief information security officer at Box. "Access controls and permissions are the foundation, but the challenge is they were designed for humans," Ceylan says. "Permissions are still the foundation, but you have to think about how the agents get their permissions scoped as well." Access controls were built for a slower, more forgiving world, where a human employee with lingering permissions to a decade-old folder will rarely go looking for any data inside it, or even remember they have access at all. But an agent will explore all of its permissions, operating at a scale no person can match and surfacing forgotten misconfigurations and stale permissions far faster than a human ever could. Identity and scoped permissions remain the first layer of any credible defense, however, because agents raise the stakes, clean access hygiene is more valuable than ever. A steady drip of incidents in recent months underlines the potential for major damage, where models have slipped the sandboxes they were supposed to stay contained within , reached systems they weren't scoped to touch, or read content they were not permitted to access. While the specifics vary, the overarching threat remains the same: an agent found a path through the data that it could reach, and took it. Access controls alone can't secure autonomous AI agents Complications crop up when agents are handed broad standing permissions to match broad workflows. An agent might legitimately need to call fifty tools across twenty different actions over the life of a task, reading and writing folders that span every department in the company. But granting all of that access at once inflates the blast radius of possible damage to data and content if a single step goes wrong. A stronger permissions model provides access only in the moments that it's needed, Ceylan says. "You need permissions that change based on what the agent has been asked to do, when it needs to take that action," she explains. "If it's taking one step and only needs two tools, it should be scoped to only those two. When you narrow permissions to the task in front of the agent, the number of ways any given step can misfire shrinks with it." From governing access to governing execution Tight security is also no longer tied to whether an agent has access to specific data, but whether it should take a specific action on a specific step, which is the difference between a standing access grant and a bounded one. In other words, an agent may hold legitimate rights to a finance folder but still have no business writing four thousand of those files to a new location, even if it's told to do so. "An employee with access to payroll data they were never meant to keep could be instructed to pull the payroll records and write them to a public shared folder, publishing the entire company's compensation in a single move," she says. "Every access check passed, but the behavior still has catastrophic consequences." Prompts alone can’t reliably govern agent behavior because instructions can change, agents can absorb injected instructions, or they can be steered by files they read along the way. Durable controls need to live one level down, with the tool calls themselves, as well as the content those calls act on. Deciding in advance what an agent is permitted to execute, regardless of how its prompt is manipulated, keeps behavior inside fixed bounds that no clever input can talk it out of. Legacy content platforms weren't built for AI agents Most of what an agent touches inside a company is unstructured content, including contract, policies, customer records — the accumulated files that make up the overwhelming majority of corporate data. Much of it still sits in systems that were designed for human filing habits: network drives, aging ECM platforms, and many SaaS tools. Those systems were built to ask the question, does this person have permission? and they answer it with folder-level access that hasn't been audited for years. The legacy platforms don't have metadata for an agent to reason over or classification for an enforcement layer to act on, and their logs aren't detailed enough to show what an agent read. Bolting an AI connector onto that stack doesn't fix any of it, it just hands agents the same blind spots, only at machine speeds, Ceylan says. "Every agent action eventually resolves to content," Ceylan says. "If the content layer can't tell you what it's holding, who it belongs to, and what should never leave it, there's nothing underneath your controls." How to decide which AI actions need human approval Two years ago, everyone assumed security would always mean including humans in the loop. Building and living with agents has changed that assumption, Ceylan says, and Box now sorts actions into three tiers: fully autonomous actions, monitored actions and high-risk actions that require human approval. Fully autonomous , reserved for actions that are reversible, bounded, logged, and free of untrusted input, and where a mistake doesn't cost too much. Monitored actions , once a team has built enough confidence in an agent and paired it with alerting and rollback that can catch and undo a problem in flight. Irreversible, high-risk actions that are always routed through a person — for instance, when an agent that wants to delete a large number of files or wipe the primary folder in a structure. Each team must calibrate the lines between these layers to match its own risk tolerance. Box’s approach is to put controls in the platform rather than the workflow, enforcing protections such as data classification, labeling and expiration without requiring a human checkpoint for every action. "The right configuration should be enforced at the outset, instead of blocking an action at the end," Ceylan says. Building trust in AI agents through behavior over time An organization can make an access decision once, but trusting an agent's behavior should only happen over time, observing how the agent runs, collaborates, and builds on the outputs of other agents. Published frameworks tend to lag the technology, or are too abstract to implement; instead Box works from a smaller set of principles: tightly scoped agent identities and actions, clear expectations about what can be rolled back, three approval tiers, and a way for teams to test and iterate quickly. "The sanctioned path has to be the fast path, because when teams aren't given a safe way to experiment, they tend to route around the controls entirely," Ceylan says. "A security leader's job is to offer a way to move quickly without stepping outside the guardrails." Sustaining trust in an agent depends on being able to see how it actually behaves. But traditional monitoring tools weren’t built for the job; the behavioral baselines that underpin user and entity behavior analytics were calibrated to human activity, and suspicious agent behavior doesn’t necessarily look like human activity. And while establishing a useful agent baseline begins with logging, many agents start out as experiments, so their actions never make it into logging infrastructure. Plus, important signals are rarely individual easily flagged actions, but chains of activities across systems, as one agent's output becomes another's input, while the detections for that pattern are still being designed. Governing agents , then, requires visibility not just into what they can access, but what they actually do, and that visibility has to live where the content does. Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com .