Key Insights
Across four evaluation runs, a Claude model that couldn’t reach its fictional target found a real company with a similar name, treated it as in scope, and accessed a production database holding several hundred rows. Anthropic disclosed the incident in July, and the OWASP GenAI Security Project summarized it in its Q3 2026 exploit roundup.
The model was pursuing its assigned task. It treated reachability as permission, even after seeing signs the system was real.
It went wrong partway through, when the task couldn’t be done as written, and the model filled the gap with the nearest thing it could reach. Anyone who has managed an eager intern knows that moment. Asked to call a client whose number they can’t find, a good intern asks. An eager one calls someone with a similar name. Now multiply the intern by thousands and remove the manager.
Enterprise agents aren’t usually pointed at targets, but they get stuck all the time: a file that isn’t where it should be, an API that times out, a request that doesn’t quite match the data. What an agent does next is a governance question before it’s a detection question, and the survey data suggests most organizations haven’t answered it.
How Often AI Agents Exceed Their Intended Scope
It’s already common. A Cloud Security Alliance survey of 445 organizations, found 53% had agents exceed their intended permissions and 47% had an agent-related security incident. Only 16% were highly confident they could detect agent-specific threats.
The standards bodies have noticed. The OWASP Top 10 for LLM Applications 2026, released in September, ranks excessive agency third.
Why Access Controls Can’t Define an AI Agent’s Scope
Identity and access management was designed to answer one question: can this identity do this thing? An access policy has no way to tell a fictional target from a real company with a similar name. If both are reachable, both pass.
For an agent, the more important boundary is scope, meaning what it’s supposed to be working on. Scope is usually set in a prompt nobody reviewed, and the agent decides what that prompt means. When the literal task is blocked, a goal-seeking system looks for the closest version of the task it can still finish. That’s what makes agents useful. It’s also why least privilege isn’t enough on its own, because you can’t trim permissions to fit a scope nobody wrote down. That’s why non-human identities need governance of their own.
The Governance Agents Are Missing
People come with built-in governance. A person has a manager, a performance review, a job description, and a badge that stops working when HR processes their exit.
An agent usually has none of that. It runs under a borrowed service account or a developer’s personal token. Nobody reviews its work. And it outlives the project that created it, because nothing tells anyone to turn it off.
Survey data backs up that last gap. A separate Cloud Security Alliance survey of 418 organizations, sponsored by Token Security, found 82% had discovered previously unknown agents in the past year. Only 21% had a formal process for decommissioning one. Nearly every company has an exit process for employees. Four in five have none for their agents.
Write the Job Description Before the Agent Starts
For most agents, the job description is the only place anyone writes down its scope. Three lines cover most of it.
- What it’s for, in one sentence, along with the systems it should touch to do that job.
- What’s off-limits: the systems, data, and actions it should never touch, even when it can. An intern picks up the unwritten rules by watching. An agent doesn’t.
- What to do when it’s stuck. When the task can’t be done as written, the agent should stop and report to its owner instead of substituting the nearest match. This line is easy to leave out, and it’s the one the Anthropic model needed.
The job description is also what detection measures drift against, which we’ll take up in the next post.
The Rest of the Org Chart: An Owner, an End Date, and Limits
Every company has an org chart for its people: who reports to whom, who approves what, and who’s accountable when something goes wrong. Almost no company has one for its agents. Either security builds it deliberately, or the first serious incident builds it for you, in a post-mortem with lawyers in the room. Someone will be asked who owned the agent, who approved its scope, and why it was still running. “Nobody” won’t survive that meeting.
Every agent in production, first-party and third-party, needs a named human owner who is accountable for it. Agents without owners get retired or reassigned.
Every agent also needs a condition that retires it, such as the project shipping, the owner leaving, or thirty days without use. This is the badge deactivation agents never got.
Owners and security should decide ahead of time when an agent gets paused or has its access cut. People set the policy. They can’t approve every action an agent takes.
Three Questions to Ask About Your AI Agents
Three questions are a good place to start. First, can you list every agent in production and the human who owns it? Second, for each one, can you name something it must never do, even though it can? Third, what does each agent do when it can’t finish its task? If the answer is “whatever it decides,” that agent is your first risk register entry.
None of the three needs a new product to start. Make an owner, a written scope, and an end date requirements before any new agent reaches production.
We track how AI agents are changing enterprise risk and how teams are governing them. Our AI security resource hub collects our latest research, guides, and analysis in one place.
Read the Latest AI Security Resources