Key Insights
In April 2026, six cybersecurity agencies from five countries, including the National Security Agency (NSA) and the Cybersecurity and Infrastructure Security Agency (CISA), released joint guidance on agentic AI that calls for rigorous monitoring and human oversight. AI agent monitoring puts that advice into practice by continuously observing what an agent does across model calls, tools, memory, and handoffs. It answers questions uptime dashboards cannot, including why an agent took an action and on whose authority.
This guide follows a single agent run from its first model call to its final tool action. It also shows how telemetry records that run and which metrics separate a finished task from a correct one. It then covers threats that leave traces and how those traces drive approvals, stop controls, and audit records.
What Is AI Agent Monitoring?
AI agent monitoring continuously observes an agent's actions and outcomes in production, measured against the task it received and the policies it must follow. It checks known signals against expectations, such as error rates, token usage, or using a tool outside its permitted scope. AI observability practices go further, letting an engineer explain behavior nobody anticipated because the telemetry keeps enough context to ask new questions later.
Application performance monitoring tracks infrastructure health, and single-call model observability adds prompts, responses, and token counts for each request. Neither captures how an agent plans, chooses tools, reads and writes memory, and delegates work across many steps, often taking a different path on the next run.
Those agent behaviors move the unit of observation from a request to a run. A sequence of actions can be unsafe even when each step looks acceptable. Monitoring therefore keeps order, context, and authority together in one record.
How AI Agent Monitoring Captures an Agent Run
AI agent monitoring captures a run as a trace, with one parent record for the task and child records for every model call, tool call, retrieval, and handoff beneath it. OpenTelemetry, the open-source telemetry project, defines generative AI (GenAI) conventions for this structure, and its 2026 GenAI observability guidance describes them as in use and still evolving.
Run, Model, and Tool Spans Preserve Execution Order
Run, model, and tool spans record each step of an agent run in the order it happened. In the OpenTelemetry model, a top-level invoke_agent span represents the run. Each model request becomes a child chat span, and each tool invocation becomes an execute_tool span.
Because every child points to its parent, the trace preserves sequence, showing that the agent asked the model, the model chose a tool, the tool returned, and the model decided again. Each span carries its own duration and outcome, and model spans record token counts such as gen_ai.usage.input_tokens and gen_ai.usage.output_tokens.
Suppose a hypothetical expense-review agent slows down sharply, and three problems that look identical from the outside appear differently in its spans:
Slow Tool: One execute_tool span accounts for most of the run's duration, and the chat spans stay short.
Slow Model: The chat spans run long, often alongside high output token counts.
Retry Loop: The trace shows the same tool call repeated with near-identical arguments, each attempt failing or returning the same result.
Each pattern points to a different owner, whether the tool's maintainers, the model configuration, or the agent's planning logic.
Identity, Memory, and Delegation Records Preserve Context
Spans record what happened, and identity logs show whose authority allowed it. Each run should record who or what started the task and, as agentic identity security requires, the agent's own identity separate from the user it serves. Keeping identity and privilege changes in the same log lets reviewers spot drift, impersonation, or misconfiguration over time.
Retrieval and memory records should name what the agent read and wrote, and a handoff should link the receiving agent's trace to the sender's. Abnormal's AI Agent Security, in private preview, is designed to map agent ownership and track each agent's privileges, tool access, and data reachability.
The most useful context is also the most sensitive, and the OpenTelemetry guidance cited earlier notes that GenAI telemetry captures no prompt content or tool arguments by default, precisely because those fields can hold sensitive data. Turning content capture on records full prompts, system instructions, tool arguments, and results, exposing the data the default protects. A workable pattern captures metadata for every run and full content only for high-risk agents or active investigations, with access limited to named roles.
Which Metrics Show Whether an Agent Succeeded Safely?
An agent succeeds safely when it finishes the task with a verified-correct outcome, within policy, at acceptable latency and token usage. Completion only records that a run ended without an error, and verified correctness checks the outcome itself, such as whether a refund matched policy.
Correctness alone leaves safety and efficiency unmeasured, which these four signals cover:
Policy Adherence: Every tool call, data access, and output stays within the agent's approved scope.
Tool Failures: Errors, timeouts, and malformed arguments per tool often surface integration problems early.
Escalation Quality: The agent hands off at the right moment, with enough context for a person to act.
Latency and Token Usage: Run duration and tokens consumed track efficiency, measured as usage rather than billed cost.
Tracked separately, these signals keep one good number, such as low latency, from masking a policy or correctness failure. Evaluation owners validate model-based judges, which score outputs automatically, against human grades on a sample. They repeat the check after each model or prompt change, because these judges can score the same output differently and adversarial text in tool outputs can fool them.
Which Threats and Failures Should Monitoring Reveal?
Monitoring should reveal both deliberate attacks and ordinary failures, since each leaves evidence in tool calls, identity use, and outcomes. Among deliberate attacks, indirect prompt injection hides instructions in documents, web pages, or tool outputs the agent reads. Memory poisoning plants records that alter later sessions, and identity abuse turns valid agent credentials toward out-of-scope actions.
Every failure type leaves a different trace, and the trace determines the response:
Failure Type | Observable Evidence | Response |
|---|---|---|
Indirect prompt injection | Retrieved content followed by actions outside the task | Holding the action and treating the source as untrusted |
Tool misuse | Unexpected write or delete calls | Gating the tool and narrowing its permissions |
Memory poisoning | A memory record that introduces new approvals or policy exceptions | Quarantining and rolling back the record |
Identity or privilege abuse | Actions under valid credentials beyond the task's scope | Revoking or rotating credentials and auditing privilege changes |
Retry loop | Repeated identical tool calls, climbing tokens per run | Tripping the circuit breaker and escalating to a person |
Gaming the metric | Metric passes while verified outcomes fail | Re-checking outcomes and revising the evaluation |
Cascading error | Downstream agents consuming faulty output | Pausing the chain and tracing the first bad span |
In one hypothetical example, a procurement agent summarizing a supplier's onboarding packet retrieves a page whose hidden text tells it to email the company's vendor list to an outside address. Its trace shows a retrieval span for the packet, a chat span planning a send, and an execute_tool call to the email tool with an external recipient, under an identity whose task never required outbound email.
How Monitoring Triggers Controls and Human Oversight
Monitoring becomes control when specific signals trigger specific responses, such as requiring approval before risky actions, limiting ongoing execution, and the ability to stop and revoke an agent. In ISACA's 2026 AI Pulse Poll, only 12% of respondents said their organization has a documented, regularly tested process for shutting down or overriding AI systems.
Action Risk and Reversibility Determine Approval Gates
Approval gates belong on actions that carry high impact and are hard to reverse, so approval design starts by classifying actions on both measures. Reading a knowledge base sits low on both, and writes, deletes, payments, production changes, and external sends sit high enough to warrant a gate. Authorization should happen on every action rather than only at session start, with task-scoped, short-lived credentials that limit excessive agency in AI agents.
Routing every action through review produces rubber-stamp approvals, and routing none leaves autonomy unbounded. Reviewers need context to make a real decision, including the requested action and its arguments, the sources that led to it, the agent's identity, and the identity of the user who started the task. The approval record should capture the reviewer's identity and a timestamp, and if nobody responds before a timeout, the safe default is to block the action.
A policy check that runs before a tool executes can stop an action, and an alert after execution can only start a response. In the procurement scenario, a pre-execution gate on external email would have held the send and shown the reviewer the packet text that prompted it.
Budget Limits and Stop Controls Bound Ongoing Execution
Budgets handle the failures that gates miss because no single action looks risky. Layered per-cycle, per-task, and per-day limits catch a runaway loop, aggregate overwork, and sustained elevated usage.
These limits should track both iteration count and cumulative tokens and run as checks before each invocation, outside the agent's own control loop, so a manipulated agent cannot raise them. A circuit breaker adds a shared view, tracks retries across concurrent executions, and trips as they rise.
Stronger stop controls let responders disable a tool or switch it to read-only, revoke the agent's credentials, and pause or terminate the job. Rollback applies only where the action is reversible, so staged deletes and a restorable last-known-good configuration must exist before an incident.
Without a gate, an external-recipient alert would flag the procurement agent's email. Responders would disable the agent's email tool, revoke its credentials, pause other runs that read the same packet, and quarantine memory written during the run.
They would keep the trace, retrieved document, tool arguments, and identity context as evidence. Because nobody can recall a sent email, recovery means notifying affected people and closing the injection path by treating retrieved content as untrusted.
How to Implement and Maintain an AI Agent Monitoring Plan
An AI agent monitoring plan works when every agent has an owner, success criteria, instrumentation, an evaluation method, and a named responder, and when the plan reruns after every change. Ownership comes first, and nothing later in the plan holds without it:
Inventory and Ownership: The inventory lists every agent, its tools, its data access, and the accountable person.
Task and Policy Criteria: Criteria define a correct outcome and which actions are out of bounds.
Instrumentation: Each agent emits run, model, tool, retrieval, and identity records, with content capture set by risk.
Evaluation: Scoring covers verified outcomes and policy adherence, with human review of a sample to validate automated scores.
Release Testing: Scripted synthetic tests run after any model, prompt, tool, or permission change.
Response Ownership: The plan names who approves gated actions and who can pause or revoke an agent.
Remeasurement: The evaluation owner compares post-change metrics against the agent's baseline before its autonomy widens.
Without ownership or success criteria, evaluation and response have no reference point. Regular false-positive reviews prevent alert fatigue and keep responders from ignoring alerts.
What Should Agent Audit Records Capture, and What Can Monitoring Miss?
Audit records should let a reviewer reconstruct any consequential action, and security leaders should know what monitoring cannot see. For each consequential decision, the record should name the agent and its human sponsor, state the task intent and the data used, and identify the tool invoked, the authority level, any approval or escalation, and the outcome.
Those records need tamper-evident storage, access limited to named roles, and a defined retention period. The EU AI Act makes logging a legal duty for in-scope high-risk systems. They must record events automatically, and the Act's log retention rules require providers to keep those logs, to the extent they control them, for at least six months unless other law provides otherwise.
Reasoning traces may be unavailable, and a 2025 study on chain-of-thought faithfulness by Iván Arcuschin and co-authors found that chain-of-thought reasoning is not always faithful to what drove a model's answer, even on ordinary prompts. Readable reasoning can support a review, but tool calls and outcomes remain stronger evidence.
Make Agent Actions Reviewable and Controllable
Agent monitoring follows each run through model calls, tools, memory, and delegation, then ties that evidence to verified outcomes and to controls that act before damage spreads. An owner who cannot reconstruct an agent's last consequential action, or interrupt the next one, has monitoring in name only.
AI Agent Security, part of Abnormal's AI Security suite and in private preview, is designed to detect off-task runs and support automated containment. Explore the AI Security platform to see how Abnormal tracks what each agent can reach.
Frequently Asked Questions
Can security teams monitor AI agents without access to model internals?
Yes, because most useful evidence sits outside the model. Tool calls, arguments, retrieved sources, identity context, and outcomes are all observable through telemetry the surrounding application emits. Those records show what an agent did and whether anyone permitted it, even though they cannot fully explain why the model chose that path.
Does agent monitoring replace pre-deployment testing?
No, pre-deployment testing checks a change against known scenarios before release, while runtime monitoring watches real traffic for failures nobody predicted. Organizations that rely on monitoring alone discover problems through user reports, and those that rely on testing alone miss drift that appears after launch.
Can monitoring logs contain sensitive data?
Yes, when content capture records prompts and tool arguments that may contain personal or confidential data. The telemetry store then needs the same access scrutiny as the source systems.