When AI agents hand each other work, one agent's wrong answer can become the next agent's set of instructions. Every added agent gives an error another place to spread. Limiting how far a mistake travels belongs at the center of agentic AI risk management.
What is agentic AI risk management?
Agentic AI risk management identifies, assesses, and controls the risks created when autonomous agents plan and act through tools, data access, and other agents. Unlike traditional model risk management, it has to account for behavior that changes at runtime, combining risk categories, human oversight, identity and access controls, monitoring and audit, and governance frameworks.
The Agentic AI Risk-Management Standards Profile, published in February 2026 by researchers at UC Berkeley's Center for Long-Term Cybersecurity (CLTC), is a practical reference. The profile is designed to complement the National Institute of Standards and Technology (NIST) AI Risk Management Framework (AI RMF) and adds agent-specific practices to the framework's Govern, Map, Measure, and Manage functions. It names five risk-management levers:
Human control and accountability: The organization defines roles, intervention points, escalation paths, and shutdown mechanisms.
System-level risk assessment: Reviews cover the whole system, especially multi-agent interactions and tool use.
Continuous monitoring: Oversight continues after deployment because agent behavior can change.
Defense-in-depth and containment: Sufficiently capable agents are treated as untrusted.
Transparency and documentation: Teams record and communicate boundaries, limitations, and risk decisions.
The profile also grades autonomy on six levels, from L0, where the user has direct control, to L5, where the user only observes. It asks for controls proportionate to each agent's level.
Why autonomous agents need a different approach than traditional AI risk
Agents need a different approach because traditional model risk management assumes something stable to validate. Pre-release testing and sign-off suit a classifier that returns an answer and stops.
Agents keep going after the answer. They call tools, act with credentials, and often keep memory between sessions, so the same agent can behave differently next week with no change to its code. Memory adds its own risk, because corrupted context can stay available and shape the agent's later planning and tool use.
Inputs stay risky after launch too. The UK National Cyber Security Centre (NCSC) warns that prompt injection may never be fully mitigated, because large language models cannot reliably distinguish data from instructions.
The main risk categories of agentic AI
Six risk categories matter most, and the Open Worldwide Application Security Project (OWASP) Top 10 for Agentic Applications 2026 covers several of them:
Excessive agency: An agent holds more functionality, permissions, or autonomy than its task needs, so a bad output can trigger damaging actions.
Prompt injection: Hidden instructions in content the agent reads, such as a web page or ticket, redirect its actions.
Tool misuse: An agent uses authorized tools in harmful ways, often within its granted permissions.
Memory poisoning: Corrupted long-term memory persists across sessions and alters later reasoning.
Cascading failures: A single fault spreads across agents, tools, and workflows, compounding into wider harm.
Goal manipulation: Manipulated inputs change the agent's objective, which OWASP calls agent goal hijacking.
These risks rarely stay separate. OWASP's agentic list notes that memory poisoning frequently leads to goal hijacking, because corrupted context can change how an agent interprets its goal. Any of these faults does the most damage once another agent acts on it.
How one agent's mistake cascades into many
A cascading failure happens when one agent's error becomes another agent's input and spreads through every action built on it.
A cascade is a trust failure. Each agent treats the previous agent's output as a checked input and acts on it with its own permissions. Errors typically spread through:
Inherited trust: A downstream agent accepts an upstream conclusion without checking it, so one hallucinated fact drives several actions.
Borrowed authority: A low-privilege agent passes a request to a higher-privilege agent, which carries it out with access the first agent never had.
Machine speed: Agents act and retry faster than people review, and the CLTC profile warns of "rapid and iterative action execution that can outrun monitoring and response."
Borrowed authority has already shown up in production software. AppOmni disclosed in November 2025 that a low-privileged user could embed instructions in a ServiceNow case description and steer a higher-privilege Now Assist agent into carrying out those instructions, such as changing records and sending outbound emails. ServiceNow said the system worked as designed and updated its documentation.
Teams already know what happens when AI agents act without oversight, and in a multi-agent chain, human review has to sit at the handoffs.
Core controls that contain the damage
Containment works in layers. Scoped identity comes first. Human approval covers irreversible actions. Spend and reach limits cap volume. Guardrails, kill switches, and testing govern what an agent does within those limits. Each layer assumes the one before it will sometimes fail.
Identity and least-privilege access
Each agent needs its own scoped, non-human identity so one agent can't borrow another's permissions.
On April 25, 2026, according to the AI Incident Database, a Cursor coding agent at PocketOS deleted the company's production database and its backups, causing a disruption that lasted 30 hours. The New Stack reported that the agent had used an access token it found in an unrelated file, and that the token carried authority across the company's entire Railway account.
In Tines 3B, the complete environment for enterprise AI, teams connect agents to systems such as Okta through the credential proxy, which injects secrets into outbound requests at runtime so workflow code never holds them.
Human-in-the-loop approval
Approval should align with risk. Singapore's Infocomm Media Development Authority (IMDA) ties default approval requirements to each action's risk level in its Model AI Governance Framework for Agentic AI. Its case study on responsibly deploying OpenClaw names financial transactions, executing code in production, deleting critical data, and sending external communications on the organization's behalf as actions that warrant human review. An agent stays a tool under human oversight, never a peer or subordinate in the workforce.
The agent should never decide when to ask. In February 2026, Meta's Summer Yue told an agent to confirm before acting, and Business Insider reported that the agent lost the confirm-first instruction and mass-deleted her emails. Authorization belongs in downstream systems, not in the model's judgment of whether an action is allowed.
Spend and reach limits
Volume limits catch failures that look like normal work. Help Net Security, summarizing a Google Mandiant report, described an accounting agent that entered a runaway loop and made more than 15,000 high-cost API calls in under an hour, generating about $50,000 in cloud charges and disrupting active business transactions. Circuit breakers should trip on abnormal cost, latency, or spikes in tool calls, and limits should cap how many times agents can retry or hand work onward.
Three limits are worth setting:
Spend caps per agent and per workflow, with alerts before the cap
Rate limits on tool calls and outbound requests
Transaction thresholds above which a payment, refund, or permission change pauses for a person
These limits only measure volume, so guardrails, kill switches, and adversarial testing have to govern what an agent does within them.
Guardrails, kill switches, and adversarial testing
Policy belongs outside the model. NIST's design tenets for agent identity require architectural enforcement, saying "Guardrails alone are insufficient" and that the separation of instructions and data "must be enforced by architecture design." Sandbox tool calls to limit injection harm.
Shutdown covers what guardrails miss. CLTC recommends automated shutdown when an agent reaches outside its authorized scope, manual shutdown as a last resort, and isolation from dependent systems. Many teams haven't tested shutdown. In ISACA's 2026 AI Pulse Poll, 56% of more than 3,400 digital trust professionals didn't know how quickly they could halt an AI system after a security incident.
Red-teaming tests both layers. A UK AI Security Institute competition saw 1.8 million prompt-injection attacks produce more than 60,000 policy violations. Any change to prompts, tools, memory, retrieval, or model provider is a reason to retest.
Monitoring, audit trails, and accountability
Continuous monitoring and logged decision lineage replace one-off validation as the main source of assurance. A useful log captures prompts, responses, tool calls, approvals, and identity context in centralized, tamper-resistant storage.
When an agent goes wrong, a short sequence limits the spread:
Revoke credentials: Suspend the agent's keys and tokens the same way teams handle a compromised service account, then confirm access has actually ended.
Alert dependent agents: Isolate the agent from dependent systems, then pause or notify the agents and workflows that consume the agent's output.
Preserve logs: Freeze memory and tool-call history without deleting them, since the context behind a prompt injection is often lost when a session expires.
Accountability stays with the organization. In Moffatt v. Air Canada, a British Columbia tribunal rejected the airline's attempt to treat its chatbot as responsible for itself and wrote, "It should be obvious to Air Canada that it is responsible for all the information on its website." Every agent should have a named owner before it reaches production.
Frameworks to align with
Five frameworks give auditors, regulators, and internal reviewers a shared reference for agent controls:
NIST AI RMF: The base framework the CLTC profile extends.
ISO/IEC 42001:2023: A certifiable AI management system standard that covers agents as AI systems.
EU AI Act: Under the 2026 Digital Omnibus amendment (Regulation (EU) 2026/1744), high-risk obligations apply from December 2, 2027 for Annex III systems and August 2, 2028 for Annex I products.
OWASP Top 10 for Agentic Applications 2026: Its community-reviewed list covers most of the categories above.
Berkeley CLTC profile: It adds autonomy levels and shutdown guidance.
The mapping below is an interpretation of how each risk category lines up with NIST functions and, for systems classed as high-risk, EU AI Act obligations:
This table is accurate as of publishing in November 2026. It is general guidance, not legal advice.
How to put agentic AI risk management in place
Start with a low-risk, contained workflow where errors are visible and reversible, and expand only as controls prove out.
Inventory agents: List every agent, its owner, its tools, and the agents it hands work to. In a Cloud Security Alliance survey commissioned by Token Security, 82% of respondents found unknown AI agents running in their environments.
Score each agent: Rate autonomy, tool access, and impact, and let the highest score set the controls.
Pilot with approvals: Require sign-off on every consequential action and review the logs.
Scale: Loosen approvals only where the logs show the agent behaving as intended.
Consider an agent that reads access requests, checks Okta group membership, and posts a recommendation in Slack. At CLTC's L2, where user and agent collaborate, with read-only access and low impact, it needs only spot checks during a pilot. Once it can write to Okta groups, every grant should pause for a human-in-the-loop approver, because a wrong grant changes who can reach sensitive systems.
Containment at the handoffs, with scoped access, gated changes, and a record of every action, keeps one agent's error from becoming many. Tines 3B gives teams a governed environment for that work. Each step runs in its own isolated sandbox, and a credential proxy keeps secrets out of workflow code. Teams test changes on an isolated branch before pushing them live, with every commit kept for rollback. Audit logs record every user and system action, and role-based access control is scoped per space, connector, and network.
Teams can test these controls on their own agents in Tines 3B Explore Edition, a free version with limits that includes AI, single sign-on (SSO), and unlimited integrations.
