The risk lands on the team that deployed the agent. Security engineers get paged for actions no human initiated, IT owners find changes in systems no ticket approved, and legal and compliance teams inherit an audit trail that can't answer the basic question of who, or what, did the thing. The agent's apparent autonomy doesn't shift accountability; it concentrates it on whoever put the agent in front of the tools, which is why AI agent governance has become a board-level concern rather than a purely technical one.
This article works through that gap end to end: what "acting without oversight" means for an AI agent, the documented failures that followed when oversight was missing, who carries the liability, why agent sprawl widens the exposure, and what runtime controls close it. The through-line is simple. AI agent governance has to live in the execution path, not in the prompt.
What "acting without oversight" means for an AI agent
An AI agent acts without oversight when nothing outside the model constrains what it executes: standing permissions persist, and tool calls face no per-call authorization.
In that setup, teams also cannot reliably attribute or reverse its actions. NIST defines agentic AI as a system "specifically designed to understand complex workflows and achieve goals autonomously, with little to no human interference," and notes that what distinguishes it is "more authority to act" on its own decisions.
NIST autonomy levels break that authority into four levels:
Assistance: the AI doesn't affect standard actions.
Collaboration: the AI and human work together on decisions.
Operation: the AI carries out actions under human supervision.
Fully autonomous: the algorithm runs "without reliance on human intervention or oversight."
In practice, most teams set one level for the whole agent, which is the mistake; the autonomy level belongs on each action, based on how bad the action could be and whether a human could catch a mistake in time. That per-action framing is the foundation of practical AI agent governance.
The resulting vulnerability class, sometimes called Excessive Agency, arises when agents receive tools they don't need, privileges beyond the task, or the ability to take high-impact actions without confirmation. Runtime tool access deepens the problem, because the model decides on the fly which tool to invoke and with what data, an attack surface perimeter controls were never designed to cover.
The common failure modes — model misbinding, context spoofing, prompt-state manipulation, insecure memory references, and covert channel abuse — sit behind nearly every incident that follows.
The real consequences: what unchecked agents have already done
From 2024 through mid-2026, documented agent failures show a consistent pattern: agents with standing permissions, no per-call authorization, and no external stop gate produced outcomes their operators could not predict, reverse, or in some cases even attribute.
The four incidents below span coding assistants, productivity copilots, and integrated developer tools, and each maps directly to a control missing from the execution path.
1. Gemini CLI destroys a user's files during a routine reorganization
In July 2025, Google's Gemini CLI tool wiped a user's files while attempting a straightforward folder reorganization. Product manager Anuraag Gupta asked the agent to move a batch of files into a new folder; the folder-creation step silently failed, but the runtime proceeded as if it had succeeded.
The Windows move command it issued next targeted a destination that didn't exist, renaming each source file to the same target name in sequence and overwriting one after another until only a single file remained.
Asked what happened, the model attributed the outcome to its own "gross incompetence" — an explanation the runtime had no way to convert into a rollback, because none existed.
2. A Cursor agent destroys the PocketOS database in nine seconds
In April 2026, a Cursor agent running Anthropic's Claude Opus 4.6 hit a credential mismatch in staging and resolved it by deleting the Railway cloud volume holding the PocketOS production database, along with all volume-level backups, in a single unauthorized API call that took nine seconds. Its post-facto explanation: "I violated every principle I was given."
3. EchoLeak turns Microsoft 365 Copilot into an exfiltration channel
The EchoLeak exploit (CVE-2025-32711, CVSS 9.3) let a single crafted email, requiring no action from the recipient, cause Microsoft 365 Copilot to exfiltrate internal files to an attacker-controlled server. The agent's own tool access became the delivery mechanism, with no user action, click, or approval anywhere in the chain.
4. Attackers hijack Claude Code through a Sentry integration
In June 2026, attackers hijacked Claude Code through its Sentry integration: malicious content in error events made the agent execute attacker-controlled commands, and the same exposure exists in Datadog, PagerDuty, and Jira integrations.
Security Operations Center (SOC) teams had no existing playbook to distinguish a developer running a command from an agent running it in response to a poisoned error event.
Who's accountable when an ungoverned agent causes harm
The deploying organization is. The EU AI Act and US agency law reach that answer from different directions, but both put AI agent governance squarely on the party that put the agent into production.
Under EU AI Act Article 26, deployers of high-risk AI systems must assign human oversight to people with the necessary competence, monitor system operation, inform providers and authorities of identified risks, and keep logs for at least six months.
Article 14 requires the systems themselves to be designed so natural persons can effectively oversee them, with oversight "commensurate with the risks, level of autonomy and context of use." On August 2, 2026, the Commission's enforcement powers against general-purpose AI model providers were activated. The fines can reach up to 3% of global annual turnover or €15 million, whichever is higher.
In the US, agency law attributes an electronic agent's actions to the party that deployed it, so the system's apparent independence doesn't shift responsibility. Baker McKenzie's June 2026 analysis notes that a California statute now forecloses defendants from arguing that AI autonomously caused alleged harms.
A June 2026 executive order directs the DOJ to prioritize enforcement against bad actors using AI agents, and companies must "govern, monitor, and explain their AI agents." The negligence question has shifted from "did you program this behavior" to "did you implement appropriate oversight and controls."
Internally, NIST GOVERN 2.3 assigns AI deployment risk decisions to executive leadership. A governed execution surface turns those decisions into records instead of assumptions, making the audit trail and the operating record the same artifact. Check whether vendor indemnification clauses written before agentic AI cover autonomous agent actions or downstream data harm.
Why agent sprawl makes AI agent governance harder
You can't govern agents you can't see, and most organizations can't see theirs. A Cloud Security Alliance survey in April 2026 found 82% of enterprises had discovered previously unknown AI agents in their environments within the past year; 41% had found them more than once.
The sanctioned and unsanctioned populations grow together, and both fall inside the scope of AI agent governance whether the deploying team realizes it or not. Gartner predicts 40% of enterprise applications will include task-specific AI agents by the end of 2026, up from less than 5% in 2025. Governance maturity is not keeping pace; the Cloud Security Alliance survey found enterprises repeatedly discovering agents they did not know were running.
Tines' Voice of Security 2026 found that 73% of security professionals expect their tech stack to expand further, so consolidation alone will not close the oversight gap. Visibility constrains governance. The same sprawl reaches IT and operations workflows as well as security, meaning governed intelligent workflows apply wherever agents and automations touch business systems.
The identity layer wasn't built for this population either. Most organizations rely on shared API keys rather than per-agent identities, which means logs record agent actions under the identity that held the key, not under a distinct agent identity.
The cost of staying here is high: many enterprises are expected to demote or decommission autonomous AI agents due to governance failures. Oversight has to operate where the agents do, at runtime.
How to govern AI agents at runtime: a practical checklist
Effective controls live in the execution path, outside the model, not in the prompt. A guardrail written into a prompt asks the model to comply; a runtime control blocks the tool call before it executes. Prompt injection and context drift bypass prompt-based guardrails trivially, so a compromised agent must be structurally unable to perform high-impact actions.
Here are the practical steps enterprises can take to govern AI agents effectively at runtime:
Issue scoped, short-lived credentials: Give agents just-in-time tokens bound to the specific resource, repository, or branch they need. When the task ends, the credential expires, so a compromised agent cannot pivot into adjacent systems.
Put approval gates on irreversible actions: Require human sign-off before deletions, external communications, billing changes, or bulk updates. If no human responds within the set window, the default should be deny or pause, never approve.
Enforce allowlists outside the model: A component external to the model must decide which tools and actions the agent is allowed to invoke. The model should never be the sole arbiter of what it can call.
Maintain tamper-evident audit trails: The authoritative log must live on infrastructure the agent runtime cannot reach, and it should record which human initiated the goal, which agent acted, what tool path was used, and whether the action stayed in scope.
Build and test kill switches: Treat kill-switch response times as behavioral requirements, and verify them through hands-on drills rather than assuming they work.
Assign every agent a distinct identity: Replace shared API keys with per-agent, non-human identities so every action can be attributed to a specific actor in the logs.
Apply per-action autonomy, not per-agent: Set the autonomy level on each action based on its potential impact and reversibility, rather than granting an agent blanket authority across all tasks.
Route consequential steps to humans: Pair deterministic enrichment and agentic scoring for routine work, but reserve the consequential final step for human review.
These controls share one architectural requirement: every agent action must pass through a layer that can inspect and stop it. That is what per-action autonomy looks like in production.
Oversight is what makes delegation possible
The incident record makes one thing clear: agents need action-level governance, and prompts alone cannot deliver it. Every agent action has to pass through a layer that can inspect, log, and stop it, on infrastructure the agent itself cannot alter. Delegation without that layer isn't delegation; it's exposure with extra steps.
That's the gap Tines is built to close. Whether a team builds with Claude Code, Codex, or Tines' own builder, the work runs inside a governed environment: every step, agentic, deterministic, or human, produces the same auditable record, honors the same role-based access and approval gates, and can only reach the tools it's explicitly granted. IT and security don't have to choose between letting teams build fast and knowing what's running in production; they get both.
Your agents are already in production, sanctioned or not. The organizations closing this gap aren't the ones banning the tools; they're the ones giving that work somewhere governed to run.
Frequently asked questions
Which AI agent actions require human approval?
High-impact and irreversible actions (wiping endpoints, disabling accounts, deleting data) need human gates. Routine enrichment and triage can run without approval. The practical progression is to start human-in-the-loop, where analysts approve closures and response actions, then graduate to human-on-the-loop as confidence builds. Train the approver too: putting someone in the loop without teaching them what to approve or when to escalate produces automation complacency, not oversight.
How do you audit an AI agent's actions?
Effective audits require two components: action logging, a complete record of what changed and which version of the agent or model made the decision, and decision context, proof the agent based its decision on appropriate, authorized data. Every agent needs its own distinct non-human identity, because shared credentials make logs unable to attribute actions to specific actors. The authoritative trail must sit on infrastructure the agent cannot modify.
When can AI agents run fully autonomously?
Low-impact, reversible actions (enrichment, triage, classification) are candidates for full autonomy once monitoring shows stable behavior. Consequential actions still need gates. As TIAA CISO Upendra Mardikar has put it, complete autonomy isn't the goal; a human always has to stay in the loop. Autonomy tends to expand gradually, per action rather than per agent, as safe deployments build the trust to relax specific gates.
How does the EU AI Act affect AI agent governance in 2026?
The EU AI Act's concrete deployer duties include assigning oversight and monitoring operation. It also requires log retention, with general-purpose model enforcement activating in August 2026. In the US, NIST launched its AI Agent Standards Initiative in February 2026, focused on agent identity, authentication, action logging, auditability, and containment boundaries, and the Senate's AI AGENT Act would require user-linked accountability and FTC registration for AI agents. China also introduced new regulatory developments addressing AI agents and anthropomorphic AI in July 2026.