In April 2026, a Claude-based AI agent used for software development tasks deleted a company’s production database in nine seconds. Explicit security rules were in place, yet the agent still found a way around them.
Months earlier, an AI agent with access to email began deleting messages from the inbox of an AI security specialist, despite specific instructions designed to prevent it.
In both cases, security instructions were in place. The issue was how those controls had been implemented: the agent itself could interpret the limits and act beyond them.
These incidents point to an increasingly common problem. According to Kore.ai’s Agent Productivity Index 2026, 82% of companies report that their AI agents have autonomously taken actions with real-world consequences, even with active security controls in place.
The underlying issue is where those controls are implemented. Many organizations still rely on prompt instructions to enforce limits that should be technically built into the architecture.
In traditional software, permissions and access controls define in advance what each component can do. AI agents introduce another variable: an agent can decide in real time how to use the tools and access available to it. As a result, every permission expands the range of actions the agent may be able to take.
A guardrail is a technical control that sets clear boundaries around what an agent can say, what information it can access, and what actions it can execute. Its effectiveness depends on keeping those boundaries outside the model’s decision-making authority.
A permission works as a security control when enforcing it is outside the agent’s decision-making authority.
These risks can arise even when the model behaves as expected. It only takes access to an action that should have been restricted by design.
A prompt instruction can tell an agent to avoid deleting data or to request approval before taking an action. As an interaction becomes longer and more complex, those instructions can carry less weight within the context the model uses to make decisions.
Detecting sensitive data during a conversation also addresses only part of the problem. Decisions about which tools an agent can use, which systems it can access, and which actions it is allowed to perform should be defined at the configuration and architecture levels.
The level of control should also reflect the risk of each use case. A read-only agent requires different controls from one that can initiate a transaction. Gartner projects that applying uniform governance without accounting for the actual risk of each agent could lead 40% of companies to discontinue autonomous agents by 2027.
If more than one of these questions does not have a clear answer, there is likely an area of risk that still needs greater visibility and control.
Building security controls into the design makes it possible to define permissions, agent responsibilities, and actions that require validation from the start. When these controls are added after agents have already been deployed, the changes may require significant adjustments to their architecture and configuration.
Organizations that incorporate security from the early stages are better positioned to move agents into production, expand their use cases, and maintain control as their level of autonomy increases.
Does your organization already have AI agents in production, or are you evaluating them for critical processes?
At EDSA, we help organizations design AI agent security across architecture, configuration, and platform layers, with controls aligned to the autonomy and risk level of each use case. Write to us at talk@edsa.com and let’s start a conversation.