AI & Enterprise Intelligence
AI agents need boundaries before they need more tools
A capable agent also needs a clear definition of authority: what it may read, what it may prepare and which actions require a person to decide.
Preparing a purchase request and placing an order are different responsibilities. An agent may be useful at the first without being authorised to perform the second. That distinction should survive an unclear instruction, an unexpected document and a convincing but mistaken recommendation.
A helpful starting point is an action map. List what the agent can observe, propose, change and send. For each action, identify the relevant records, the identity under which it runs and the conditions that must be satisfied before it takes effect.
Give the task a suitably narrow tool
A tool that retrieves an order is different from a general-purpose tool that can edit every table. Prefer functions with explicit responsibilities and constrained arguments. OWASP identifies excessive functionality, permissions and autonomy as causes of excessive agency, and recommends removing capabilities the task does not need. [1]
For a hypothetical purchasing assistant, the initial toolset might search an approved catalogue, inspect a request and prepare a draft. Sending that draft for approval can be a separate function from submitting an order. The distinctions make the workflow easier to explain and allow each operation to have its own policy. Tool availability should reflect the current task rather than every integration the organisation owns.
Enforce authority outside the model
A prompt can describe a rule, but downstream services must enforce it. OWASP recommends validating tool requests against security policies and executing actions within the user's authorised scope. Check the operation and the target record, not merely whether the tool name is on an allowlist. [1]
Use credentials appropriate to that responsibility. Google recommends dedicated service accounts for separate applications and limiting their privileges; shared powerful identities make activity harder to attribute and can accumulate unnecessary access. Avoid long-lived keys where a more suitable workload identity or temporary credential is available. Short-lived credentials still need a narrow permission scope. [2]
In the purchasing example, permission to read a supplier record should not imply permission to change its payment details. That boundary belongs in the application and service permissions even when the model has correctly understood the business request.
Make approval a decision about a specific action
A useful approval screen shows the proposed change: the destination, the affected record, the content being sent and any consequential side effect. It gives the reviewer enough context to decline or correct the action. If the proposal changes, require review of the changed proposal rather than treating an earlier approval as unrestricted consent.
Check what the platform actually governs. Anthropic's Managed Agents documentation distinguishes server-evaluated permission policies from human approval and states that its automatic policy is not a human checkpoint. It also makes the application responsible for deciding whether custom tools should execute. A framework's approval feature therefore needs to be mapped to the actual tools and execution path. [3]
Keep low-impact routine work practical. Escalation should be proportionate to the action, with deliberate approval boundaries around consequential changes instead of a stream of prompts that reveals little about what is being authorised.
Keep retrieved content on the data side of the boundary
Agents may encounter instructions inside a page, email or document they were asked to examine. OWASP describes this as indirect prompt injection: external content can alter a model's behaviour in unintended ways. Retrieval and fine-tuning do not fully remove that vulnerability. [4]
For example, an illustrative supplier document might tell the assistant to send an internal file to a different address. The document is evidence about a supplier, not authority to change the user's task. Treat external content as untrusted, preserve where it came from and restrict what the execution layer can do with a resulting tool request. Instructions, validation, restricted access and review work together; none makes every possible attack disappear. [4]
Design what happens when execution becomes uncertain
A request can succeed even when its response never reaches the caller. Retrying an action blindly may then repeat its side effect. AWS's guidance on idempotent APIs explains how a caller-provided request identifier can help a service recognise retries and avoid creating duplicate results. Where an integration supports that contract, use it; otherwise define how to inspect the existing state before trying again. [5]
Set limits on run time, spending and repeated attempts. Keep an audit trail of the proposed action, policy decision, approval and execution result, with appropriate protection for sensitive content. Provide a stop mechanism and a route for a person to resolve uncertain outcomes. OWASP recommends logging, monitoring and rate limits to reduce the impact of undesirable actions. These controls help contain an error while someone determines what happened. [1]
Sources & further reading
The references behind this perspective. Reviewed . Product documentation can change; check the source before making an implementation decision.
- OWASP — Excessive Agency
Primary security guidance on limiting tool functionality, privileges and autonomy, enforcing downstream authorisation, human approval, logging and rate limits.
- Google Cloud — Best practices for using service accounts securely
Primary identity guidance on single-purpose service accounts, least privilege, avoiding service-account keys and temporary elevation or downscoped credentials.
- Anthropic — Managed Agents permission policies
Primary product documentation distinguishing automatic tool-policy evaluation from human approval and identifying custom-tool execution as the application's responsibility.
- OWASP — Prompt Injection
Primary explanation of direct and indirect prompt injection, limitations of retrieval and tuning as mitigations, and defence-in-depth considerations.
- AWS Builders' Library — Making retries safe with idempotent APIs
Primary engineering discussion of uncertain request outcomes, repeated side effects and caller-provided identifiers for idempotent retry contracts.
