AI Agent Approval Policies: Designing Human Checkpoints by Risk
Not every tool call needs a human. Approval should be tied to the potential impact, reversibility, and uncertainty of the action.
Not every tool call needs a human. Approval should be tied to the potential impact, reversibility, and uncertainty of the action.
Create risk-based approval policies so agents can automate routine work while humans retain control over consequential operations.
Putting a human in front of every agent action defeats much of the value of automation.
Putting a human in front of none of them creates unnecessary risk.
The better approach is risk-based approval.
Classify actions by their potential consequence.
Low-risk examples include searching public information or formatting a document.
Moderate-risk actions might update a customer record or create an internal ticket.
High-risk actions include deleting data, moving money, changing permissions, deploying production code, or sending sensitive external communications.
Do not ask a user to approve a vague statement such as "the agent wants to continue."
Show the actual proposed action:
The human should approve the operation, not blindly approve the agent.
If users approve dozens of low-risk actions, they will eventually approve high-risk actions without reading them.
Use automatic policies for routine work and reserve approval for meaningful boundaries.
Approval should not become a reusable token that lets the agent perform unrelated actions.
Bind it to the resource, operation, relevant parameters, user, and short time window.
If the agent changes the proposed action materially, require another decision.
Record the proposal, policy classification, approver, timestamp, final operation, and result.
This makes the approval chain reviewable.
Human approval works best as a targeted control, not a universal brake.
Automate low-risk work, stop high-impact actions, show humans exactly what will happen, and bind approval to the specific operation.
Source: AI Agent Security and least-privilege guidance.
The permission decision should be made by trusted application code rather than by the language model. Validate the authenticated user, agent identity, tenant, resource, operation, and current policy before executing a side effect. Return only the data needed for the task, and record important allow and deny decisions in an audit trail.
When permissions are changed, invalidate affected sessions or credentials where appropriate. Keep development and production authorization separate, and make privileged operations easy to revoke. A secure agent is not one that promises to stay inside its permissions; it is one that cannot cross those permissions without another trusted control.
Traditional application authorization often assumes that a human chooses the operation. Agents change that assumption because the model can select tools dynamically. A permission system therefore has to assume that the requested operation may be surprising, malformed, or influenced by untrusted content.
The safest pattern is to make every capability explicit. Instead of giving an agent a broad API client, expose narrow operations with clear input schemas. The authorization layer should then evaluate the requested operation independently of the model's explanation.
For each tool, document the principal, resource, operation, tenant, environment, data sensitivity, reversibility, approval requirement, and expiration. This produces a permission map that can be reviewed by engineering and security teams.
Then test the negative cases. Ask what happens when the agent requests another tenant, an expired resource, a deleted record, an operation outside its role, or a privileged action without approval. Every one of these cases should fail before sensitive data or side effects reach the underlying system.
Keep authorization decisions close to the resource being protected. API gateways can provide coarse controls, but the final service should still verify ownership and scope. Cache permissions carefully because stale authorization can become a security bug. When a role or tenant changes, invalidate affected sessions and cached decisions where necessary.
Also make privileged operations observable. An allow decision is important evidence, especially for actions involving customer data, payments, deployments, permissions, or deletion.
Use four layers: identity, capability, resource scope, and risk policy. Identity establishes who is acting. Capability defines what the agent can request. Resource scope defines where it can act. Risk policy determines whether additional approval or temporary access is required.
This model remains understandable as the product grows because each layer answers a different question. It also makes incident response easier: a security engineer can see whether the problem came from identity, an overly broad capability, a missing resource check, or a policy decision.
Before shipping an agent capability, ask whether the permission is narrower than the underlying service credential, whether a user can access the same resource, whether tenant isolation is enforced server-side, whether the operation can be reversed, and whether the permission can be revoked quickly.
The goal is not to create a perfect authorization matrix on day one. It is to make every new capability deliberate, scoped, testable, and observable.
Permission design becomes clearer when the team tests realistic failures rather than only ideal requests. Try an agent that receives a stale session, an unexpected tenant identifier, a resource owned by another customer, a missing approval, or a tool argument outside the documented schema. Also test what happens when the policy service is unavailable. Sensitive operations should fail closed rather than silently falling back to a broad service credential.
Test delegated workflows too. If one agent asks another agent to perform an action, the downstream agent should not automatically gain the first agent's entire permission set. Carry the original user and tenant context through the delegation chain and authorize the final operation independently.
Authorization is not static. Users change roles, organizations change ownership, projects are archived, credentials expire, and products add new tools. A permission that was safe yesterday can become inappropriate tomorrow.
Build revocation into the lifecycle. When access changes, invalidate affected cached decisions and sessions according to the risk of the system. For high-impact operations, prefer short-lived grants so that changes naturally take effect quickly.
The agent can explain why it wants to perform an action, but that explanation should never be the authorization proof. A persuasive model response is still untrusted input.
The final decision should come from identity, policy, resource ownership, and explicit permissions evaluated by trusted code. This separation is what allows the product to remain secure even when the model is manipulated by a prompt injection or simply makes a bad decision.
Someone should own the permission map. For a small SaaS this may be the founder or engineering lead. As the product grows, document which team owns each capability, who can approve privileged changes, how emergency access works, and how old permissions are reviewed.
A permission system is successful when developers can explain it quickly and security reviewers can verify it without reading the model's internal reasoning.
Community
0 comments
React to this article
Trending now
Written by
Kirtesh Admute
Founder
Kirtesh Admute is the founder of IndieFounder, a platform for founders, builders, and people curious about technology. He writes about AI, startups, software, product building, and the lessons that come from building in public.
See an issue with this story?
Continue reading
