Human-in-the-Loop AI Agents: How to Design Approval Gates

Human-in-the-loop AI agents pause before risky actions so a person can approve, edit or reject them. Good design means pausing only where human judgment changes the outcome, giving the reviewer enough context to decide quickly, and treating silence or errors as a "no."

Route everything to people and approvals become rubber stamps. Route nothing and the agent acts without limits. This guide shows how to sort actions into risk tiers, build the pause-and-resume flow, and keep an audit record.

What a Good Approval Gate Does

AWS's Agentic AI Lens describes risk-tiered approval: pause agents only for decisions where human judgment matters, with enough context to make those decisions meaningful. Agent frameworks now support this directly:

• OpenAI Agents SDK: a tool call that needs review creates an interruption and a resumable state

• LangChain: human-in-the-loop middleware supports approve, edit, reject and respond decisions

• Durable workflow engines: tools such as Temporal keep a paused approval alive across restarts, with timers for deadlines

The framework handles the pause. Your team still decides which actions need it and what the reviewer sees.

Step 1: Classify Actions by Risk

List every tool the agent can call, then sort each into a tier.

Tiers depend on context. A refund of $20 and a refund of $2,000 are the same tool but different tiers. Encode thresholds as rules, not prompt instructions.

Step 2: Pause Before the Side Effect

The gate must sit between the agent's proposal and the system that acts. Put the check next to the tool, not only at the start or end of the conversation, because agent-level checks may not run on every tool call in a multi-agent flow.

• Persist state: save the proposed action and run state so it can resume hours later

• Approve exactly what runs: do not regenerate the action after approval

• Revalidate on resume: check that the record has not changed since the request

• Use idempotency keys: a retry should not send the email twice

Step 3: Give Reviewers the Right Context

Reviewers who must read raw logs will approve blindly. Show a short decision card:

• Proposed action: what will happen, in plain words, with the exact values

• Why: the agent's stated reason and the evidence it used

• Impact: amount, customer, or systems affected

• Policy: the rule that triggered review

• Options: approve, edit, reject with reason, or escalate

Link the full trace for anyone who needs it, but keep the card readable in under a minute.

Step 4: Define Timeouts and Escalation

Decide in advance what happens when nobody answers.

• Default to reject: a missing or malformed decision is not approval

• Remind, then escalate: send a reminder, then route to a backup reviewer

• Tell the requester: the customer or employee should know the action is pending

• Cap queue age: expire requests that are no longer valid

Step 5: Record Every Decision

Log the reviewer's identity, timestamp, decision, edits made, the policy version and a copy of what the reviewer saw. That record is what auditors and incident reviewers will ask for. Our guide to AI agent audit trails covers the log design in detail.

Worked Example: Refund Agent

Illustrative example. A support agent can look up orders and issue refunds.

• Tier 1: order lookup runs automatically

• Tier 2: adding a case note runs automatically

• Tier 3: refunds up to $100 for the order's own customer run automatically; $100 to $500 need one supervisor

• Tier 4: refunds over $500, or to a different payment method, need a supervisor and a finance approver

• Timeout: no decision in four business hours sends a reminder; eight hours escalates to the team lead

• Record: decision card, reviewer, policy version and final refund ID stored with the case

After a month the team reviews which approvals were never rejected. Those are candidates for raising the threshold. Which were often edited? Those need a better agent prompt or tool.

Measuring Whether Gates Work

• Approval rate by tier: near 100% may mean the gate is unnecessary or rubber-stamped

• Edit rate: frequent edits point to a weak tool or prompt

• Time to decision: long waits push teams to bypass the agent

• Post-approval incidents: approved actions later reversed show where context was missing

Questions to Ask Any Agent Development Partner

• Where exactly is each approval enforced, and can we see it in code?

• How is paused state stored, and what happens after a restart?

• What does the reviewer see, and can our approvers test it before launch?

• How are timeouts, rejections and edits recorded?

• Who can change thresholds, and is each change logged?

Pre-Launch Checklist

• Every tool listed with its tier and threshold

• Gate enforced in code next to the tool

• State persisted and resumable

• Decision card reviewed by real approvers

• Timeout defaults to reject

• Backup reviewer named for each queue

• Decision log stored with retention rules

• Tests cover approve, edit, reject, timeout and changed record

For broader pre-launch testing, see our AI agent testing checklist.

Common Mistakes to Avoid

• Approval in the prompt only: "ask before refunding" is not a control; enforce it in code

• Gating everything: reviewers stop reading and approve by habit

• Approving a summary, executing something else: show exact values

• No owner for thresholds: limits never change as the agent improves

• Treating approval as security: approvals do not replace least-privilege permissions

Limitations

Human review adds delay and cost, and reviewers make mistakes. Some regulations, such as Article 14 of the EU AI Act for high-risk systems, set their own human oversight requirements; confirm which apply to you with counsel. Thresholds in this guide are illustrative.

Next Step

If you are building an agent that writes to business systems, LuMay's AI agent development team can help define risk tiers, approval flows and audit records for your workflows.

• AWS Agentic AI Lens, AGENTSEC04-BP02 human-in-the-loop

• LangChain human-in-the-loop overview

• Temporal, human-in-the-loop document approval guide

• EU AI Act Article 14, Human Oversight