An approval button is not proof that an agent waited. The tool may have run before the decision, the executor may have used different arguments, or a resume may repeat the effect.

The test must separate two questions: what decision did the policy record, and what happened in the external system? The guide to deciding when an AI agent should ask for human approval explains the gate design. This article focuses on proving it under approval, rejection, changes, and restarts.

Hand-drawn flow shows a tool proposal rejected or approved before one execution.

Short answer

  • A pending call must produce no effect before the decision.
  • Rejection or expiration must leave the effect counter at zero.
  • Approval must execute the same payload the reviewer saw, once.
  • Changing the target, arguments, or policy version must require a new decision.

What does the test need to prove?

The OpenAI Agents SDK human-in-the-loop guide describes a flow where a call needing approval does not execute, the run returns an interruption, and the state can resume later. That gives the minimum contract: pending before the effect, an explicit decision, and a resume of the same run.

Do not test only whether the interface displayed a question. Record the proposed call, decision, normalized payload, and state of the resource that could change. A green test should say whether the policy decided correctly and whether the executor respected that decision.

A small matrix already catches important failures:

Case Expected decision Expected effect
Low-risk action the policy allows allow One valid execution
Action that requires review pending No effect while waiting
Reviewer rejects deny No effect
Reviewer approves the original payload allow One effect with the same arguments
Payload changes after approval deny or new pending state No effect with the changed payload
Resume is repeated decision already consumed At most one effect

How do you test the decision without calling the real service?

Start with a fake executor. The test does not need to send an email, change a CRM, or delete a record to prove that the policy blocks the action. It needs to receive the same call the real executor would receive and leave simple evidence when it would be reached.

The OpenAI Agents SDK testing guide recommends deterministic utilities for testing workflows without provider requests. That boundary matters here: a model can propose a call in an integration test, but the approval rule and executor need predictable tests.

This is an illustrative fixture skeleton. A real application must persist state and define its own reviewer identity.

type Action = { tool: string; target: string; body: string };
type Pending = { id: string; action: Action; status: "pending" | "used" | "rejected" };

const effects: Action[] = [];
const pending = new Map<string, Pending>();

function propose(action: Action): Pending {
  const item = { id: "approval-1", action, status: "pending" as const };
  pending.set(item.id, item);
  return item;
}

function resume(id: string, decision: "approve" | "reject", action: Action) {
  const item = pending.get(id);
  if (!item || item.status !== "pending") throw new Error("stale approval");
  if (JSON.stringify(item.action) !== JSON.stringify(action)) {
    throw new Error("payload changed");
  }
  if (decision === "reject") {
    item.status = "rejected";
    return;
  }
  item.status = "used";
  effects.push(action);
}

The point is not to use JSON.stringify in production. The point is to make the relationship between the pending ID, approved payload, and one-time consumption explicit. Use a canonical comparison, policy version, and a transaction or idempotency key suitable for your storage.

How do you prove that rejection blocks the effect?

Send a call that would normally write to a fake adapter. Before resuming, assert that the state is pending and the adapter has not been called. Then reject the call and check both sides: the recorded decision is negative and the external state is unchanged.

That separation prevents a false green. A log can say denied while another path uses the same credential and performs the operation. The WorkOS method for testing agent authorization policies recommends checking the verdict and the state produced by the service separately. The recommendation applies even when the service is not email.

Include at least these cases:

  • explicit rejection before the executor;
  • expiration or timeout without automatic release;
  • an approval-screen error without a fallback to execution;
  • an alternate path that tries to call the same tool without crossing the gate.

If any case changes the counter, outbox, or test record, approval exists only in the interface. The gate belongs at the executor boundary, where every mutating call passes before the effect.

How do you test approval with a changed payload?

Approval belongs to a concrete action, not a vague intention. Approve one payload and change a field before resuming: recipient, amount, record, message, or scope. The executor should reject the call or create a new pending approval.

This is easy to hide. The reviewer sees "send the summary to the team," but the runtime fetches new data and rebuilds the body after the click. A test that compares only the text shown in the UI will miss the swap.

Store what the reviewer actually approved:

  • tool name and version;
  • agent and user identity;
  • normalized target and arguments;
  • policy version;
  • decision expiry;
  • an ID connecting proposal, approval, and effect.

When the application changes a protected field, invalidate the approval. Do not keep the decision because the change looks small. The test should make the rule observable, and the policy can decide whether a new review is automatic or required.

How do you test resume, restart, and one-time execution?

An approval that lives only in a worker's memory disappears on restart. The SDK shows resuming from the same run state and recommends keeping state on the server when approval may take time. The test should simulate that interval, not approve inside the same function that created the pending item.

Run the sequence in stages:

  1. Create the pending item and persist a serializable representation.
  2. Stop the process or discard the in-memory object.
  3. Read the state in a new execution context.
  4. Approve or reject with the expected identity.
  5. Resume twice and check that the effect appears at most once.

The last step matters because an API response can disappear after the external effect occurred. Resume needs an idempotency key, a consumable reservation, or a state query that reconciles the result before trying again. A retry must not turn one valid approval into two charges, messages, or updates.

Which failures belong in regression tests?

Turn each discovery into a fixed case with input, starting state, expected decision, forbidden effect, and final evidence. The AI agent trajectory testing guide can check call order and interventions. This article adds the authorization condition and the state outside the agent.

Include scenarios that change one variable at a time:

  • the correct tool is missing from the catalog;
  • the agent asks for approval for a read even though policy allows it;
  • rejection returns to the model and it tries an alternate path;
  • the reviewer lacks authority for the target;
  • arguments change while the pending item is open;
  • policy changes before resume;
  • the first send times out after reaching the service;
  • two workers consume the same pending approval.

Use TypeScript tool-call validation for format, identity, and arguments. Use the approval test for decision and consumption. Use an integration test for the adapter and external service. Combining all of this into one slow test makes it harder to find the broken boundary.

Checklist before releasing the agent

  • Is the pending item created before any mutating effect?
  • Does the test check the decision and external state separately?
  • Do rejection, timeout, and resume errors leave zero effects?
  • Is approval bound to the payload, target, and policy version?
  • Does a change after the click require a new decision?
  • Does the state survive worker restart?
  • Can two workers consume the same pending item?
  • Does the test use a fake adapter before a real sandbox?
  • Does the suite prove an allowed path, not only refusals?
  • Is the test's boundary written down so nobody treats it as a security certification?

The desired result is not "the agent showed a screen." It is a short trail connecting proposal, decision, execution, and final state. When the test fails, the team should know whether policy decided incorrectly, the executor ignored the decision, or the external service produced an uncertain result.

What does this test not prove?

An approval suite does not prove that the model understands every request, that policy covers every resource, or that an external integration always responds correctly. It proves only the cases and authorities described by the fixtures.

It also does not replace authentication, service-side authorization, credential isolation, logging, recovery, or operational review. If the agent can reach the endpoint through another path, a button test does not protect that path. Approval must be a runtime barrier, not a promise in the prompt.

Quick questions

Should I use the real model?

For policy, pending state, rejection, and the executor, start with controlled responses. A real model helps test selection and behavioral variation, but it should not decide whether its own suite passed. Keep effect expectations in deterministic code.

Must every tool approval block the tool?

No. Policy may allow a read or reversible action without intervention. The test should prove the chosen classification: allowed calls proceed, pending calls wait, and rejected calls never reach the effect.

What should the report retain?

Keep identity, policy version, proposed call, decision, executed payload, and state before and after. Do not retain secrets just to simplify debugging. A synthetic identifier is usually enough to connect the evidence.

How this article was produced

Samuel Fajreldines is the accountable author. The research compared the current OpenAI Agents SDK documentation for human-in-the-loop and deterministic testing with the WorkOS method for testing authorization policies. Recent public discussions were used only for language and recurring failure modes. The approval fixture ran locally with an in-memory effect counter. It did not call a model, real account, or external service.

I use RemoteCode as the author's tool for keeping context between sessions. It supported work organization, but it did not run the fixture or prove the flow's security.

Sources consulted