How to Verify an AI Agent Tool Call Before Retrying

An AI agent can reach an API and lose the response on the way back. The agent sees a timeout and assumes that nothing happened. If it retries, it may create a second order, send another message, or write the same record twice.

The fix is to treat the response and the effect as separate facts. Before execution, the runtime creates a stable action identifier. After execution, it stores the executor's receipt or queries a postcondition in the external system. The final state must say whether the effect is confirmed, absent, or still unknown.

This is narrower than avoiding duplicate tool calls during LLM failover. Failover decides when to switch a model attempt. This article asks what to do when the external operation may already have started. For the execution layer that preserves state across attempts, see durable execution for AI agents. The code is illustrative and was not run against a provider or production system.

Short answer

• A missing response tells you what the client received, not what the server did.
• Persist an actionId before sending a side-effecting operation.
• Query an independent postcondition and classify it as confirmed, not_found, or unknown.
• Retry only after not_found, or when the API uses the same idempotency key to deduplicate the attempt.

Why does a timeout not tell you whether the tool call worked?

A timeout separates dispatch from acknowledgement. The request may have been dropped before the server acted, accepted and processed without a response, or applied only part of the change. The runtime should not turn every transport error into failed. In 2026, Isham Kalappurackal Mansoor, Abhishek Phadke, and Pratip Rana describe this gap between the response channel and the effect channel in "Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures" (arXiv, 2026).

The risk appears when a tool changes state. A read-only lookup is usually safe to repeat. Creating a payment, granting access, sending an email, or changing a record requires the runtime to know which logical operation is being resumed. The text the model received is not a service receipt.

In 2026, the paper's simulated record_invoice task produced duplicate effects in 32%, 52%, and 76% of baseline runs at low, medium, and high fault levels. Its verification wrapper recorded 0%, 16%, and 20%. Those figures come from a controlled environment with injected failures. They are not production rates, but they show why "verify, then retry" deserves its own contract.

The missing case is a third outcome. success and failure describe what the client observed. unknown describes the limit of current knowledge: the action may have happened, but there is not enough evidence to retry or close it.

What should the runtime record before the operation?

Record the intent before dispatch, with a key that remains the same across attempts of the same logical action. The minimum record should connect the agent's task to the expected effect without depending on one provider message ID.

A domain tool can receive an actionId, normalized arguments, and the postcondition it will later verify. The model provider's call ID still helps trace the conversation, but it should not be the only key. A retry or fallback may produce another provider message for the same action.

type EffectState = "planned" | "sent" | "confirmed" | "not_found" | "unknown";

type ActionRecord = {
  actionId: string;
  toolName: string;
  inputHash: string;
  expectedEffect: string;
  state: EffectState;
  externalId?: string;
  receiptId?: string;
  updatedAt: string;
};

async function prepareAction(
  actionId: string,
  toolName: string,
  inputHash: string,
  expectedEffect: string,
): Promise<ActionRecord> {
  const current = await actionStore.get(actionId);
  if (current) return current;

  const record: ActionRecord = {
    actionId,
    toolName,
    inputHash,
    expectedEffect,
    state: "planned",
    updatedAt: new Date().toISOString(),
  };

  await actionStore.insertIfAbsent(record);
  return record;
}

insertIfAbsent must be atomic in the store that coordinates workers. A process-local Map does not cover restarts, two replicas, or a new queue delivery. The example does not solve concurrency by itself. It makes the record that a real implementation must protect visible.

The AWS Well-Architected Framework, "Make mutating operations idempotent", recommends unique tokens and tracking them so repeated messages do not produce the same action twice. The key helps the service recognize a repeat. It does not replace a query when the first result is ambiguous.

What makes a receipt trustworthy?

A receipt should say what the executor observed. It does not need the secret payload or another version of the model's story. It needs the action identity, the service decision, an external identifier when available, and the time at which the evidence was recorded.

A useful result separates transport from effect:

Do not turn HTTP 200 into confirmed without defining what was confirmed. An API may accept an asynchronous command, return an identifier, and apply the effect later. The receipt then proves acceptance, while the postcondition proves the state change the agent needs.

The OpenAI Agents SDK exposes separate items for tool calls and tool results, plus a requestId when the provider supplies one. Its Results documentation also treats serializable state as a surface for retries and resumption. That helps audit a run, but the application still has to prove the effect in the service that was changed.

How should you verify the postcondition before retrying?

Query a condition that does not depend only on the lost response. The query may look for an external ID, a row containing the operation key, a workflow status, or another fact the service recognizes. When reads are eventually consistent, wait for the documented window or keep the state unknown. An immediate empty read is not always proof of absence.

Keep the order strict:

1. Create or recover the logical action record.
2. Send the tool call with the same idempotency key when the service supports it.
3. Store the receipt if the response arrives.
4. If the response is lost, query the postcondition with actionId or externalId.
5. On confirmed, continue. On not_found, retry under policy. On unknown, do not invent a result.

type Verification =
  | { state: "confirmed"; externalId?: string }
  | { state: "not_found" }
  | { state: "unknown"; reason: string };

async function retryIfAbsent(actionId: string): Promise<Verification> {
  const action = await actionStore.get(actionId);
  if (!action) throw new Error("action was not prepared");

  const observed = await verifyPostcondition(action);

  if (observed.state === "confirmed") {
    await actionStore.markConfirmed(actionId, observed.externalId);
    return observed;
  }

  if (observed.state === "unknown") {
    await actionStore.markUnknown(actionId, observed.reason);
    return observed;
  }

  const result = await executeTool({
    actionId,
    idempotencyKey: actionId,
    inputHash: action.inputHash,
  });

  await actionStore.saveReceipt(actionId, result.receiptId);
  return { state: "confirmed", externalId: result.externalId };
}

This is a teaching boundary. In production, verification and dispatch can race another worker, and the store can change between the two steps. The remote operation must accept the key, or the executor needs ownership and reconciliation that cover the race.

This article does not claim a private benchmark. The practical test is simple: if the team cannot point to the external fact that turns unknown into confirmed, its retry still depends on an assumption.

When does an idempotency key solve the problem?

The key solves deduplication when the receiving service includes it in its contract and persists the result associated with it. The same key must represent the same logical intent. Do not create a new key just because the model was consulted again.

The service also needs a rule for the same key arriving with different arguments. A safe policy rejects the conflict or returns the original result. Treating the second payload as a new intent defeats the key.

AWS guidance says services and consumers should pass the token to downstream services and avoid repeating an effect while processing the same message (AWS Well-Architected Framework, 2026). This is a responsibility at each boundary, not an automatic guarantee from the agent.

Without idempotency support, verification can still reduce duplicates. A race remains between a not_found query and two concurrent attempts. Serialize action ownership, use a unique constraint where the write lands, or leave the state unknown for reconciliation. Do not promise exactly-once execution when the infrastructure does not provide that semantic.

How should you test timeouts, stale reads, and partial effects?

Test the contract with a fake executor that separates dispatch from the moment the state becomes visible. The important case is not only an exception before the call. It is an interruption after dispatch and before the receipt.

Observe the domain effect, not only the number of calls to a mock. If a tool creates a row and publishes a message, verify both facts or document which one is the primary postcondition. A mock that always returns "ok" does not exercise the boundary that causes the failure.

The 2026 study used above evaluated its wrapper in a simulated environment with injected failures and different tool tasks. It is evidence that the technique can be tested, not proof that every architecture will get the same rate. Use the pattern with the database, queue, or API your agent actually calls.

What should you log when the effect stays unknown?

Make unknown an operationally visible state. Record the reason, the last query time, the action key, the effect type, and the next permitted action. An operator should find the case without asking the model what it thinks happened.

The OpenAI Agents SDK, "Tools" documents per-tool timeouts, error results, and execution metadata. Those features bound local execution. They do not confirm a change in an external service. The application must combine the tool event with a receipt or postcondition read.

Do not hide the state in a conversation summary. The transcript may say that the agent requested create_invoice, but it does not prove that an invoice exists. An investigation needs the actionId, argument hash, observed status, external ID, and attempts. Remove secrets and personal data from logs.

Good agent observability asks more than "which tool was called?" It asks "which effect was proven, by what evidence, and which action stayed blocked while the state was unknown?" That connects the runtime to the system that can actually cause harm.

Checklist for adding verification to an agent runtime

Use this sequence when adding a tool with an external effect:

1. Classify the tool as a read, reversible write, irreversible write, or asynchronous operation.
2. Define the actionId in the runtime before the model call or at the first deterministic intent boundary.
3. Persist the argument hash and reject the same key with different input.
4. Define the postcondition in external state, not in model-generated text.
5. Make the executor return a receipt with status and external ID when available.
6. Model confirmed, not_found, unknown, and rejected separately.
7. Query the effect after a timeout, cancellation, or lost connection.
8. Retry only with a key the service actually deduplicates, or after a trustworthy absence check.
9. Test stale reads, concurrency, partial effects, and input conflicts.
10. Show unknown in observability and assign ownership for reconciliation.

To persist this state across processes, see how to pause and resume an AI agent without restarting. To define the boundary between a model attempt and a domain action, see validating tool calls in TypeScript.

Frequently Asked Questions

Does a timeout mean I should retry the tool call?

No. A timeout proves only that the client did not receive a response in time. Query the postcondition with the action identifier or use the service's idempotency key. If the query cannot decide, keep unknown and do not create a second action by assumption.

Does HTTP 200 prove that the effect happened?

Not necessarily. 200 may confirm that a service accepted an asynchronous command, not that processing finished. Define the receipt or external state that represents the postcondition. Mark confirmed only when that evidence is linked to the right action.

Can I use the provider's tool call ID as the action key?

Use it to trace the provider message, but do not make it the only domain key. A retry or fallback can create a different provider ID for the same intent. Create a runtime key and pass it to the executor when the service contract allows it.

What if the API has no idempotency support?

Query the effect before retrying, serialize action ownership, and keep unknown when the query is not trustworthy. A race still exists without receiver-side deduplication. State the real guarantee instead of promising exactly-once execution.

Conclusion

A tool call does not end when the model receives a response. For side effects, the runtime must distinguish transport acknowledgement from external state. An actionId, a receipt, and a postcondition make that difference verifiable.

The safe path is short: record intent, send with a stable key, store the result, query after a lost response, and retry only when absence is proven or the API deduplicates the operation. When none of that is possible, unknown is an honest state. It can pause the flow, but it does not create a silent duplicate.

Production note

Samuel Fajreldines is the editorial owner of this article. The research combined a recent primary publication, official OpenAI Agents SDK documentation, and AWS reliability guidance. The state model, decision tables, and fixtures are original synthesis. The code is illustrative and was not run against a provider or production system. AI assistance supported discovery, source comparison, drafting, image creation, translation, and consistency review. It did not provide production testing or first-hand experience. I use RemoteCode, my own tool, to keep long agent sessions visible; that is not a measurement from this article.

Sources consulted

• arXiv, "Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures", retrieved 2026-09-10.
• OpenAI Agents SDK, "Tools", retrieved 2026-09-10.
• OpenAI Agents SDK, "Results", retrieved 2026-09-10.
• AWS Well-Architected Framework, "Make mutating operations idempotent", retrieved 2026-09-10.