How to Use a Circuit Breaker for AI Agent Tools

An agent tool fails, the runtime tries again, and the dependency keeps returning a timeout or 503. The model sees another error, chooses the same tool, and the loop consumes time without gaining new information.

Retries and circuit breakers solve different problems. A retry repeats a failure that may be temporary. A circuit breaker stops calls when the signals suggest that the next attempt will probably fail too. In an agent, that decision belongs at the tool boundary, before the external call, not only in the prompt. The post about stopping an AI agent from looping forever covers the general problem; this article focuses on the failing dependency.

Short answer

• Keep a separate breaker for each dependency that can fail independently.
• Count only classified failures, such as timeouts, unavailability, and rate limits, not request-validation errors.
• In the open state, reject the call quickly and return a structured result the agent can explain or route.
• After the cooldown, allow one controlled probe in half-open.
• Close the circuit after a successful probe and open it again when the dependency keeps failing.

Retry and circuit breaker are different guards

A retry makes sense when another call has a reasonable chance of working. A circuit breaker is useful when the failure looks correlated: a service is unavailable, a queue is saturated, or an API is rejecting requests. The AWS Circuit breaker pattern guidance, checked on October 22, 2026, describes a breaker as a way to prevent repeated retries against a dependency that is already failing.

The guards can work together. A short retry can handle a transient error inside a protected call. If the classification says the dependency remains unavailable, the breaker should block the next attempt. The AWS Retry with backoff guidance, checked on October 22, 2026, also separates transient faults from cases where failing fast is better.

In an agent runtime, there is a third layer: the model may see the tool error and request the same action again. The breaker must therefore wrap the path that decides whether the tool can run. It should return a stable reason when the call is blocked, rather than acting as a hidden counter inside an HTTP client.

A retry asks, "Should I try once more?" A circuit breaker asks, "Should this dependency receive any call right now?" If those questions are mixed, the agent can interpret an open circuit as an ordinary temporary error and recreate the retry loop that the guard tried to stop.

What happens in each state?

The Microsoft Circuit Breaker pattern documentation, checked on October 22, 2026, models the breaker as a machine with closed, open, and half-open states. The model works for an agent tool, but the result also needs operational context that the model and the human caller can use.

The state should belong to a dependency, not to the whole agent. If search is down, that does not mean the database or a file-reading tool is unavailable too. A global breaker turns a localized failure into an unnecessary outage for the whole agent.

Also decide who can run the probe. One controlled request gives the dependency room to recover. Releasing a whole batch when the cooldown ends recreates the spike exactly as the service is coming back.

Which failures should open the circuit?

Do not open the circuit for every exception. A 400 caused by invalid arguments, an authorization rule, or input rejected by the schema does not prove that the dependency is unhealthy. Counting it as an outage can block a healthy tool and hide a problem in the call contract.

Classify the failure before updating the breaker. Timeouts, connection failures, 429, and 5xx responses may be candidates, depending on the provider contract. Authentication errors, missing resources, and invalid arguments usually need correction or escalation, not an automatic probe. The classification should be specific to the dependency and documented with the tool.

The failure count and cooldown in the example below are not universal recommendations. They are parameters in a local fixture. A slow API, a work queue, and a model provider recover on different timelines. Measure the dependency before choosing a policy, and start with a configuration that can change without a redeploy.

How do you implement a circuit breaker for a tool?

A small implementation is easier to review when it receives the clock by injection. The test can then advance time without sleep, and the transition to half-open does not depend on machine speed.

The wrapper below protects one operation at a time, allows one probe, and keeps the decision outside the model. The shouldCount method represents application-specific classification. In production, also record the dependency, runId, transition, and reason without sending secrets or the full tool result to logs.

type CircuitState = "closed" | "open" | "half-open";

type BreakerOptions = {
  failureThreshold: number;
  cooldownMs: number;
  now?: () => number;
  shouldCount: (error: unknown) => boolean;
};

export class CircuitOpenError extends Error {
  constructor(public readonly retryAfterMs: number) {
    super("tool dependency is temporarily unavailable");
    this.name = "CircuitOpenError";
  }
}

export class ToolCircuit {
  private state: CircuitState = "closed";
  private failures = 0;
  private openedAt = 0;
  private probeInFlight = false;

  constructor(private readonly options: BreakerOptions) {}

  getState(): CircuitState {
    if (this.state === "open" && this.remainingCooldown() === 0) {
      this.state = "half-open";
    }
    return this.state;
  }

  async run<T>(operation: () => Promise<T>): Promise<T> {
    const state = this.getState();

    if (state === "open") {
      throw new CircuitOpenError(this.remainingCooldown());
    }

    if (state === "half-open") {
      if (this.probeInFlight) {
        throw new CircuitOpenError(this.options.cooldownMs);
      }
      this.probeInFlight = true;
    }

    try {
      const result = await operation();
      this.failures = 0;
      this.state = "closed";
      return result;
    } catch (error) {
      if (this.options.shouldCount(error)) {
        this.failures += 1;
        if (state === "half-open" || this.failures >= this.options.failureThreshold) {
          this.state = "open";
          this.openedAt = this.options.now?.() ?? Date.now();
        }
      }
      throw error;
    } finally {
      if (state === "half-open") {
        this.probeInFlight = false;
      }
    }
  }

  private remainingCooldown(): number {
    if (this.state !== "open") return 0;
    const now = this.options.now?.() ?? Date.now();
    return Math.max(0, this.options.cooldownMs - (now - this.openedAt));
  }
}

The method does not retry and does not call the model. It only decides whether the external operation may start. That separation makes the policy observable and prevents a hidden catch inside the tool from making the agent believe the call ran when it was rejected before the effect.

How do you return a fallback the agent can handle?

A breaker error should not look like an argument error. The executor can turn CircuitOpenError into a structured result with a temporary state, dependency name, and permitted next action. The model-facing message should say that the tool is unavailable, not suggest an immediate retry.

type ToolResult =
  | { status: "ok"; value: unknown }
  | { status: "blocked"; reason: "dependency_open"; retryAfterMs: number };

declare function searchService(query: string): Promise<unknown>;

async function callSearchTool(circuit: ToolCircuit, query: string): Promise<ToolResult> {
  try {
    const value = await circuit.run(() => searchService(query));
    return { status: "ok", value };
  } catch (error) {
    if (error instanceof CircuitOpenError) {
      return {
        status: "blocked",
        reason: "dependency_open",
        retryAfterMs: error.retryAfterMs,
      };
    }
    throw error;
  }
}

The fallback can ask the agent to answer with what it already knows, use an independent tool, or route the task. It should not promise that search will run again immediately. If the operation is required to continue, persist the state and resume later with an explicit policy. The guide to durable execution for AI agents covers that checkpoint and resume boundary.

How do you test the transitions without real waiting?

The main test does not need a model provider or a real API. Use a fake operation that fails in a controlled way and a clock the test can advance. The goal is to prove the breaker contract, not the availability of the dependency.

The code below is an illustrative TypeScript-compatible fixture, not a production-ready client.

import { strict as assert } from "node:assert";

const clock = { value: 0 };
const counted = (error: unknown) => error instanceof Error && error.message === "down";
const circuit = new ToolCircuit({
  failureThreshold: 2,
  cooldownMs: 100,
  now: () => clock.value,
  shouldCount: counted,
});

const down = async () => { throw new Error("down"); };

await assert.rejects(() => circuit.run(down));
await assert.rejects(() => circuit.run(down));
assert.equal(circuit.getState(), "open");

let calls = 0;
await assert.rejects(() => circuit.run(async () => {
  calls += 1;
  return "should not run";
}));
assert.equal(calls, 0);

clock.value = 100;
assert.equal(circuit.getState(), "half-open");
assert.equal(await circuit.run(async () => "recovered"), "recovered");
assert.equal(circuit.getState(), "closed");

A production test suite should add four cases: an unclassified error does not open the circuit, two concurrent probes do not pass together, a failed probe returns to open, and a success clears the failure sequence. If the runtime has several worker instances, the state must live in shared storage or each instance will hold a different view of the dependency.

How do you combine a breaker with retry, timeout, and budget?

These controls are not substitutes. A timeout ends a call that took too long. A retry decides whether a transient failure deserves another attempt. A breaker decides whether the dependency should receive the next call. A budget limits how much a run can spend on tokens, time, or tools. The post about cancelling an AI agent when a tool hangs covers timeouts and cancellation; the post about avoiding duplicate tool calls during LLM failover covers repetition during failover.

A practical order is to apply the run deadline, check the breaker, run the tool with a timeout, classify the failure, and only then decide whether a short retry fits. When the breaker opens, retry must stop. When the budget ends, record stop_reason and do not bypass the guard by calling the same tool through another path.

Record state changes and blocked calls. The guide to tracing AI agent tool calls with OpenTelemetry helps connect the breaker event to the runId and dependency. Without that trail, the team sees only an incomplete answer and cannot tell whether the service failed, the circuit opened, or the model kept trying after the opening.

Circuit breaker checklist before release

• Does the breaker protect one dependency instead of the entire agent?
• Does classification separate timeouts, rate limits, 5xx, authentication, validation, and missing resources?
• Is the call rejected before any external effect when the state is open?
• Does half-open allow one controlled probe without releasing a whole batch?
• Does the model receive a result that explains the outage without requesting an immediate retry?
• Does the test use a fake clock and prove success, failure, cooldown, and concurrency?
• Is shared state consistent when more than one worker runs?
• Do logs record dependency, transition, reason, runId, and duration without secrets?
• Does the fallback define an action: answer, use an alternative, pause, or route?

Do not choose the threshold because an online example used the same number. Observe the dependency behind the tool, classify the failures that actually indicate unavailability, and test the transition before allowing important effects. The breaker should make failure smaller and more legible, not turn every error into silence.

How this article was produced

Samuel Fajreldines is the responsible author. The research compared current Microsoft and AWS guidance on circuit breakers and retries with agent documentation and recent public discussions. Discussions were used only for language and recurring failure modes. An equivalent JavaScript fixture was run locally with a fake clock and an in-memory operation; the TypeScript code above is an illustrative adaptation and did not call a model, provider, or external service.

I used RemoteCode as the author's tool for preserving context between sessions. It supported research and drafting, but it did not run the fixture or provide production evidence.

Sources consulted

• Microsoft Learn, "Circuit Breaker pattern", checked 2026-10-22
• AWS Prescriptive Guidance, "Circuit breaker pattern", checked 2026-10-22
• AWS Prescriptive Guidance, "Retry with backoff pattern", checked 2026-10-22
• Microsoft Learn, "Develop an agentic RAG solution with Azure", checked 2026-10-22