A worker can receive SIGTERM while it still has a task in memory. If it treats the signal as a simple process.exit(), it can lose a checkpoint, acknowledge a message too early, or leave an external write in an uncertain state.
In Cloud Run, the safe response is a controlled handoff. Stop accepting new tasks, let the active unit reach a safe point, persist progress in durable storage, and let another instance continue. The Cloud Run comparison of Services, Jobs, and worker pools helps choose the resource before implementing this cycle.
Shutdown contract
- SIGTERM changes the process state to “stopping”.
- The loop stops fetching new work.
- The active task writes its checkpoint outside the container.
- The queue is acknowledged after durable state is confirmed.
What does Cloud Run guarantee when it sends SIGTERM?
In 2026, Google Cloud documentation says Cloud Run sends SIGTERM before SIGKILL while an instance shuts down and describes a 10-second window for the process to exit (Google Cloud, “Container runtime contract”). Treat that window as an operational limit. It does not turn an incomplete write into a transaction.
Cloud Run instances are disposable. A container's writable filesystem is an in-memory layer, so it is not a place to keep the checkpoint the next instance needs. Progress should go to a database, queue, Cloud Storage object, or another system that belongs to the application's contract (Google Cloud, “What is Cloud Run”).
In Node.js, a signal listener receives the signal name as an argument. Once you install a SIGTERM listener, your code takes responsibility for starting shutdown and the default behavior that would terminate the process is removed (Node.js, “Process”). The listener should therefore only change state and stop waiting for new work. The main loop should do the rest.
Operational rule: use SIGTERM to stop intake, not to invent a confirmation. A task marked “processing” must remain recoverable if the process dies before its checkpoint.
How do you turn SIGTERM into a safe handoff?
In 2026, Cloud Run's execution-environment documentation recommends a SIGTERM handler and mentions cleanup tasks such as flushing logs during the shutdown window (Google Cloud, “Select an execution environment for services”). For a worker, cleanup also includes closing the intake boundary and persisting work that has already been accepted.
The cycle has four parts:
- Stop intake. Mark the process as
stoppingand cancel only the queue polling or blocking wait. Do not claim new work after the signal. - Drain the active unit. Let the current task reach a point that can be repeated safely. If it cannot finish within the limit, leave it unacknowledged so the queue or scheduler can retry it.
- Persist state. Store the task identity, completed step, and input version outside the process. A local file or global variable is not enough.
- Confirm afterward. Acknowledge the message only after durable storage confirms the write. The next instance can then find the same key and decide whether to resume, reuse the output, or run the remaining step.
This design separates process life from work life. A worker may die; the task cannot depend on that worker's memory surviving. If the process runs an AI agent, the same rule applies to tool calls that create external effects. RemoteCode is an example of the author's tool category where state and evidence must leave the local loop when execution moves beyond the workspace.
When that checkpoint must survive failures, approvals, and multiple steps, the explanation of durable execution for AI agents goes deeper on the difference between a queue, a checkpoint, and a workflow.
How do you implement shutdown in a TypeScript worker?
The example below shows the loop skeleton. claimNext, processOne, saveCheckpoint, and ack are illustrative adapters. They must provide a lease, conditional writes, and an acknowledgement compatible with your queue.
import process from "node:process";
type WorkItem = { id: string; inputVersion: string };
let stopping = false;
const stopPolling = new AbortController();
const active = new Set<Promise<void>>();
process.once("SIGTERM", () => {
stopping = true;
stopPolling.abort();
console.log(JSON.stringify({ event: "shutdown_requested" }));
});
async function workerLoop() {
while (!stopping) {
const item = await claimNext({ signal: stopPolling.signal });
if (!item) break;
const run = runItem(item);
active.add(run);
void run.finally(() => active.delete(run));
}
await Promise.all(active);
console.log(JSON.stringify({ event: "shutdown_complete" }));
}
async function runItem(item: WorkItem) {
const result = await processOne(item);
await saveCheckpoint({
taskId: item.id,
inputVersion: item.inputVersion,
result,
});
await ack(item.id);
}
void workerLoop().catch((error) => {
console.error(JSON.stringify({ event: "worker_failed", error: String(error) }));
process.exitCode = 1;
});The important part is not the Set itself. It is the order. The handler blocks new claims, active represents accepted work, and saveCheckpoint happens before ack. If processOne fails or the process dies between the last two operations, the message remains recoverable. The write must be idempotent or conditional so a retry does not create a second output.
Do not use process.exit() to shorten the path. Cloud Run's best-practices documentation warns that background activity after an invocation ends may stop receiving CPU and can cause reset connections (Google Cloud, “Functions best practices”). Let the main function return after waiting for work that can still finish safely.
Where does this cycle fit: Service, Job, or worker pool?
In 2026, the Cloud Run overview separates Service, Job, and worker pool by execution model: a Service handles HTTP and events, a Job runs tasks to completion, and a worker pool processes continuous pull-based work (Google Cloud, “What is Cloud Run”). The SIGTERM handler is useful in all three, but the work boundary changes.
Service
Use a Service when a request or event starts the work. The platform gives in-flight requests time to finish when an instance must shut down, but your code still needs to persist state and avoid activity that depends on the instance after the response. For an agent, return a task identifier and move long processing to a layer with its own recovery policy.
Job
Use a Job when each execution has defined input and should end. A timeout or cancellation can also send SIGTERM to the task. The article about avoiding duplicate output on Cloud Run Job retries covers the next layer: deterministic keys, checkpoints, and idempotent writes. The handler prepares the output; the Job design decides how the task will be retried.
Worker pool
Use a worker pool when the container continuously consumes a queue without an HTTP endpoint. Current documentation says worker pools do not autoscale from queue depth and that active instances continue to be billed (Google Cloud, “Container runtime contract”). The worker must stop polling, release or renew the active message lease, and let the next instance take over anything that was not confirmed.
How do you verify SIGTERM before deployment?
In 2026, Google Cloud documents a local Docker test: start the image, send SIGTERM to the container, and observe whether the process exits correctly (Google Cloud, “Select an execution environment for services”). The test should verify the handoff, not just the final log line.
docker run --rm --name worker-under-test worker-image:dev
docker kill --signal=SIGTERM worker-under-testAdd an artificially slow task and make the test confirm four events: shutdown_requested, no new claim after the signal, a persisted checkpoint, and shutdown_complete. Then start a second worker with the same store. It should find the task in a recoverable state and produce one output for the same key.
In the deployed environment, correlate logs by taskId, inputVersion, and instance identifier. Check what happened before and after the signal. A task may have completed an external effect and died before ack. That is the window an idempotency key and conditional write must cover.
Useful verification: a green deployment proves that the container started. It does not prove that a task survived an instance replacement. The recovery test should stop the process at different points in the step.
Which mistakes make shutdown lose work?
The first mistake is acknowledging the queue before the checkpoint. If the process dies after ack, the queue has no reason to deliver the task again, even though the application never persisted the result.
The second is cancelling the active task without defining its resume policy. Aborting an external call may be correct for an operation with no side effect, but it is dangerous for a write that has already started. Mark the step uncertain, inspect the state through its idempotency key, and then decide whether to repeat it.
The third is writing only to local disk. The container's writable layer is disposable and may disappear with the instance. The final output, checkpoint, and lease information must live somewhere another instance can read.
The fourth is treating 10 seconds as a guarantee. Google Cloud also documents forced termination when a container exceeds its memory limit. SIGKILL cannot be caught by Node.js. The system must therefore remain safe when the handler cannot finish.
The fifth is hiding the process behind a shell that does not forward signals. Prefer an entrypoint that runs the main process directly, such as ENTRYPOINT ["node", "dist/worker.js"], and verify the image locally. If a script manages subprocesses, it must forward SIGTERM too.
Frequently asked questions about SIGTERM in Cloud Run
Does SIGTERM guarantee that the current task will finish?
No. In 2026, Cloud Run documentation describes a 10-second window before SIGKILL, but a task can exceed the limit, hit a memory failure, or stop before its checkpoint. The task must remain recoverable through the queue, database, or another durable state mechanism.
Should you call process.exit() in the handler?
Not as the first response. A SIGTERM listener removes Node.js's default termination behavior. Let the loop stop intake, wait for safe work, and set process.exitCode only for failures the application could not handle. The main function should finish naturally.
What changes for a worker pool?
A worker pool does not receive an HTTP request that marks the end of a work unit. The loop must stop pulling, finish or return the active item, and persist its checkpoint before leaving. Since the pool does not autoscale from queue depth, capacity and recovery also need to be observable.
Sources consulted
- Google Cloud, “Container runtime contract”, retrieved 2026-08-12. URL:
https://docs.cloud.google.com/run/docs/container-contract - Google Cloud, “Select an execution environment for services”, retrieved 2026-08-12. URL:
https://docs.cloud.google.com/run/docs/configuring/execution-environments?hl=en - Google Cloud, “What is Cloud Run”, retrieved 2026-08-12. URL:
https://docs.cloud.google.com/run/docs/overview/what-is-cloud-run - Google Cloud, “Functions best practices”, retrieved 2026-08-12. URL:
https://docs.cloud.google.com/run/docs/tips/functions-best-practices?hl=en - Node.js, “Process”, retrieved 2026-08-12. URL:
https://nodejs.org/api/process.html - Google Cloud, “Graceful shutdowns on Cloud Run: Deep dive”, retrieved 2026-08-12. URL:
https://cloud.google.com/blog/topics/developers-practitioners/graceful-shutdowns-cloud-run-deep-dive
