Transactional Outbox: How to Publish Events Without Losing Them

The order was saved, but the event that should reserve inventory never reached the broker. The process died between COMMIT and the publish call. Reverse the order and you can announce an order that later fails in the database.

The transactional outbox pattern closes that gap by writing the business change and the event in one local transaction. After the commit, a relay reads pending outbox rows and publishes them. Publication can still fail or happen twice, so the design needs retries, observability, and an idempotent consumer.

This is a spoke of durable execution for AI agents with checkpoints, queues, and workflows. The pattern also applies to ordinary APIs and services. In an agent runtime, the same boundary appears when a committed state transition must wake a task, emit an audit event, or start work outside the current process. The outbox does not make the external call part of the database transaction.

Key Takeaways

• Write the business record and the outbox message in the same database transaction.
• Publish after the commit through polling or CDC, keeping the row until the relay accepts the delivery.
• Treat duplicates as normal: the relay can crash after publishing and before recording success.
• Make consumers deduplicate with a stable ID, and preserve order only where the domain actually needs it.
• Prefer a simple local transaction when there is no second publication obligation.

The problem is a dual write

Consider this order API:

const order = await saveOrder(input);
await broker.publish("OrderPlaced", order);
return order;

If saveOrder commits and the process dies before publish, the database knows about the order, but inventory, email, or a downstream projection never receives the notification. Reverse the calls and the event may arrive while the record does not exist. A retry does not solve the problem by itself: a timeout can hide a publication that the broker already accepted.

AWS Prescriptive Guidance describes this as a write operation across two systems. Its solution is to persist the event with the update and use a separate process to send it. The Transactional Outbox pattern reference at microservices.io draws the same boundary: the local transaction confirms the obligation, and the relay handles communication afterward.

This is not a promise of exactly-once delivery. It prevents the obligation to publish from existing only in the process memory.

When is the outbox worth the cost?

Use the pattern when a committed local change must cause asynchronous work in another system and losing that notification would leave inconsistent state. For example:

The cost is real. You now operate a table, a relay, retries, retention, metrics, and a policy for messages that do not advance. If the operation can remain synchronous inside one database, adding a broker and a relay may add moving parts without solving an actual failure.

What needs to commit together?

The atomic unit should contain the state that downstream readers need and the obligation to publish the change. PostgreSQL's COMMIT documentation says that changes become visible to others and are guaranteed durable when the command succeeds (PostgreSQL COMMIT documentation). The exact shape depends on the database and driver, but the idea looks like this:

-- Illustrative shape. Adapt types, constraints, and the client to your database.
CREATE TABLE outbox_events (
  id uuid PRIMARY KEY,
  aggregate_type text NOT NULL,
  aggregate_id text NOT NULL,
  event_type text NOT NULL,
  payload jsonb NOT NULL,
  created_at timestamptz NOT NULL,
  published_at timestamptz,
  attempts integer NOT NULL DEFAULT 0,
  last_error text
);

BEGIN;

UPDATE orders
SET status = 'placed'
WHERE id = 'order-42';

INSERT INTO outbox_events
  (id, aggregate_type, aggregate_id, event_type, payload, created_at)
VALUES
  ('event-42', 'order', 'order-42', 'OrderPlaced', '{"orderId":"order-42"}', now());

COMMIT;

The UPDATE and the INSERT must fail or commit together. If the transaction rolls back, the relay should not find a ghost obligation. If it commits, the outbox row needs a stable ID, event type, aggregate key, payload, and enough information to diagnose attempts.

Do not accidentally write the outbox through a separate connection. That recreates the dual write: the order can commit while the event row fails, or the reverse. Pass the same transaction context to the code that saves the entity and the code that records the event.

The relay publishes after the commit

The relay is a worker, not an invisible part of the transaction. It finds eligible rows, claims a batch, publishes to the broker, and records the result. If publication fails, it keeps the row, records the error, and schedules another attempt. If the process dies after the broker accepts the message but before published_at is written, the same row may be tried again.

With several relays, the table needs a claim policy. In PostgreSQL, FOR UPDATE SKIP LOCKED can keep workers from waiting on the same rows, but the documentation warns that the result is not a consistent view and that this feature is suitable for queue-like tables, not general queries (PostgreSQL locking clauses). The claim also needs a lease or recovery path when a worker disappears during publication.

Two relay families are common:

1. Polling publisher: workers query the table on an interval or receive a wake-up signal. This is easy to understand, but it needs indexes, batch limits, backoff, and a policy that prevents the business database from becoming the bottleneck.
2. Change data capture: a tool captures the inserted row and forwards it to the broker. Debezium's Outbox Event Router, for example, captures changes from an outbox table and routes events using fields from the row. CDC reduces application polling, but adds configuration, operations, and a capture dependency.

Choose based on load, acceptable latency, operational capacity, required ordering, and recovery. There is no universal throughput number that tells you when to replace polling with CDC.

Duplicates and ordering remain design responsibilities

The relay should not delete a row before an accepted publication. Even after marking an event as published, the process can die between the broker and the database. That is why AWS recommends idempotent consumers for duplicate messages, and why the microservices.io pattern treats duplicate publication as a normal relay possibility.

The consumer can use the event ID in an inbox table with a unique constraint, or it can make the business operation idempotent. The important part is that the check and the effect cannot create another race. A separate SELECT followed by an INSERT does not protect you if two deliveries arrive together. Use a constraint or a transaction that fits the database.

Ordering also needs a scope. AWS warns that events should follow the order of the service's updates when the domain requires it. That does not mean every message across every aggregate needs a global queue. Choose an ordering key per aggregate, partition, or workflow, and document what a consumer does with a late event.

How to verify the contract before operating it

An outbox is not ready just because the table exists. Check the failure paths that cross the whole process:

1. Rollback: force a failure after inserting the entity and event. Neither row should remain committed.
2. Unavailable relay: confirm that local state remains recoverable and that the row stays when the broker refuses the connection.
3. Ambiguous timeout: simulate an accepted publication followed by a relay error. The consumer must tolerate the second delivery.
4. Two workers: make two relays contend for the same batch and confirm that claim, lease, and final update do not make an event disappear.
5. Per-aggregate order: write two transitions for the same order and check the order seen by the consumer. Test separate aggregates independently.
6. Poison message: record attempts, the error, and enough payload for triage. After a limit, move the message to an operable quarantine such as the dead-letter queue for AI agent tasks that keep failing.

This complements API idempotency tests without duplicate side effects and webhook tests with signatures, duplicates, and retries. They exercise different boundaries: the outbox protects the obligation to publish, while the consumer and API protect the effect of receiving or repeating a request.

What the outbox does not solve

The pattern does not make a database and a broker participate in one distributed transaction. It does not automatically undo an external effect that was accepted, make consumers fast, or remove the need for metrics, retention, and replay.

If several services must complete one business transaction, each service still has its own changes and failures. A saga can coordinate those steps with compensating actions, but that is a different decision from reliably publishing one local event.

For an agent, also record the run ID, action ID, and contract version that produced the event. I use RemoteCode as my own tool for this kind of workflow. That makes it possible to connect a committed change to the work the relay started without putting prompts or secrets into the integration payload.

Use this criterion: if losing the notification after the commit leaves a state that nobody can reconstruct, persist the publication obligation in the same transaction. Then treat the relay, duplicates, ordering, and recovery as visible parts of the system.

Sources

• AWS Prescriptive Guidance, "Transactional outbox pattern", retrieved October 31, 2026.
• microservices.io, "Pattern: Transactional outbox", retrieved October 31, 2026.
• PostgreSQL, "COMMIT", retrieved October 31, 2026.
• PostgreSQL, "SELECT: locking clause", retrieved October 31, 2026.
• Debezium, "Outbox Event Router", retrieved October 31, 2026.
• Microsoft Learn, "Idempotent Consumer pattern", retrieved October 31, 2026.