How it works
Start with the dual-write problem: an order service saves an order, then publishes OrderCreated. A crash after the database commit can leave an order that downstream services never hear about. Publishing first reverses the problem: consumers might receive an event for an order that never commits.
Instead, write both records in one local database transaction: insert the order and insert an outbox row containing an event ID, order ID, event type, schema version, and payload. Both writes commit or neither does. This requires the business data and outbox record to share a transactional boundary; a separate outbox database does not automatically provide that guarantee.
A relay publishes committed outbox events to the broker. It can poll pending rows in batches or use change data capture to read committed outbox inserts from the database log. Polling needs an index and safe coordination between workers; change data capture needs durable connector progress and sufficient log retention.
With polling, mark an event published only after the broker durably acknowledges it. If the relay crashes after publication but before recording success, it publishes the same event again after recovery. A change-data-capture relay has a similar replay window around publishing and saving its progress, so consumers must tolerate duplicate events.
Give each event a stable ID. A consumer updating its own database can insert that ID into a processed-events table and apply its business change in one transaction, with a uniqueness constraint preventing repeated effects. External calls, such as sending an email or charging a card, need their own idempotency support or reconciliation strategy.
If order matters, include a per-entity sequence or version and route related events to the same broker partition. The relay must also preserve that entity's publication order; a partition key alone cannot fix events published out of order. Monitor pending-event age, retry failures, and storage growth, and reclaim published rows only after they are safely handed off.
A worked example
Reliable dispatch from a job scheduler
Suppose a scheduler creates 200 due job occurrences per second. In one database transaction, it inserts occurrence 8142 with status ready and an outbox event with ID evt-8142. A relay polls every 100 milliseconds, publishes the event, and marks it published after broker acknowledgment. If the relay crashes after publication, evt-8142 may appear twice; the worker uses occurrence 8142 as the execution identity and applies retry-safe processing. During a five-minute broker outage, 60,000 events accumulate. After recovery, a relay sustaining 1,000 events per second drains that backlog in about 75 seconds while 200 new events per second continue arriving. External job side effects still need separate idempotency controls.
Trade-offs to say out loud
- Atomic intent, not atomic delivery
- The transaction guarantees that the business change and intent to publish commit together, but consumers see the event later.
- Duplicate handling remains necessary
- A successful publish followed by a crash can cause replay, so the outbox alone does not provide exactly-once downstream effects.
- Extra database work and storage
- Each event adds a database write, and polling, indexes, retention, and cleanup compete with application traffic.
- Relay complexity
- Polling is straightforward but adds query load and polling delay, while change data capture adds connector and log-retention operations.
- Ordering limits parallelism
- Preserving per-entity order requires coordinated publication and can limit throughput for especially busy entities.
- Outages become backlogs
- The service can keep committing during a broker outage only while database capacity and acceptable downstream delay allow it.
When to reach for it
- A successful database write must reliably trigger a notification, background job, feed update, or search-index change.
- The interviewer asks what happens if the service crashes between committing data and publishing a message.
- A workflow spans services, but each service can commit its own state and outgoing event in a local transaction.
- Downstream work may happen asynchronously, but silently losing the work is unacceptable.
How to bring it up in the interview
- Name the invariant first: every committed business transition must leave a durable event to publish. Draw the business row and outbox row inside the same transaction boundary.
- Walk through three failures: before commit, after commit but before publication, and after publication but before the relay records success. The last case explains why duplicates remain.
- Choose polling or change data capture based on latency, volume, and operational constraints rather than presenting either as universally better.
- Expect follow-up questions about consumer deduplication, per-entity ordering, relay worker coordination, backlog limits, and cleanup.
- Separate durable dispatch from external completion: an outbox can reliably request a payment or notification, but it cannot prove that an external provider performed the action exactly once.
Common mistakes
- Writing the outbox row in a separate transaction from the business change, which recreates the original failure window.
- Claiming that the pattern provides exactly-once delivery or makes consumer idempotency unnecessary.
- Marking an event published before receiving a durable broker acknowledgment, which can lose the event.
- Assuming timestamps or a shared broker partition automatically preserve order across concurrent relay workers.
- Ignoring poison events, old pending rows, and database growth during a prolonged broker outage.
Where it comes up
Interview questions where transactional outbox pattern is part of a strong answer, and why.
- Design a notification systemThe outbox closes the gap between saving a notification request and durably queuing its delivery.
- Design a news feed like FacebookCommitting a post with an outgoing event prevents a crash from silently skipping asynchronous feed fan-out.
- Design a distributed job schedulerThe database-to-queue handoff must preserve every committed job occurrence despite scheduler or relay crashes.
- Design a wire transfer systemA transfer state change and its outgoing bank-processing request need a durable atomic handoff before external execution.
- Design a price alert systemRecording an alert as triggered must atomically create durable notification work so one-shot alerts are not lost.
- Design a food delivery app like DoorDashOrder transitions need reliable outgoing events to coordinate restaurant, payment, and delivery workflows across failures.
- Design Facebook post searchPost changes and deletions need a reliable handoff to asynchronous indexing without losing updates after a database commit.
Questions people ask
Does the transactional outbox pattern guarantee exactly-once delivery?
No. It atomically records the business change and the intent to publish, but relay recovery can produce duplicate messages. Exactly-once database effects require consumer deduplication and the effect to commit together; external effects require additional support.
What is the difference between an outbox and change data capture?
An outbox explicitly records application events alongside business writes. Change data capture reads committed database changes and can serve as the outbox relay. Capturing ordinary table changes directly is another approach, but consumers must then interpret storage-level changes rather than an explicit event contract.
How is the outbox pattern different from two-phase commit?
Two-phase commit coordinates an atomic outcome across participating transactional systems. An outbox uses one local transaction and asynchronous publication, accepting downstream delay and possible duplicates. It avoids requiring a shared transaction across the database and broker, but does not make their state change simultaneously.