Notestransactions🌱 seedling

Transactional Outbox

Also: Outbox Pattern · Outbox · Dual-Write Problem · Dual Write · Inbox Pattern · Idempotent Consumer

To update a database and publish an event atomically, insert the event into an outbox table in the same local transaction as the change, and let a separate relay publish outbox rows to the broker. Delivery is at least once, so consumers must be idempotent.

Why it exists

The dual-write problem: writing to a database and a broker are two operations with no shared transaction. Commit then crash loses the event; publish then fail to commit emits an event for something that never happened.

How it works

  • Business row + outbox row commit in one local transaction: the event exists if and only if the change does.
  • A relay publishes outbox rows: by polling the table, or by tailing the database log (Change Data Capture).
  • The relay can crash after publishing but before recording progress, so it republishes: at-least-once.
  • Consumers dedupe by event ID, recorded in their own transaction (an inbox table).
  • Key events by aggregate ID so all events for one entity stay ordered in one partition.

Trade-offs

Gains Costs
atomic "write + publish" with no coordinator a relay to run and monitor; publish lag
no broker call inside the transaction duplicates: every consumer must be idempotent
ordering per aggregate outbox table growth (clean up, or delete right after insert when using CDC)

In practice

  • Debezium's outbox event router routes rows by aggregatetype to topics, uses aggregateid as the message key, and puts the event id in a header for deduplication.

Pitfalls

  • Assuming exactly-once: exactly-once effects come from idempotent consumers, not from the outbox.
  • Publishing directly from application code "after commit": a crash in between still loses the event.

Recall

These cards are also in your review deck, scheduled with spaced repetition.