Notesstream🌱 seedling

Unbundling the Database

Also: Database Inside-Out · Turning the Database Inside Out · Log-Centric Architecture · Unbundled Database

A database already derives tables, indexes, views, caches and replicas from its own log. Unbundling pulls that log out into a shared system and builds each derived structure as a separate, specialised store kept current by stream processors.

Why it exists

No single product is best at search, analytics, caching and transactions at once. Composing several stores needs one mechanism to keep them consistent, and the database's own log-and-derive design is the proven one.

How it works

  • The log (Kafka, or a database WAL exposed via Change Data Capture) is the shared, ordered write path and the effective system of record.
  • Each store (Postgres, Elasticsearch, Redis, a warehouse) follows the log at its own offset.
  • Stream processors play the role of the database's internal index and view maintenance code: triggers that work across systems.
  • Stream–table duality: a table is the latest value per key of a changelog stream, and a stream is a table's history.
  • New views are added the way CREATE INDEX works: backfill from a snapshot or a replay, then follow the live log.

Trade-offs

Gains Costs
each read pattern gets the best tool no transaction spanning the stores
loose coupling: stores fail and lag independently reads across stores aren't one snapshot
new views by replay, without touching writers more systems to run and monitor

In practice

  • Federated systems (Trino, foreign data wrappers) unify reads over many stores. Unbundling unifies writes through one log. Large platforms often use both.
  • Streaming databases (Materialize, RisingWave) re-bundle the idea: SQL views maintained from Kafka or CDC streams.

Pitfalls

  • Unbundling a system that one database serves well: more moving parts for no gain.
  • Assuming the stores are mutually consistent at any instant. They converge; they aren't in lockstep.

Recall

These cards are also in your review deck, scheduled with spaced repetition.