Notesdistributed🌱 seedling
Leader Election
Also: Master Election · Electing a Leader · Election Timeout
Choosing exactly one node to lead, such that every node agrees and two leaders can never act in the same term. Done properly it is consensus: a majority vote per numbered term.
Why it exists
Leader-based systems need a single writer. Promoting by timeout alone, or by script, can elect two leaders under a partition (Split Brain).
How it works
- Terms: every election increments a term; a node seeing a higher term steps down. Old-term messages are rejected.
- One vote per term, granted to the first eligible candidate; a majority wins. Two majorities can't coexist, so one leader per term.
- Randomized timeouts stop candidates colliding (Raft's split votes).
- Eligibility: in Raft, only candidates with up-to-date logs win, so no committed data is lost.
- PreVote (ask "could I win?" first) and CheckQuorum (leader steps down without a majority) reduce disruption.
- Outsourced: applications elect a leader via a Coordination Service by creating a key with a Lease; the next contender watches it.
Trade-offs
| Choice | Gains | Costs |
|---|---|---|
| short election timeout | fast failover | spurious elections under load |
| long election timeout | stable leadership | longer write outages |
| leader leases | local reads | failover waits for the old lease |
In practice
- etcd: 1 s election timeout by default; guidance is at least 10× the member round trip.
- Cloudflare (Nov 2020): a partially failed switch left etcd members with conflicting views; elections kept tying and writes were blocked.
- Elasticsearch 7.0 (2019) replaced manually configured
minimum_master_nodeswith a self-managed voting configuration.
Pitfalls
- Being elected is not proof of being leader later: a paused leader must still be fenced with a Fencing Token.
- Partial connectivity (A sees B, B sees C, A can't see C) causes leader churn even when every node is up.
Connections
- Part of
- Consensus
- Learn first
- QuorumFailure Detection
Recall
These cards are also in your review deck, scheduled with spaced repetition.