Notesfoundations🌱 seedling
Timeout
Also: Deadline · Deadline Propagation · Request Timeout
A timeout bounds how long a caller waits for a remote call before treating it as failed. A deadline is the same budget set once at the edge and propagated, so every hop gives up when the original caller does.
Why it exists
A remote call can hang for many reasons (paused process, lost packet, held lock). Without a bound, the caller holds a thread, a connection and memory indefinitely; enough stuck calls exhaust the caller too, and a slow dependency becomes a Cascading Failure. Slow is more dangerous than dead: dead fails fast.
How it works
- Pick the value from data: choose an acceptable false-timeout rate (say 0.1%) and set the timeout at the matching percentile of the dependency (p99.9), plus padding for network variance.
- Propagate a deadline: the edge sets an absolute budget; each hop spends some and forwards only the remainder. Sending remaining time rather than a wall-clock time avoids clock skew.
- The server must stop too: a cancelled request should abort its own downstream work, or the deadline only saved the caller.
- Many clients ship with no default timeout (gRPC sets no deadline unless you do), so the bug is invisible until a dependency slows down.
Trade-offs
| Too long | Too short |
|---|---|
| stuck calls tie up threads and connections | abandons calls that would have succeeded |
| failures surface slowly | false failures trigger retries (Retry with Backoff), adding load |
In practice
- gRPC: deadlines are first-class; the client gets
DEADLINE_EXCEEDED, the server sees the call cancelled. Java and Go propagate deadlines to outgoing calls automatically. - Load shedding: a server that knows a request's deadline can drop it before doing any work if it has already waited too long in a queue (Load Shedding).
Pitfalls
- Independent per-hop timeouts longer than the caller's: servers keep working for clients that already left.
- One global timeout for every call, regardless of each dependency's latency profile.
Connections
- Part of
- Foundations
- Learn first
- Latency Percentiles
- Builds on this
- Circuit BreakerFailure DetectionRetry with Backoff
- Related
- Cascading FailureFLP ImpossibilityFailoverLoad SheddingRPCTail Latency AmplificationUnreliable Network
- In real systems
- gRPC
Recall
These cards are also in your review deck, scheduled with spaced repetition.