NotesSystem
Envoy
Also: Envoy Proxy
A high-performance L4/L7 proxy, originally built at Lyft, used as an edge proxy, a sidecar and the data plane of many service meshes (Istio, Consul). Its defining choice is that everything (pool, routes, listeners) is configured dynamically by a control plane over the xDS APIs.
Also in: Load Balancing › In the real world
Design choices
- Service Discovery: the pool arrives over xDS (EDS for endpoints), so a control plane can update thousands of proxies without restarts.
- Load Balancing Algorithms: round robin, least request, random, ring hash (Ketama-style consistent hashing) and Maglev (table size 65,537 by default), plus client-side weighted round robin driven by server load reports.
- Power of Two Choices: least request samples 2 hosts by default when weights are equal.
- Health Check: active checks plus outlier detection (5 consecutive 5xx eject a host for 30 s, growing on repeats, at most 10% of hosts by default), and a 50% panic threshold below which it ignores health.
- L4 vs L7 Load Balancing: it runs as either, with HTTP/2 and gRPC understood natively, so it balances per request.
Notable trade-offs
- Huge configuration surface, which is powerful and easy to get subtly wrong. In practice it wants a control plane (Istio, Envoy Gateway, a custom xDS server) rather than hand-written config.
- A sidecar per instance costs memory and CPU, and adds a hop on every call.