What a load balancer does
A load balancer sits in front of a pool of servers and spreads requests across them. It also runs health checks and stops sending traffic to servers that fail them, which lets you deploy, scale and survive crashes without users noticing.
| Layer 4 (transport) | Layer 7 (application) | |
|---|---|---|
| Sees | IP addresses and ports | HTTP method, path, headers, cookies |
| Can route by | Connection | URL path, host, header, user |
| Speed | Very fast, little processing | More work per request |
| Extras | — | TLS termination, compression, caching, auth, rate limiting |
Routing algorithms
| Algorithm | How it picks | Good for |
|---|---|---|
| Round robin | Next server in turn | Identical servers, similar requests |
| Weighted round robin | In turn, in proportion to weights | Mixed server sizes |
| Least connections | Server with fewest open connections | Requests of very different lengths |
| Least response time | Fastest recent responses | Uneven or noisy servers |
| IP / key hash | hash(client or key) mod N | Affinity, such as keeping a user on one cache shard |
| Consistent hashing | Position on a hash ring | Affinity that survives adding or removing servers |
| Power of two choices | Pick two at random, take the less loaded | Large pools; near-optimal with little state |
Plain hash mod N remaps almost every key when N changes. Consistent hashing moves only about 1/N of the keys; see the sharding module.
Avoiding a single point of failure
A single load balancer is itself a single point of failure. Common fixes: an active-passive pair sharing a virtual IP (the passive one takes over on a missed heartbeat), several active balancers behind DNS, or a managed cloud balancer that's redundant across availability zones.
Global traffic is usually split first by DNS (GeoDNS or anycast) to the nearest region, then by regional load balancers to servers.
A request through a load balancer
The client resolves the domain. DNS returns the address of the load balancer in the nearest region.
See it animated
Interview questions
Try answering out loud first, then open the model answer.
Q1What's the difference between an L4 and an L7 load balancer?intermediate
L4 balancers route TCP/UDP connections using IPs and ports, without looking at the payload: fast and protocol-agnostic. L7 balancers understand HTTP, so they can route by path or header, terminate TLS, and add features like caching and rate limiting, at more cost per request.
Q2Which algorithm would you use when some requests take 10 ms and others 10 s?intermediate
Least connections (or least outstanding requests). Round robin would keep sending new work to a server stuck on long requests; counting open connections approximates real load.
Q3How does a load balancer know a server is unhealthy?beginner
Health checks: it periodically calls an endpoint (or opens a connection) and marks a server down after several failures, and up again after several successes. Passive checks also watch real traffic for errors and timeouts.
Q4How do you make the load balancer highly available?intermediate
Run at least two. Active-passive pairs share a virtual IP that moves on failure; active-active setups put several balancers behind DNS or anycast. Cloud balancers do this for you across zones.
Quick check
0 of 3 answered