GamifiedGamified challengesPracticeRaceLeaderboardPricing

Learn data structures by watching them move, then doing it yourself. Press M anytime to toggle sound.

A product of Amypo Technologies, Coimbatore.

Learn

  • All topics
  • Learning path
  • How it works
  • About dotprep

Practice

  • Practice arena
  • Daily challenge
  • Gamified challenges
  • Race mode
  • Leaderboard

Interview prep

  • Java interview
  • Coding questions
  • Company questions
  • System design
  • Jobs & careers

You

  • Dashboard
  • Settings
  • Pricing
  • Create account

© 2026 dotprep

Content last updated 20 November 2026

GamifiedGamified challengesPracticeRaceLeaderboardPricing
System designDesign a distributed rate limiter

Design a distributed rate limiter

Limit requests per user across many servers: algorithm choice, Redis counters, atomicity and failure modes.

intermediate 12 min read 4 interview questions

On this page

1/6
  1. 01Requirements
  2. 02Where it runs
  3. 03Algorithm and storage
  4. 04Scale and failure
  5. 05Interview questions
  6. 06Quick check
Back to top

Requirements

  • Limit each client (user id, API key or IP) to N requests per window, with different rules per endpoint or plan.
  • Work across many API servers: the limit is global, not per server.
  • Add very little latency (a millisecond or two) and stay available.
  • Tell clients when they're limited: HTTP 429 with Retry-After and X-RateLimit-Remaining headers.

Where it runs

Usually in the API gateway or as middleware in each service, backed by a shared fast store (Redis) so all servers see the same counters. Rules ("free plan: 100/min on /search") live in config and are cached in memory by each limiter.

Algorithm and storage

Token bucket and sliding-window counter both need just a couple of numbers per client, so they fit easily in Redis. The key point is atomicity: two servers doing read-then-write concurrently can both see 99 and both allow request 100. Do the check-and-update in one atomic step, with a Lua script or atomic INCR.

text
-- fixed-window counter, run atomically as a Redis Lua script
-- KEYS[1] = "rl:{user}:{endpoint}:{minute}", ARGV[1] = limit
local n = redis.call('INCR', KEYS[1])
if n == 1 then redis.call('EXPIRE', KEYS[1], 60) end
if n > tonumber(ARGV[1]) then return 0 end
return 1

A fixed window allows up to 2× the limit around a window boundary. The sliding window counter fixes most of that: estimate = current count + previous count × (fraction of the previous window still inside the last 60 seconds).

Scale and failure

  • Shard counters across Redis nodes by client key; each client's counters live together.
  • If Redis is unreachable, decide in advance: fail open (allow traffic, protect availability) or fail closed (reject, protect the backend). Most public APIs fail open.
  • To cut latency, servers can keep local counters and sync to Redis in small batches, accepting slight overshoot.
  • Rate limit early: at the edge or gateway, before expensive work happens.

See it animated

Sliding windowQueueHash functions

Interview questions

Try answering out loud first, then open the model answer.

Q1Why can't each server just keep its own counter?beginner

With 10 servers behind a load balancer, a client could get 10× the limit, and the effective limit would change whenever you scale. A shared store gives one global count. (Local counters are fine as an approximation if you divide the limit by the number of servers.)

Q2How do you prevent race conditions in a distributed limiter?intermediate

Make the read-modify-write atomic in the store: Redis INCR is atomic, and a Lua script runs as one unit, so the check and update can't interleave between servers.

Q3Fail open or fail closed?intermediate

Depends on what the limiter protects. If it mainly enforces fair usage, fail open so a Redis outage doesn't become a full outage. If it protects a fragile backend or stops abuse such as login brute-forcing, fail closed or fall back to conservative local limits.

Q4Which algorithm would you choose?intermediate

Token bucket when short bursts are acceptable and you want a steady average (most API limits). Sliding window counter when you need a smooth limit without boundary bursts at low memory cost. Sliding window log only when exactness matters and volumes are small.

Quick check

Question 1

1.The HTTP status for a rate-limited request is…

Question 2

2.Fixed-window counters can allow up to how many requests in a short span around a window edge?

Question 3

3.Atomic check-and-increment in Redis is done with…

0 of 3 answered

Design a URL shortenerDesign a chat system
Log inStart free