Live System Design Simulator

Distributed Rate Limiter — Live System Simulator

Every request checks a shared token bucket at the gateway. Under limit → forwarded. Over limit → 429. Watch tokens drain & refill in real time.

token bucket10 tokens · refill 2/sper-user keyatomic in Redis

speed

Requests

Allowed

429 Throttled

—

Allow ratio

QPS (live)

Token Bucket

Algorithm

Token buckets (Redis)

User A

10/10

User B

10/10

Capacity 10 · refills +2 tokens/sec · each request spends 1

Step log

Ready. Send a request, or fire Burst ×20 to watch the bucket empty and requests start getting 429'd.

request in allowed → service 429 throttled token refill

How it works: the gateway runs one atomic token-bucket check in Redis keyed by user. A full bucket lets a short burst through, then the bucket must refill (2/s) — so sustained traffic above the rate gets throttled while bursts are tolerated. Users A and B have independent buckets (per-key), so one noisy user can't starve another. If Redis is unreachable, the limiter fails open (allows) to protect availability.

Why token bucket, and why fail open

The token bucket algorithm is popular for rate limiting because it naturally tolerates bursts while still enforcing a long-run average rate. Each user key gets a bucket with a fixed capacity; every request spends one token, and tokens regenerate at a steady rate. As long as the refill rate matches sustained traffic, requests are never throttled — only traffic that exceeds the sustainable rate for longer than the bucket can absorb gets a 429.

The bucket state lives in Redis so it's shared across every gateway instance — critical in a horizontally scaled deployment where the same user's requests can land on different nodes. The check-and-decrement has to be atomic (a Lua script or `INCR`+`EXPIRE` pattern) to avoid race conditions under concurrent requests. When Redis itself is unreachable, this simulator fails open — allowing all traffic through rather than blocking it — because an outage in the limiter should never be allowed to take down the entire API; that's a deliberate availability-over-strictness trade-off.