teacher
Sign in

Rate Limiting

Cap how many requests a client can make, using token bucket, leaky bucket and window counters, usually backed by Redis.

30 min5 interactive

What rate limiting is

A rate limiter caps how many requests a client may send in a period of time. Extra requests are rejected or delayed.

It protects you from:

  • Abuse: scrapers, brute-force login attempts, credential stuffing.
  • Accidents: a client bug that retries in a tight loop.
  • Overload: one noisy customer starving everyone else (the "noisy neighbour" problem).
  • Cost: paid downstream calls, such as an SMS or LLM API.

It also lets you sell tiers: 60 requests/min free, 600/min paid.

Decide three things before picking an algorithm:

  1. The key: who is limited? User id, API key, IP address, or a combination. IP alone is weak because many users share one IP behind a NAT (Network Address Translation, for example an office or mobile carrier).
  2. The limit: for example 100 requests per minute, with a short burst allowed.
  3. The action: reject with an error, queue, or slow down.
1 of 9