Rate limiting restricts how many requests a client may make in a given time window. It protects services from overload, abuse and runaway automation — and it protects other users from a single client consuming all capacity.

Why you need it

  • Protection from abuse: Credential stuffing, scraping, and brute-force logins are all high-volume patterns.
  • Cost control: Expensive operations (report generation, LLM calls, bulk exports) cost real money per request.
  • Fair usage: One misbehaving script should not starve everyone else.
  • Stability: A sudden flood is stopped before it cascades into a full outage.
  • Third-party protection: Your upstream API may have its own limits; respecting them keeps you in service.

The four common algorithms

1. Fixed window
   Counter per minute, reset at the minute boundary.
   Simple, but allows 2x the limit across a boundary.
   |---- 100 allowed -----|--- 100 allowed ---|
                           ^ burst of 200 here

2. Sliding window
   Count over the last N seconds precisely.
   Smooth, slightly more memory per client.

3. Token bucket  (most common)
   A bucket holds tokens; each request consumes one.
   Tokens refill at a fixed rate, up to a maximum.
   Allows short bursts up to the bucket size.
   capacity 100, refill 10/s:
     idle 30s -> bucket full at 100
     burst of 100 permitted, then 10/s sustained

4. Leaky bucket
   Requests queue and drain at a fixed rate.
   Smooths traffic, adds latency. Good for outbound
   calls to a partner API with strict limits.

Redis implementation of a token bucket (atomic via Lua):
  EVAL "local t=redis.call('get',KEYS[1]) or ARGV[1]
        local ts=redis.call('ttl',KEYS[1])
        if tonumber(t) > 0 then
          redis.call('decr',KEYS[1])
          return 1
        end
        return 0" 1 bucket:user42 100

What to limit on

  • API key / account — the fairest dimension for authenticated traffic.
  • IP address — necessary for unauthenticated endpoints, but problematic behind NAT (a whole office shares one address) and easy to rotate.
  • Endpoint — limits per route, because /login and /reports have very different risk profiles.
  • Tenant — in multi-tenant systems, per customer, so one tenant cannot consume the shared capacity.
  • Combination — for example per API key per endpoint, with an additional global fallback.

Sensible defaults

Public unauthenticated endpoints
  /login          5 per 15 min per IP + account   (lockout after)
  /register       3 per hour per IP
  /password-reset 3 per hour per email
  /search         30 per minute per IP
  anything else   60 per minute per IP

Authenticated API
  read            1000 per minute per key
  write           100 per minute per key
  expensive ops   10 per hour per key
  global ceiling  whatever the infrastructure survives

Delivery / e-mail
  50 recipients per message, 500 per day per account
  -> otherwise you are a spam cannon

The response when the limit is hit

HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1758921600

{"error": "rate_limited",
 "detail": "Limit 100 requests/minute exceeded.",
 "retry_after_seconds": 30}

# Rules:
#   - 429, never 403 or 500. Clients must be able to detect it.
#   - Always send Retry-After.
#   - Report the limit and remaining count on EVERY response,
#     not just when the limit is hit - clients can then slow down
#     proactively instead of hammering until refused.
#   - Document it. An undocumented limit is an outage waiting
#     for a customer to discover it in production.

Where to enforce it

  • Reverse proxy / API gateway (nginx, Envoy, Traefik, cloud gateways): protects everything behind it, central configuration, no application changes. The first line of defence.
  • Application layer: needed for business-specific logic (per account, per plan, per tenant) which a gateway cannot know.
  • Both: the normal setup. A generous coarse limit at the edge, finer limits in the application.
# nginx - coarse protection at the edge
limit_req_zone $binary_remote_addr zone=allgemein:10m rate=60r/m;
limit_req_zone $binary_remote_addr zone=login:10m     rate=5r/m;
limit_conn_zone $binary_remote_addr zone=verbindungen:10m;

server {
    location /login {
        limit_req  zone=login burst=3 nodelay;
        limit_conn verbindungen 10;
        proxy_pass http://api;
    }
    location / {
        limit_req  zone=allgemein burst=20 nodelay;
        proxy_pass http://api;
    }
}
# nodelay: reject immediately instead of queueing - better for
# interactive clients (queueing makes them feel slow, not limited).

# Proof it works - the most important step
for i in $(seq 1 40); do
  curl -s -o /dev/null -w '%{http_code} ' http://localhost/login
done; echo
# Expect: 200 200 200 200 429 429 ...

Strategies that accompany rate limiting

  • Circuit breaker: after repeated failures, stop calling a dependency entirely for a while, instead of piling up timeouts.
  • Load shedding: under overload, drop the least important traffic first (health checks and reads before payments).
  • Exponential backoff with jitter: what you ask clients to do. Jitter matters — without it, all clients retry in lock-step.
  • WAF and bot detection: handling scraping and credential stuffing at a different layer than plain rate limits.
  • Hardware/OS level: connection limits, SYN cookies, and per-source conntrack limits for raw packet floods.

Pitfalls

  1. IP-only limiting behind a proxy. All requests appear to come from the load balancer — you either limit everyone together or nothing. Trust X-Forwarded-For only from known proxies.
  2. No per-endpoint distinction. One global limit means a single search flood can lock out payments.
  3. Per-instance counters. Four application instances each allow 100/min = 400/min effective. Use a shared store (Redis) or enforce at the gateway.
  4. Limiting your own monitoring. Health checks and metrics scraping must be excluded, or your alerts fire because you starved yourself.
  5. Locking out legitimate customers. Verify by testing with realistic traffic patterns; publish limits and provide a way to request more.
  6. Silent limiting. Returning 200 while dropping requests makes debugging impossible for the client.
  7. Not limiting outbound calls. A retry loop against a partner API can get your account suspended.
  8. No monitoring. If the 429 count is not on a dashboard, you will learn about the limit from an angry customer.

Related: Cache, REST, Proxy, Rate Limiting (DE).