API Rate Limiting Best Practices: A Complete Guide for Engineering Teams

It was 2:14am when the pager went off. A single customer's misconfigured cron job was firing 400 requests per second at our /exports endpoint, each one spinning up a Postgres query that touched half the orders table. The database CPU was pinned. Every other tenant on the cluster was getting 30-second response times. We had rate limiting, technically. It was IP-based, and this customer was behind a corporate NAT, sharing an egress IP with their billing system that we very much did not want to block.

That's the thing about rate limiting. It's not "add a middleware and move on." It's a set of architectural decisions about reliability, fairness, security, and how your API behaves under load from users who are sometimes hostile, sometimes buggy, and usually just in a hurry. Get it right early and you'll sleep through most incidents. Get it wrong and you'll retrofit it under load, which is one of the worst engineering experiences available on this planet. This guide covers the four jobs rate limiting actually does, the four algorithms worth knowing, where to apply them, what to return, how consumers should handle limits, and the specific mistakes that keep shipping to production.

What API rate limiting actually does (and what it doesn't)

Rate limiting does three jobs. First, it protects your infrastructure from traffic that would otherwise degrade or take down the service, whether that traffic is malicious, buggy, or just enthusiastic. Second, it enforces fairness between tenants so one customer's batch job doesn't starve everyone else. Third, it mitigates a specific class of security threats: credential stuffing, scraping, enumeration attacks, and brute-force attempts against auth endpoints.

What it is not: it's not authentication, it's not authorization, and it's not a cost-control mechanism on its own. If your only defense against a $40,000 cloud bill is a rate limiter, you've built a tripwire, not a budget. Rate limiting also doesn't replace proper load testing. A limit of 1,000 requests per second means nothing if your system falls over at 600. Know your real capacity before you pick your limits.

The four rate limiting algorithms, compared

There are four algorithms worth knowing. Every production rate limiter is some variation or combination of these.

Token bucket. A bucket holds N tokens. Each request consumes one token. Tokens refill at a steady rate (say, 10 per second, capped at 100). If the bucket is empty, the request is rejected. The virtue of token bucket is that it allows bursts up to the bucket size while enforcing a sustained rate over time, which matches how real API traffic actually behaves. Stripe uses token bucket for its rate limiter, and it's the default choice for most public APIs.

Leaky bucket. Requests enter a queue (the bucket). The queue drains at a fixed rate (the leak). If the queue is full, new requests are rejected. Unlike token bucket, leaky bucket smooths bursts into a constant output rate, which is useful when the downstream system genuinely cannot handle spikes (think: a legacy system, a paid third-party API, a database with fragile connection pooling). The trade-off is that legitimate bursts get queued or dropped even when capacity is available.

Fixed window. Count requests in discrete time buckets: 0–60 seconds, 60–120 seconds, and so on. If a user exceeds the limit within a window, reject. Fixed window is trivial to implement (a Redis INCR with an expiring key) and easy to reason about, but it has a well-known edge case: a user can send 100 requests at second 59 and another 100 at second 61, effectively doubling the limit across a two-second span.

Sliding window. Fixes the fixed-window burst problem by tracking request timestamps in a rolling interval. Two common variants: sliding log (store every timestamp, count those within the window) and sliding window counter (weighted average across the current and previous fixed window). Sliding log is accurate but expensive in memory. Sliding window counter is a good compromise and the one most production systems actually ship.

A real production system typically runs more than one. Stripe runs a request rate limiter, a concurrent request limiter, and a fleet usage load shedder that reserves capacity for critical API methods like creating charges versus non-critical ones like listing charges. That's three different defenses addressing three different failure modes, and it's a reasonable mental model for any API that matters.

Implementation scopes: where to apply limits

Picking the algorithm is half the work. The other half is picking the dimension you're limiting against.

IP-based is the crudest scope. Easy to implement, useful as a last line of defense against unauthenticated abuse, but broken for any real customer scenario: corporate NATs, mobile carriers, and shared office networks all mean a single IP can represent hundreds of legitimate users. Use IP limits for unauthenticated endpoints and public marketing pages, not for authenticated product traffic.

User-based (by user ID, session, or account) is the right default for most authenticated endpoints. It's fair (each customer gets their own budget), it's explainable ("you hit your account limit"), and it aligns with how you actually bill and reason about customers.

API-key-based is what you want for B2B APIs where customers integrate programmatically. It lets you tier limits by plan (free tier: 10 req/s, Pro: 100 req/s, Enterprise: custom) and it gives customers a clean identity to hand to their own monitoring.

Endpoint-specific limits layer on top of the above. A list endpoint that does a 50-table join needs a tighter limit than a key-value lookup. Stripe applies stricter limits to individual resources that count against the global limit, with a default of 25 requests per second per API endpoint, which is a good example of stacking endpoint limits under a global ceiling.

In practice, most mature APIs use a combination: a global per-account or per-API-key limit, plus per-endpoint limits on expensive routes, plus a concurrency cap on long-running requests. Single-dimension rate limiting is almost always wrong for anything that matters.

The response contract: HTTP 429 and the headers you must return

When a request is rate-limited, return HTTP 429 Too Many Requests. This is not optional and it is not negotiable. Returning 503 Service Unavailable is acceptable only when the limit reflects a system-wide overload rather than a per-client limit. Returning 200 with an error body in it is a war crime.

The headers you should include:

• Retry-After (how many seconds the client should wait before retrying). This is the single most important header because it lets well-behaved clients self-throttle without guessing. Without it, every client library falls back to hardcoded backoff constants, which are either too aggressive or too lazy.
• X-RateLimit-Limit (the ceiling for the current window: requests per second, per minute, whatever your unit is).
• X-RateLimit-Remaining (how many requests the client has left in the current window).
• X-RateLimit-Reset (when the window resets, usually as a Unix timestamp).

These X-RateLimit-* headers are de facto standard but not formally standardized in an RFC. The IETF working group draft (draft-ietf-httpapi-ratelimit-headers) reached revision 10 and expired in March 2026 without being published as an RFC, so the X-RateLimit-* convention remains de facto rather than formally standardized. GitHub uses exactly this format. Stripe takes a different approach: when a 429 is returned due to rate limiting, Stripe sets a header indicating the specific limiter that triggered, with values like global-concurrency, global-rate, endpoint-concurrency, endpoint-rate, or resource-specific. If a 429 comes back without that header, it wasn't a rate limit. It was a lock timeout on the object being modified, which is a different failure mode with similar symptoms.

A sample response:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 12
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1765432112

{
  "error": {
    "type": "rate_limit_exceeded",
    "message": "Too many requests. Retry after 12 seconds.",
    "docs_url": "https://api.example.com/docs/rate-limits"
  }
}

The body should tell a human what happened, what to do, and where to read more. The headers should tell a machine the same thing.

Best practices for API consumers (handling rate limits gracefully)

If you're on the consumer side of someone else's API, assume 429s will happen and design for them from day one. The core pattern is exponential backoff with jitter. Double the wait between retries, add randomness so you don't thundering-herd with every other client retrying at the same instant, and cap the total retries so a dead endpoint doesn't retry forever.

import random
import time

def call_with_backoff(request_fn, max_retries=5, base_delay=1.0, max_delay=60.0):
    for attempt in range(max_retries):
        response = request_fn()

        if response.status_code != 429:
            return response

        # Prefer the server's Retry-After if present
        retry_after = response.headers.get("Retry-After")
        if retry_after is not None:
            delay = float(retry_after)
        else:
            # Exponential backoff with full jitter
            delay = min(max_delay, base_delay * (2 ** attempt))
            delay = random.uniform(0, delay)

        time.sleep(delay)

    raise Exception("Max retries exceeded")

Three things this pattern gets right. First, it honors Retry-After when the server provides it, which is almost always more accurate than your local guess. Second, it uses full jitter (random between 0 and the computed delay), which is the variant AWS recommends in their exponential backoff and jitter guidance, and is demonstrably better than fixed exponential backoff at preventing synchronized retries. Third, it caps total retries. A client that retries forever is indistinguishable from an attacker.

Beyond backoff, good consumers watch X-RateLimit-Remaining proactively and slow down before hitting the wall, cache aggressively on endpoints where data staleness is acceptable, and use webhooks or long-polling instead of tight polling loops wherever the API supports it. As of the stripe-node library, 429 responses are not automatically retried by the built-in maxNetworkRetries mechanism, so explicit backoff in your own client code is still required.

Common mistakes that ship to production

Five mistakes I've seen (and shipped) more than once.

1. Forgetting to rate-limit auth endpoints. Your /login and /forgot-password endpoints need the tightest limits you have, and they should be limited by IP and by target account, not just by session. Credential stuffing attacks rotate through stolen username/password pairs at thousands of attempts per minute. If your login endpoint has the same limit as your /users/me endpoint, you've built a credential-stuffing-as-a-service platform.

2. Using fixed window without understanding edge-case bursts. A user can send 100 requests at second 59 of window A and 100 more at second 0 of window B. You just served 200 requests in under two seconds against a "100 per minute" limit. If that matters for your system, use sliding window or token bucket.

3. Returning 429 without Retry-After. Your clients will pick a random backoff constant and either hammer you back too fast or wait too long. Neither is what you want. Including Retry-After takes one line of code and makes every client library in the world behave correctly.

4. Rate limiting on the wrong dimension. IP-based limits on B2B APIs where half your customers share a corporate NAT. User-based limits on a scraper-facing marketing site. API-key limits on a product that lets users rotate keys every hour. The dimension has to match how customers actually use the product.

5. Not distinguishing burst from sustained limits. "100 requests per minute" and "10 requests per second averaged over 60 seconds" are very different contracts. The first allows all 100 in the first second. The second doesn't. Token bucket naturally expresses both dimensions (bucket size = burst, refill rate = sustained). Document which contract you're offering, and enforce it.

Choosing the right approach for your stack

The right approach depends on who's calling your API and what happens when they misbehave.

If you're a B2B SaaS with tiered pricing, use token bucket keyed on API key, with per-endpoint limits layered on top, a concurrency cap for expensive routes, and tier-specific ceilings that map directly to your pricing page. Store counters in Redis (with a replica or cluster if you care about durability) because you need low-latency reads on every request and you don't need the counter to survive a full datacenter outage.

If you're a public API with potential for abuse, use IP-based limits as an unauthenticated floor, user-based limits once authenticated, and a separate, much tighter limit on auth and write endpoints. Log every 429 with enough context to investigate, because persistent 429s from a single source usually mean either a misconfigured legitimate client or an attacker doing reconnaissance, and you want to know which.

If you're an internal service mesh, the answer is usually different: you're less worried about abuse and more worried about cascade failures. Concurrency limits and circuit breakers at the service-to-service layer (Envoy, Linkerd, or similar) often matter more than request-rate limits.

Whatever you pick, build kill switches. Stripe's own guidance is to have feature flags in place so you can disable rate limiters if they trigger erroneously, dark launch each limiter to watch the traffic it would block before enforcing, and set up alerts to understand how often limits fire. A rate limiter that fires without observability is a production outage waiting for an excuse.

Rate limiting is a reliability investment with a brutal asymmetry: the cost of building it early is small, and the cost of retrofitting it under load is the worst week your on-call rotation will have that quarter. Pick your algorithms, pick your scopes, return the right headers, document the contract, and build the kill switches before you need them.