Debugging & Real-World · Lesson 06

Handling 429s & throttling

A 429 Too Many Requests response is not a bug — it's a boundary marker. The API is telling you exactly how much it can absorb. Getting 429s means you've hit that boundary; handling them correctly means you stay on the right side of it permanently.

Why you're getting 429s

Rate limiting is the API provider's mechanism for distributing capacity fairly across all callers and protecting the service from overload. See Lesson rel-03 for the full taxonomy of rate-limiting algorithms. For debugging purposes, the most important question is: which limit did you hit? APIs commonly enforce limits at multiple granularities simultaneously:

• Per-second or per-minute limit — a burst limit that prevents a single caller from flooding the service in a short window. You're sending too many requests too quickly.
• Per-day or per-month limit — a quota limit based on a billing tier. You've used up your allocation for the current period.
• Per-endpoint limit — some endpoints (e.g., a search endpoint that's expensive to serve) have lower limits than others.
• Per-user or per-key limit — your API key may have a different limit than the global limit; or a specific user's actions may be throttled independently of your overall quota.

The rate-limit headers in the response tell you which limit was hit and how to calculate the wait time.

Reading the rate-limit headers

The first step is to read the response headers from the 429:

In the scenario above: 100 requests per minute, you're sending ~90 per minute from the sync job. Why are you hitting the limit? The job sends all 90 requests in the first 5 seconds of each minute, not spread over 60 seconds. From the API's perspective, that's 90 requests in 5 seconds — potentially triggering a per-second burst limit even if the per-minute quota wouldn't be exceeded. The fix is not to reduce total volume, but to spread the requests over time.

Exponential backoff with jitter

When you receive a 429, the worst response is to retry immediately. If multiple instances of your service all hit the rate limit at the same moment and all retry immediately, they'll all hit it again at the same moment — a thundering herd that extends the outage. Exponential backoff spreads retries out; jitter breaks the synchrony between instances.

Reduce call volume: batch and cache

Backoff handles the "what to do when you've already hit the limit" case. The better strategy is to not hit the limit in the first place. Two techniques cut call volume significantly:

Batch requests

Many APIs support fetching multiple resources in a single call. Instead of calling GET /users/{id} once per user ID, check whether the API offers a batch endpoint like GET /users?ids=1,2,3,4,5 or POST /batch/users. One batch request that fetches 50 records costs 1 rate-limit unit, not 50.

Cache responses

If you're calling the same endpoint for the same resource repeatedly, cache the response. Use the Cache-Control and ETag headers from the API response to determine how long the cache is valid and when to revalidate. Even a 60-second in-process cache on a high-frequency operation can reduce call volume by orders of magnitude.

Add a client-side rate limiter

A client-side limiter prevents your code from ever sending more than N requests per second, regardless of how much traffic the rest of your application generates. This is proactive throttling: you enforce the limit yourself before the server has to. It's especially valuable in bulk-processing jobs where you know the total volume will exceed the quota.

When to request a quota increase

Sometimes the right answer is not to optimise your code — it's to ask the provider for a higher quota. Signs you've done the engineering work and genuinely need more:

• You've batched everything batchable.
• You've added caching where the data allows it.
• You've added a client-side limiter and you're still hitting the limit at the rate your business requires.
• The limit is a billing-tier limit, not a technical one.

When you contact the provider, give them specifics: current limit, current usage, required throughput, and what you've already done to reduce volume. Providers are more likely to grant increases to callers who demonstrate they've already optimised.

Putting it together: handling a 429 in production

1. Receive the 429. Read Retry-After (preferred) or X-RateLimit-Reset. Calculate the wait time.
2. Do not retry immediately. Log the 429 with the wait time, rate-limit headers, and the endpoint that was called. This log is essential for diagnosing which limit was hit.
3. Wait with jitter. Sleep for Retry-After + random(0, 0.5 × Retry-After). The jitter prevents a stampede if multiple instances hit the limit at the same time.
4. Retry with backoff. If the first retry also returns 429, double the wait. Cap the maximum wait at a sensible ceiling (60–120 s).
5. After N retries, surface the error. Don't retry forever — eventually return the 429 error to the caller so they can handle it. Alert if 429s are sustained (more than X per minute for more than Y minutes) — that's a sign your traffic pattern has changed.
6. Review the access patterns that generated the 429s. Were they from a bulk job? Add batching and a client-side limiter to that job. Were they from real-time user traffic? Consider caching upstream or request coalescing.

Under the hood: how it actually works

The 429 headers are not guesses — the server derives them from its internal rate-limiting state. Understanding that derivation tells you exactly what each number means and how a correct client should read and act on it.

How the server computes remaining quota: token bucket and fixed window

Most APIs use one of two algorithms to track quota. The values in the response headers directly reflect the algorithm's state at the moment of your request.

Token bucket. A bucket holds up to capacity tokens. One token is consumed per request. Tokens refill continuously at a rate of capacity / window tokens per second. If the bucket is empty, the request is rejected with 429. This permits short bursts up to capacity while enforcing a steady-state throughput ceiling. The server computes X-RateLimit-Remaining as floor(current_tokens).

Fixed window. A counter resets to 0 at each window boundary (e.g., the start of each minute). Each request increments the counter. When counter == limit, subsequent requests in that window are rejected. X-RateLimit-Remaining = limit - counter. X-RateLimit-Reset is the Unix timestamp of the next window boundary. This is simpler but allows a burst of 2× the limit if requests straddle two windows (the last second of window N + the first second of window N+1).

Exact header semantics: where each value comes from

On a 429 the server still writes these headers from its internal state — so Remaining will be 0, and Reset/Retry-After tells you exactly how many seconds until the counter resets. On a 200 the same headers reflect remaining capacity after deducting the current request.

How a correct client reads Reset/Retry-After and backs off with jitter

The algorithm a well-behaved client should follow on every response:

# Pseudocode — run after every HTTP response
def handle_response(resp, attempt):
    remaining = int(resp.headers.get("X-RateLimit-Remaining", 999))
    limit     = int(resp.headers.get("X-RateLimit-Limit", 999))

    # Proactive slow-down on successful responses (before hitting 0)
    if resp.status_code == 200 and remaining < limit * 0.10:
        wait_until_reset(resp)   # or reduce request rate

    if resp.status_code != 429:
        return resp              # success path

    # 429: compute the mandatory floor from server hints
    retry_after = resp.headers.get("Retry-After")
    if retry_after:
        mandatory_floor = float(retry_after)
    else:
        reset_ts = int(resp.headers.get("X-RateLimit-Reset", 0))
        mandatory_floor = max(0, reset_ts - time.time())

    # Add exponential backoff on top of the mandatory floor
    base_backoff = min(60, 1 * (2 ** attempt))   # 1, 2, 4, 8... capped at 60 s
    wait_seconds = max(mandatory_floor, base_backoff)

    # Full jitter: randomise within [0, wait_seconds] to de-synchronise instances
    import random
    jitter = random.uniform(0, wait_seconds)
    time.sleep(jitter)
    return None   # signal caller to retry

The key points: Retry-After (when present) is the server's authoritative floor — never wait less. Exponential backoff adds a delay on top of that floor for repeated 429s. Jitter randomises across instances.

Worked trace: requests crossing the limit and client recovery

• RFC 6585 §4 — 429 Too Many Requests (original specification)
• MDN — Retry-After header
• AWS Builder's Library — Timeouts, retries, and backoff with jitter

Sources & further reading

• MDN — 429 Too Many Requests
• RFC 6585 §4 — 429 Too Many Requests (original specification)
• AWS Builder's Library — Timeouts, retries, and backoff with jitter
• Google Cloud — Rate limiting strategies and techniques
• Stripe — Rate limits documentation