Side by side
| Dimension | Rate limiting | Throttling |
|---|---|---|
| Core purpose | Enforce a request or resource-use budget | Control the pace of traffic or execution |
| Typical action | Admit within budget; reject excess | Delay, pace, reduce throughput, or reject |
| Excess traffic | Often returned to the caller as HTTP 429 | May wait in a bounded queue |
| Burst behavior | Depends on window and burst allowance | Can smooth bursts before downstream execution |
| Latency trade-off | Fast rejection avoids server-side waiting | Pacing adds waiting time if work is queued |
| Typical controls | Per-user budgets, token buckets, sliding windows | Dispatch pacing, bandwidth caps, adaptive controls |
| State required | Counters, timestamps, or token balances | Pacing state; queue state if work is delayed |
| Main failure risk | Inconsistent enforcement or unfair rejection | Growing backlog, missed deadlines, or dropped work |
The differences that matter
The vocabulary is not standardized. Some APIs call every HTTP 429 response throttling, while others reserve throttling for slowing requests down. In an interview, describe the actual behavior: reject immediately, hold briefly, enqueue durably, or reduce the producer's sending rate.
A rejection-based rate limiter makes an admission decision without taking responsibility for delayed execution. A queue-based throttle accepts responsibility for waiting work, so the design needs queue capacity, expiration, cancellation, and overflow rules. For durable asynchronous work, return an accepted response and a job identifier rather than leaving an HTTP connection open indefinitely.
Neither term determines burst behavior. A token bucket can allow a burst while enforcing a sustained rate; a fixed window can allow concentrated traffic around a window boundary. A paced dispatcher can smooth delivery, but its queue still needs an admission limit. Name the algorithm and its parameters rather than assuming that rate limiting means a strict, evenly spaced flow.
The controls often belong at different layers. An API gateway can enforce per-tenant budgets, while workers pace calls to a constrained dependency. Use both when needed: tenant limits provide fairness, while shared downstream controls protect aggregate capacity. A request-rate cap also does not bound in-flight work when request duration varies, so concurrency limits may be necessary.
Choose Rate limiting when
- A synchronous API should reject excess requests quickly rather than spend its latency budget waiting.
- You need per-user, per-key, or per-tenant budgets to prevent one caller from consuming everyone else's capacity.
- Abusive traffic should be rejected before expensive authentication, database access, or application processing.
- Callers can handle HTTP 429 responses and retry later using bounded exponential backoff with jitter.
Choose Throttling when
- Background jobs can wait, and smoothing a short burst is preferable to rejecting useful work.
- A downstream provider has a throughput allowance that workers must respect across all instances.
- A crawler must pace requests to each destination instead of exhausting its allowance in a burst.
- Bulk transfers or migrations should reduce their sending rate when production traffic needs capacity.
A worked example
Notification admission limits and worker pacing
Suppose a notification service gives each tenant a 100-request-per-second token bucket with room for 200 tokens. Requests without tokens receive HTTP 429. Separately, workers pace delivery to a provider at 500 messages per second. Assume accepted traffic across tenants reaches 1,000 messages per second for 30 seconds, with one message per request and an initially empty queue. The backlog grows by roughly 15,000 messages, representing about 30 seconds of work at the provider's allowed rate. Per-tenant rate limits did not protect aggregate capacity. Add a bounded queue and global admission policy; use message deadlines to decide whether to reject, defer, or expire excess work.
In the interview
- Start with semantics: identify the protected resource, the limiting key, the sustained rate, the burst allowance, and what happens to excess work. Say explicitly whether throttling means delaying or rejecting in your design.
- Separate fairness from capacity protection. Explain why per-tenant limits may need an additional global limit or concurrency cap, then expect questions about noisy tenants and uneven request costs.
- For distributed enforcement, discuss atomic updates, shared-store latency, and failure behavior. Local budgets reduce coordination but can exceed a global allowance unless capacity is explicitly allocated among instances.
- If you queue work, estimate backlog growth and waiting time. State the maximum queue size, deadline policy, and overflow action; expect probes about sustained overload and recovery.
- Explain feedback to callers and operators: HTTP 429 for caller limits, Retry-After when useful, and metrics for rejection rate, queue age, queue depth, and downstream saturation.
Common mistakes
- Claiming that rate limiting always rejects and throttling always delays. That is a useful local convention, not a universal definition.
- Using an unbounded queue to avoid rejection. Sustained overload then becomes increasing latency, storage consumption, and eventually failure.
- Treating a requests-per-second limit as a concurrency limit, even though slow requests can accumulate far more in-flight work.
- Choosing a nominal rate without specifying burst allowance, distributed enforcement accuracy, retry behavior, or what happens when the limiter fails.
Where it comes up
Interview questions where choosing between Rate limiting and Throttling comes up.
- Design a distributed rate limiterThe design requires explicit admission semantics, burst handling, distributed enforcement, and failure policies.
- Design a notification systemPer-tenant admission limits and per-provider delivery pacing protect different resources and require bounded backlogs.
- Design a web crawlerPer-host politeness requires paced scheduling rather than merely allowing a burst within a time window.
- Design a large data migration to the cloudBulk-copy throughput must be controlled to respect the limited link and avoid overwhelming source or destination systems.
- Design a flash giveaway: 6 million burgers in an hourAdmission limits and bounded waiting protect the claim path during bursts, while atomic inventory allocation still enforces correctness.
Questions people ask
Are rate limiting and throttling the same thing?
They are often used interchangeably. A useful distinction is that rate limiting specifies an allowance, while throttling describes controlling traffic through rejection, delay, or pacing. Always check the system's documented behavior rather than relying on the label.
Does throttling return HTTP 429 or delay requests?
It can do either. Many APIs call rejection with HTTP 429 throttling, while traffic-shaping systems delay traffic to control its pace. If requests are delayed, specify the maximum wait and what happens when that limit is reached.
Can you use rate limiting and throttling together?
Yes. A gateway can reject requests that exceed a tenant budget while a worker pool paces accepted jobs against a downstream allowance. The queue between them must have explicit capacity and expiration policies because pacing alone cannot absorb sustained overload.