How it works
Clients call a public API endpoint rather than individual backend services. The gateway matches the request's host, path, or method to a route: for example, GET /products/123 goes to the catalog service, while POST /orders goes to the order service.
Before forwarding a request, the gateway can validate credentials, enforce rate limits, reject oversized payloads, and attach tracing context. A gateway might verify a JWT locally using cached public keys; checking an opaque session token may require a session lookup.
The gateway selects a healthy backend instance using configured endpoints or service discovery, then forwards the request over a pooled connection. It may terminate TLS itself or sit behind another TLS-terminating load balancer; internal connections can also use TLS.
Backend services still enforce resource-specific authorization and business rules. A valid token can identify a user, but the order service must still verify that the user may view order 123. Services must accept forwarded identity only from a trusted, authenticated gateway path.
Run multiple gateway instances behind a load balancer and distribute versioned route and policy configuration to them. Keep request handlers mostly stateless; shared quotas or session checks need separate state, with explicit behavior when that state is unavailable.
Bound upstream timeouts, concurrent requests, and retries so a slow service cannot consume all gateway capacity. Forward idempotency keys unchanged, and do not automatically retry a write merely because its response timed out.
A worked example
Protecting a giveaway claim endpoint
Suppose a giveaway receives 40,000 claim requests per second, but load tests show the claim service can safely handle 5,000. Put replicated gateways behind a load balancer. They validate credentials, apply per-account limits, and enforce an aggregate admission budget below backend capacity, returning 429 for quota violations and 503 for overload. With eight gateway instances, independent limits of 5,000 each would admit 40,000, so use coordinated limits or conservatively allocated local budgets. The claim service still atomically enforces stock and one claim per account. Forward each request's idempotency key unchanged; a timeout must not trigger an unbounded retry. Check gateway capacity and shared-limiter latency separately.
Trade-offs to say out loud
- Consistent policies versus a shared failure point
- Central enforcement reduces duplicated security and traffic-control code, but a bad gateway configuration can affect many otherwise healthy services.
- Simpler clients versus another network hop
- One public endpoint hides backend topology, but proxying and policy checks add latency and consume connection, CPU, and memory capacity.
- Local checks versus fresh shared state
- Local token validation and local limits are fast, while prompt revocation and strict fleet-wide quotas require coordination or additional lookups.
- Availability versus enforcement during outages
- If a quota store fails, allowing traffic preserves availability but weakens protection, while rejecting traffic protects backends at the cost of valid requests.
- Useful mediation versus business-logic coupling
- Small protocol transformations can simplify clients, but workflows and domain decisions in the gateway couple service releases and complicate ownership.
- Centralized visibility versus sensitive-data exposure
- Gateway metrics and traces help diagnose request failures, but logging tokens, cookies, or request bodies can expose sensitive data.
When to reach for it
- Several backend services need one stable public API with consistent authentication, routing, and traffic policies.
- You need to reject abusive or excess requests before they reach expensive application work.
- Public API versions or client-specific routes must remain stable while internal service boundaries change.
- You need tenant-aware quotas, request limits, or traffic isolation across externally accessible endpoints.
How to bring it up in the interview
- Start with responsibilities, not a box on the diagram: name the routes and policies it owns, and keep business invariants in the services; expect questions about authorization boundaries.
- Draw the request path through the load balancer, gateway, and backend, then identify every synchronous dependency; interviewers may probe whether token or quota lookups dominate latency.
- Explain how replicas scale and which state is shared; expect a follow-up on why a per-instance rate limit is not a strict account-wide limit.
- State failure behavior explicitly: use bounded timeouts and concurrency, avoid unsafe retries, and choose fail-open or fail-closed behavior per policy rather than for the entire gateway.
- Describe safe configuration rollout, rollback, and monitoring of rejection rates, upstream errors, and tail latency; a gateway fleet can be healthy while a bad route breaks the API.
Common mistakes
- Treating an API gateway as mandatory for every application when a load balancer and a single backend would be sufficient.
- Assuming gateway authentication replaces resource-level authorization inside backend services.
- Putting inventory allocation, payment orchestration, or other business workflows in the gateway.
- Claiming that adding gateway replicas automatically provides globally consistent quotas or immediate session revocation.
- Enabling broad retries without accounting for duplicate writes, retry amplification, and ambiguous timeout outcomes.
Where it comes up
Interview questions where api gateway is part of a strong answer, and why.
- Design a distributed rate limiterThe gateway is a key enforcement location, making authenticated limit keys, shared counters, and limiter-outage behavior central design decisions.
- Design a login and authentication systemGateway credential checks and abuse controls help protect login flows, while session revocation requirements determine whether validation can remain local.
- Design a flash giveaway: 6 million burgers in an hourGateway admission control rejects excess claim traffic before it overwhelms the service that atomically allocates scarce inventory.
Questions people ask
What is the difference between an API gateway and a load balancer?
A load balancer distributes traffic across backend instances, while an API gateway typically adds API-aware routing and shared policies such as credential validation and quotas. Their capabilities overlap: an application-layer load balancer can also route by host or path. Explain the responsibilities you need rather than assuming they require separate products.
Is an API gateway the same as a reverse proxy?
An API gateway usually acts as a reverse proxy: it receives client requests and forwards them to backend servers. The gateway label emphasizes API management policies and routing across services, while reverse proxy describes the broader forwarding role.
Is an API gateway a single point of failure?
A single gateway instance is, but a replicated fleet can survive individual instance failures. Shared configuration, quota stores, identity services, and regional infrastructure can still create common failure modes. Replication must be paired with dependency isolation, safe rollouts, and a capacity plan for failures.