How it works
Clients connect to a public endpoint such as api.example.com, which resolves to the reverse proxy's entry point. The proxy accepts the connection and forwards requests to a backend over a separate connection; the client does not need to know which backend handled the request.
An HTTP reverse proxy can choose a destination using the hostname, path, or other request properties. For example, /users can go to the user service while /orders goes to the order service. If a service has several instances, the proxy can also load-balance requests among healthy instances.
The proxy can terminate TLS, meaning it decrypts the client's HTTPS connection. It can then establish a separate TLS connection to the backend. Terminating TLS at the proxy does not mean the internal network should automatically use plaintext.
The proxy forwards responses back to clients and may also enforce request-size limits, timeouts, rate limits, or caching rules. These are configurable capabilities, not guarantees that every reverse proxy provides or enables. Forwarded client-IP and scheme headers must be sanitized and trusted only from known proxies.
For WebSockets or streaming responses, the proxy must support the protocol and avoid settings that interrupt or buffer the stream. An established WebSocket stays attached to its selected backend; moving that client to another backend requires a reconnect and application-level recovery.
A worked example
Routing a chat service through a reverse proxy
Suppose a chat service expects 60,000 concurrent WebSocket connections across six chat servers. A redundant reverse-proxy tier terminates client TLS and distributes new connections, aiming for roughly 10,000 per backend while monitoring actual load. Each proxy is sized for open sockets and bandwidth, not just HTTP requests per second. A connection remains attached to its selected chat server. During a deployment, the proxy stops assigning new connections to a draining server; existing connections get a grace period before clients reconnect with backoff. Clients resume from their last acknowledged message. Proxy failure still drops affected connections, so reconnect handling and durable message history remain application responsibilities.
Trade-offs to say out loud
- Centralized control versus a shared failure point
- A proxy simplifies routing and policy enforcement, but its data path and configuration need redundancy because failures can affect many services at once.
- Backend protection versus proxy resource limits
- Connection limits and bounded queues protect applications, but the proxy itself consumes CPU, memory, sockets, and network bandwidth.
- TLS offload versus a new trust boundary
- TLS termination centralizes certificate management while making the proxy responsible for decrypted traffic and secure backend connections.
- Automatic retries versus duplicate work
- Retrying a failed upstream request may improve availability, but a timeout does not prove the backend did nothing, so writes need explicit retry safety.
- Buffering versus streaming latency
- Buffering can shield backends from slow clients, but adds resource use and can delay server-sent events or other streamed responses.
When to reach for it
- You need one public endpoint that routes requests to several internal services or backend instances.
- You want a common place for TLS termination, request limits, or admission control before requests reach application servers.
- The design includes WebSockets or server-sent events, making connection limits, idle timeouts, and connection draining important.
- You need weighted routing for a canary release or a controlled migration between old and new backends.
How to bring it up in the interview
- Draw the request path and name the responsibility of each hop: client, reverse proxy, service, and storage. Avoid adding a separate load balancer and API gateway unless each has a distinct job.
- Separate the concepts: reverse proxying describes acting on behalf of servers, load balancing distributes traffic, and an API gateway adds API-focused policies. One component can perform all three roles.
- State timeout and retry behavior explicitly. Expect follow-ups about a backend committing a write before the proxy times out, and explain where idempotency prevents duplicate effects.
- Explain how the proxy tier survives failures and deployments: redundant instances, health checks, bounded queues, and connection draining. For WebSockets, expect questions about reconnect storms and restoring session state.
- Define the trust boundary: restrict direct backend access, sanitize forwarded headers, and decide whether proxy-to-backend traffic needs TLS or mutual TLS.
Common mistakes
- Confusing a reverse proxy, which represents servers, with a forward proxy, which makes outbound requests on behalf of clients.
- Treating reverse proxy, load balancer, and API gateway as mutually exclusive products rather than overlapping roles.
- Assuming adding proxy replicas automatically moves existing WebSocket connections or removes the need for application-level routing.
- Retrying every upstream failure without considering duplicate writes or retry amplification during an outage.
- Trusting client-supplied forwarding headers or assuming that hiding backend addresses is sufficient access control.
Where it comes up
Interview questions where reverse proxy is part of a strong answer, and why.
- Design a distributed rate limiterA reverse proxy is a key enforcement location, so a good design explains how requests are limited before reaching backends and what happens when the limiter fails.
- Design a chat app like WhatsAppWebSocket ingress requires explicit proxy behavior for long-lived connections, backend selection, idle timeouts, and draining during deployments.
- Design a login and authentication systemProtecting password-verification capacity requires admission controls at ingress, including a trustworthy way to derive client identity through the proxy.
Questions people ask
What is a reverse proxy?
A reverse proxy accepts requests on behalf of backend servers and forwards those requests to an appropriate backend. It can provide a stable public endpoint while internal servers change, and may also handle TLS, routing, and traffic limits.
What is the difference between a reverse proxy and a forward proxy?
A forward proxy acts on behalf of clients making outbound requests, such as clients accessing the internet through a corporate proxy. A reverse proxy acts on behalf of servers receiving inbound requests. The distinction is whose side of the connection the proxy represents.
Is a reverse proxy the same as a load balancer or API gateway?
No, but the roles overlap. A reverse proxy can forward traffic to just one backend; a load balancer distributes traffic among multiple backends. An API gateway commonly uses reverse proxying while adding API-focused features such as authentication checks, quotas, and request transformations.