# Nginx for system design interviews > Nginx is a web server, reverse proxy, and load balancer that commonly sits between clients and application servers. In a system design, it can terminate TLS, route requests, serve static files, cache responses, and protect backends with traffic limits. Source: https://sysround.com/tech/nginx Updated: 2026-11-05 ## How it works Nginx uses a master process and multiple worker processes. The master reads configuration, manages workers, and coordinates reloads. Workers handle client traffic, so the master is not in the per-request data path. Each worker runs an event loop using operating-system facilities such as epoll on Linux or kqueue on BSD systems. Nonblocking network I/O lets a worker manage many connections without assigning a thread to every connection. CPU-heavy work and blocking operations can still stall progress, so event-driven does not mean unlimited concurrency. For HTTP traffic, Nginx selects a virtual server using the listening address and requested hostname, then matches a location to determine request handling. Configured modules perform operations such as access checks, rate limiting, content serving, and proxying. A proxied request opens or reuses an upstream connection, and Nginx forwards the response to the client. Workers have separate process memory, while selected state, such as rate-limit counters and cache metadata, can live in explicitly configured shared-memory zones. Cached response bodies typically live on disk. During a graceful configuration reload, new workers start with the new configuration while old workers finish existing work. ## What it gives you in a design - **Reverse proxying and TLS termination.** Nginx can terminate client TLS and route requests by hostname or path. Backend TLS is a separate configuration decision, including whether upstream certificates are verified. - **Upstream load balancing.** HTTP upstream groups support weighted round robin, least connections, IP hashing, and generic hashing, including consistent hashing. Choose based on request duration, backend capacity, and whether affinity is actually necessary. - **Response buffering and caching.** Response buffering can reduce the time a backend remains tied to a slow client, using memory and potentially temporary files. Proxy caching can eliminate repeated backend reads, but requires explicit cache keys, freshness rules, and controls for private content. - **Traffic admission controls.** Request-rate and connection limits help reject or delay excess traffic before it reaches applications. Shared-memory zones coordinate these limits across workers on one instance, not automatically across a fleet. - **Static file delivery.** Nginx can serve files directly without involving an application server, with support for efficient file transfer and HTTP caching headers. A CDN is usually a separate layer when geographic distribution and edge offload matter. - **Long-lived connections and stream proxying.** Nginx can proxy WebSocket connections when upgrade headers are configured correctly. The optional stream module provides TCP and UDP proxying, but HTTP path routing and HTTP caching do not apply to that layer. ## Scaling, replication and failure Scale within a host by tuning worker count, connection limits, operating-system file-descriptor limits, and resource budgets. Client sockets and upstream sockets both consume capacity. Estimate active requests, idle keepalive connections, long-lived connections, TLS CPU cost, and buffering needs rather than treating worker_connections as a throughput promise. Scale across hosts by placing multiple Nginx instances behind a load balancer or another traffic-distribution layer. Configuration, certificates, and upstream discovery must reach every instance. Local caches and rate-limit zones are not automatically replicated, so cache hit rates and admission behavior change as instances are added or removed. Open-source Nginx supports passive upstream failure detection based on observed request failures; NGINX Plus also offers active health checks. Failover applies to eligible subsequent requests or retry attempts, not transparent migration of an established connection. Retries need a bounded budget, and a lost response does not prove that a backend failed to perform a write. Nginx does not supply a distributed consistency protocol for application state. Keep authoritative state outside the proxy and decide explicitly whether cached responses may be stale. Graceful reloads reduce disruption, but host failures still drop connections, and long-lived sessions need reconnect behavior. ## Limits and defaults worth knowing - **Worker connections:** The documented worker_connections default is 512 per worker. This includes upstream connections as well as client connections, and effective capacity is also constrained by file-descriptor limits. - **Request body size:** client_max_body_size defaults to 1m. Larger bodies receive HTTP 413 unless the limit is increased; setting it to 0 disables this size check. - **Upstream connection timeout:** proxy_connect_timeout defaults to 60 seconds. It is configurable, though operating-system behavior can impose additional constraints. - **Upstream read timeout:** proxy_read_timeout defaults to 60 seconds and applies between successive read operations, not to the total response duration. - **Rate-limit rejection status:** limit_req_status defaults to HTTP 503. It is configurable; HTTP 429 is often a clearer choice for an API rate-limit response. ## Use it when - You need an HTTP entry layer for TLS termination and hostname- or path-based routing to application services. - You want straightforward load balancing across stateless backends without building routing into every client. - Public, repeatedly requested responses can be cached close to the application to reduce backend work. - You need local admission controls, request-size limits, or buffering to protect backend resources. - You need to proxy WebSockets or serve static assets alongside application traffic. ## Avoid it when - You need globally exact quotas, inventory allocation, or another transactional decision. Local proxy state cannot provide that guarantee. - You need durable message delivery, replay, or persistent work queues. Nginx is not a message broker. - You need worldwide edge caching and large-scale network-layer attack absorption. A single Nginx deployment is not a replacement for a CDN or managed protection service. - You need extensive dynamic service policy, identity-aware routing, and fleet-wide configuration management but do not want to build the surrounding control plane. ## What interviewers ask about it ### Why can Nginx handle many connections with relatively few workers? Workers use event-driven, nonblocking network I/O rather than dedicating a thread to each connection. Capacity still depends on file descriptors, memory, CPU, upstream connections, and how much work each request requires. ### What happens when an application server becomes unhealthy? Passive checks can temporarily exclude it after qualifying failures, subject to the upstream configuration. Retry only eligible requests within a bounded budget, and use application-level idempotency for writes because an uncertain response can hide a completed operation. ### Does adding Nginx instances preserve a global rate limit? No, a shared-memory rate-limit zone shares counters across workers of one instance, not across independent instances. A strict global limit needs coordinated state or a separate quota service, while local limits remain useful for protecting each proxy and its backends. ### How would you safely cache API responses? Cache only responses with a clear sharing and freshness policy, and make the cache key cover every input that changes the representation. Treat authorization, cookies, tenant boundaries, and invalidation explicitly; a high hit rate is not worth leaking private data. ### How do WebSockets change the capacity plan? A proxied WebSocket typically keeps both a client connection and an upstream connection open for the session. Budget file descriptors and connection memory, configure idle timeouts and heartbeats, and explain how clients reconnect after a proxy or backend failure. ### Can Nginx reload configuration without dropping traffic? A graceful reload starts new workers and lets old workers finish existing connections. Long-lived connections can keep old workers alive, and an explicitly configured shutdown timeout can bound draining at the cost of closing remaining connections. ## Common mistakes - Multiplying worker_processes by worker_connections and presenting the result as guaranteed client capacity, without accounting for upstream sockets or operating-system limits. - Assuming every Nginx deployment includes active health checks, advanced gateway features, or automatic service discovery without checking the edition, modules, and configuration. - Treating local caches and rate-limit counters as a replicated, globally consistent service. - Enabling broad retries or caching without considering duplicate writes, private responses, and stale data. ## Alternatives - **HAProxy:** A strong choice when the main requirement is dedicated TCP or HTTP load balancing with detailed traffic controls rather than static file serving and proxy caching. - **Envoy:** Often a better fit when dynamic configuration, service-to-service traffic policy, and integration with a service-mesh control plane are central requirements. - **Managed load balancer or API gateway:** Wins when reducing operational ownership matters more than low-level proxy control. Check the service's timeout, connection, routing, authentication, and pricing constraints. - **CDN:** Wins for geographically distributed delivery of cacheable content and reducing traffic reaching the origin. It often complements Nginx rather than replacing every origin-proxy function. ## Interview questions where it comes up - [Design a URL shortener like Bitly](https://sysround.com/questions/design-url-shortener): Nginx can route redirect traffic and cache eligible redirects when freshness, analytics, and link-update semantics are explicit. - [Design a distributed rate limiter](https://sysround.com/questions/design-rate-limiter): Its local request limiting provides a concrete starting point for explaining why per-instance protection differs from a distributed quota. - [Design a chat app like WhatsApp](https://sysround.com/questions/design-whatsapp): WebSocket proxying exposes connection capacity, idle timeout, backend routing, and reconnect trade-offs. - [Design a flash giveaway: 6 million burgers in an hour](https://sysround.com/questions/design-flash-sale): Nginx can reject excess ingress traffic, while the application still needs atomic claims and retry-safe allocation. ## FAQ ### What is Nginx architecture? Nginx uses a master process to manage configuration and worker processes. Workers run event loops that handle many network connections with nonblocking I/O. Modules provide request routing, proxying, caching, access control, and other behavior. ### Is Nginx a web server or a load balancer? It can be both. Nginx can serve files directly, reverse proxy requests to applications, and distribute requests across upstream servers. State which role it plays in your design rather than adding it as an unexplained box. ### What is the difference between Nginx and an API gateway? Nginx supplies many gateway building blocks, including routing, TLS termination, and traffic limits. A broader API gateway may also provide managed authentication integrations, API lifecycle tools, developer access management, and centralized policy distribution. The distinction depends on the product and modules, not just the proxy engine. ## Related concepts - [Reverse proxy](https://sysround.com/concepts/reverse-proxy) - [Load balancing](https://sysround.com/concepts/load-balancing) - [Caching strategies](https://sysround.com/concepts/caching-strategies) - [Rate limiting algorithms](https://sysround.com/concepts/rate-limiting-algorithms) - [TLS and HTTPS](https://sysround.com/concepts/tls) --- SysRound (https://sysround.com) is an AI practice tool for system design interviews: a voice or typed AI interviewer, a shared whiteboard, then feedback on every skill and a model answer. Its feedback is AI-generated and can be wrong.