How it works
In a sidecar-based mesh, each application instance runs with a nearby proxy, often in the same Kubernetes pod. Traffic interception sends service calls through these proxies without requiring each application to implement the same networking policies. A call typically travels from application A to A's proxy, then to B's proxy, and finally to application B.
The proxies form the data plane: they handle actual requests and connections. A separate control plane distributes service endpoints, routing rules, identity material, and policy. The control plane is not normally consulted for every request; proxies use locally held configuration.
For example, an order service calls an inventory service by its logical name. The outbound proxy chooses a healthy inventory endpoint using discovery information and a load-balancing policy. Routing rules can direct a small share of eligible traffic to a new inventory version.
Proxies can establish mutual TLS, where both ends authenticate using workload identities. A receiving proxy can enforce rules such as allowing the order service to call inventory while denying unrelated services. This does not replace application checks about which user may reserve a particular item.
For supported protocols, proxies can enforce timeouts, bounded retries, connection limits, and unhealthy-endpoint ejection. They can also record request counts, errors, latency, and spans. Applications still need to propagate tracing context across incoming and outgoing calls and enforce business-level correctness.
A sidecar is a common deployment model, not the definition of a service mesh. Other designs use shared node-level proxies or separate proxies for application-layer processing. These change isolation, resource costs, and failure boundaries, so describe the actual request path rather than assuming every mesh uses sidecars.
A worked example
Securing an inventory rollout without retrying reservations
Suppose an order service sends 1,000 inventory requests per second across 20 inventory instances. A sidecar-based mesh authenticates both services with mutual TLS and permits only approved workloads to call inventory. During a rollout, a routing rule sends 5% of requests, about 50 per second, to the new version. For read-only availability checks, configure at most one retry within a 200 ms overall deadline, and disable overlapping application retries. Reservation writes are not automatically retried unless the application supplies durable idempotency handling. Compare errors and tail latency between versions before increasing traffic. If the control plane becomes unavailable, existing proxies may retain these rules, but endpoint updates and certificate renewal remain concerns.
Trade-offs to say out loud
- Uniform policy versus operational complexity
- Centralized networking policy reduces repeated application code but introduces proxy upgrades, certificate rotation, configuration rollout, and another system to debug.
- Extra request processing
- Proxies consume CPU and memory and add processing to each call, so measure tail latency and resource use under realistic traffic rather than assuming negligible overhead.
- Resilience versus retry amplification
- Retries can hide transient failures but multiply downstream load, especially when both application clients and proxies retry independently.
- Consistent security versus shared failure risk
- Workload identity and mutual TLS simplify service authentication, but a bad authorization rule or failed certificate rotation can disrupt many services.
- Control-plane independence has limits
- Existing proxies may continue using their last configuration during a control-plane outage, while new workloads, endpoint changes, and certificate renewal can fail.
- Network visibility is not business visibility
- Proxy metrics explain transport behavior but cannot determine whether a payment succeeded or whether a retried write duplicated a business operation.
When to reach for it
- Many independently deployed services need the same identity, encryption, authorization, and traffic-management policies.
- Services use different languages, making shared client-library behavior difficult to maintain consistently.
- The requirements explicitly call for workload-to-workload authentication and centrally enforced communication rules.
- Operators need controlled traffic shifts between service versions and consistent request-level telemetry across internal calls.
- The organization can operate the additional infrastructure; for a few services, client libraries and simpler proxies may be sufficient.
How to bring it up in the interview
- Start with the requirement, not the product: explain which repeated networking concerns justify a mesh and why simpler application libraries or platform features are insufficient.
- Draw one complete request path and separate the data plane from the control plane; expect questions about what happens when either one fails.
- Define retry ownership, a maximum attempt count, and an overall deadline; interviewers may ask whether a timeout means the downstream operation never happened.
- Separate workload authentication from user authorization, and explain how identities, certificates, and access policies reach proxies.
- Discuss rollout and rollback of mesh configuration, proxy resource limits, and tail-latency measurement; do not present the mesh as free reliability.
Common mistakes
- Adding a service mesh merely because the design has microservices, without a concrete policy or operational need.
- Confusing a service mesh with an API gateway: a gateway primarily handles traffic entering a system, while a mesh primarily manages communication between workloads.
- Enabling automatic retries for non-idempotent writes or stacking proxy retries on top of application retries.
- Claiming the mesh provides distributed transactions, exactly-once effects, or application-level authorization.
- Assuming tracing becomes complete automatically or that a control-plane outage has no effect because proxies cache configuration.
Questions people ask
What is the difference between a service mesh and a sidecar proxy?
A service mesh is the overall system for managing communication between workloads, including its control plane and traffic-handling data plane. A sidecar proxy is one way to deploy that data plane alongside an application instance. Not every service mesh requires a proxy per instance.
What is the difference between a service mesh and an API gateway?
An API gateway typically manages client-facing entry points, including routing, authentication, and request limits. A service mesh primarily manages internal workload communication. They can coexist, and some capabilities overlap, but their placement and policy responsibilities differ.
Do microservices need a service mesh?
No. Service discovery, TLS, telemetry, and resilience policies can be implemented through client libraries and platform features. A mesh becomes useful when consistent cross-service policy is worth the added infrastructure and operating cost.