# Design a contextual bandit system: system design interview question

> Context-based choices with delayed rewards, hard item restrictions, and limited evidence for policy changes.

Source: https://sysround.com/questions/design-contextual-bandit-system
Updated: 2026-11-05


Difficulty: hard. Levels: Senior, Staff+.

## How the interviewer opens it

"Design a service that chooses a recommendation or advertisement for each request. The service must learn from user responses and respect safety limits."

## What to clarify first

- Decision inputs
- Scale and API mix
- Reward
- Feedback delay
- Latency and policy updates
- Hard restrictions and retries
- Exploration and policy loss
- Consistency and availability
- Durability and retention
- Scope and privacy

## Scope of a strong answer

Core requirements:

- Choose one eligible item for each request
- Learn from confirmed impressions and attributed conversions
- Limit exploration and enforce current item restrictions
- Compare policy results against a concurrent baseline

Usually out of scope:

- Find candidate items
- Run an advertisement auction
- Resolve user identity
- Build the conversion attribution system
- Enforce restrictions when the caller later displays the item

## What interviewers probe

### Choices and retries

Each request has different candidates and context. Retries must preserve the original choice unless current restrictions forbid it.

### Late responses and model changes

Recent impressions lack complete rewards. Frequent model refreshes can create many comparisons before any comparison has enough mature evidence.

### Limits on live changes

Hard restrictions require immediate authority. Delayed conversion evidence cannot prevent all reward loss before a policy stop.

### Evidence for policy changes

The policy controls which outcomes become visible. Rare choices and repeated tests can make weak evidence appear decisive.

## Common mistakes

- Describes a bandit algorithm but omits the decision service, feedback path, and release process.
- Treats missing or recent feedback as a negative reward.
- Replays a forbidden item because the original decision already exists.
- Exposes every refreshed model before earlier trials have mature evidence.
- Confuses a low exploration rate with a safe policy release.
- Claims improvement without a fair comparison or valid error control across repeated tests and releases.

## The bar by level

- **Senior:** Defines exact choice probabilities and mature reward cohorts. Separates frequent model refreshes from fixed live trials. Specifies exposure caps, durable records, and failure behavior. Uses a valid loss test.
- **Staff+:** Defines the authorization boundary for hard restrictions. Controls statistical error across releases and repeated tests. Challenges dependence and action-support assumptions. Quantifies delayed harm exposure and explains recovery without changing trial meaning.

Practice this question with an AI interviewer: https://sysround.com/practice/new?problem=contextual-bandit


---
SysRound (https://sysround.com) is an AI practice tool for system design interviews. It has a voice or typed AI interviewer and a shared whiteboard. At the end, you get feedback on each skill and a model answer. The AI feedback can be wrong.