# Design a semantic search system: system design interview question

> Document search by meaning: passage relevance, filtered vector search, fresh results, and safe model changes.

Source: https://sysround.com/questions/design-semantic-search-system
Updated: 2026-11-05


Difficulty: hard. Levels: Senior, Staff+.

## How the interviewer opens it

"Design a service that finds documents by meaning, not only by exact words. The service must support a large document collection and return fresh, relevant results."

## What to clarify first

- Search behavior
- Corpus size
- Traffic and read:write mix
- Embedding model
- Tenants and filters
- Search latency
- Freshness and deletes
- Availability and durability
- Relevance
- Scope boundaries

## Scope of a strong answer

Core requirements:

- Add, replace, and delete documents
- Find relevant documents and return useful text snippets
- Restrict each search to its tenant and requested filters
- Reflect document changes within the freshness target

Usually out of scope:

- Answer generation
- Model training
- Text extraction from files
- Access rules for individual documents
- Cross-region failover

## What interviewers probe

### Result quality

A document can cover many topics. A relevant passage can rank well while duplicate passages or weak document scores reduce result quality.

### Search at full scale

Tenant sizes and traffic differ. Filters can reduce a large tenant to a small set, which changes the best retrieval method.

### Results after document changes

Text, vectors, and document state change at different times. Retries can restore old text or deleted documents unless the service controls document versions.

### Changes to the embedding model

Different models produce incompatible vector spaces. A full corpus update takes time while document changes and searches continue.

## Common mistakes

- The design treats a vector database as the complete system and omits a durable document source.
- The estimates ignore passage counts or assume every query scans all tenants.
- The search selects a small global result set before tenant and date filters.
- The service returns multiple passages from one document as separate document results.
- The design assumes an asynchronous delete satisfies the delete acknowledgment rule.
- The index mixes vectors from incompatible models in one search space.

## The bar by level

- **Senior:** Quantifies tenant-scoped retrieval costs and embedding load. Chooses exact search or ANN from eligible set sizes and traffic. Handles versions, safe deletes, relevance tests, and a failed search node.
- **Staff+:** Defines freshness and safety contracts across stores. Designs model transitions with rollback. Sets tenant isolation and recovery budgets. Validates retrieval choices across tenant sizes, filter selectivity, and traffic concentration.

Practice this question with an AI interviewer: https://sysround.com/practice/new?problem=semantic-search


---
SysRound (https://sysround.com) is an AI practice tool for system design interviews. It has a voice or typed AI interviewer and a shared whiteboard. At the end, you get feedback on each skill and a model answer. The AI feedback can be wrong.