# Design a real-time speech translation system: system design interview question

> Live speech translation with incomplete sentences, recognition errors, caption changes, and audio that the system cannot retract.

Source: https://sysround.com/questions/design-real-time-speech-translation-system
Updated: 2026-11-05


Difficulty: hard. Levels: Senior, Staff+.

## How the interviewer opens it

"Design a system that translates live speech into another language. People can receive captions or translated audio. Keep the delay short enough for a conversation."

## What to clarify first

- Session behavior
- Scale
- Read and write mix
- Audio format
- Latency targets and measurement
- Incomplete speech and quality
- Consistency and availability
- Recovery and retention
- Languages and models
- Scope limits

## Scope of a strong answer

Core requirements:

- Accept a live audio stream from one speaker
- Show translated captions with provisional text and committed text
- Produce translated audio with a preset voice
- Recover from brief connection failures without replay of heard audio

Usually out of scope:

- Train speech or translation models
- Support conferences with several speakers
- Identify speakers or imitate their voices
- Match translated audio to video

## What interviewers probe

### Incomplete speech and corrections

Later words can change recognition and translation. Captions can change, but the listener cannot forget audio that already played.

### Conversation delay

Recognition, translation, and synthesis each add delay. A translation can also take longer to speak than its source.

### Stream failures

Recovery can change translation boundaries and wording. An audio offset cannot safely identify equivalent content in a new translation.

### Traffic bursts

Model capacity limits active streams, not just request counts. Long queues turn a live conversation into delayed playback.

## Common mistakes

- Treats a changing recognition hypothesis as final translated text.
- Waits for complete sentences and ignores continuous speech.
- Sends provisional text to speech synthesis without a correction policy.
- Measures server response time instead of listener delay.
- Removes difficult phrases or fallback sessions from latency results.
- Promises no repeated audio without stable source positions and retained recipient progress.

## The bar by level

- **Senior:** Defines commit rules and a complete latency budget. Specifies source-based recovery, partial playback behavior, and worker ownership. Handles model limits and audio backlog.
- **Staff+:** Defines testable quality and latency contracts for each language pair. Proves recovery safety despite changed translations. Quantifies regional capacity and states limits after recipient state loss.

Practice this question with an AI interviewer: https://sysround.com/practice/new?problem=real-time-speech-translation


---
SysRound (https://sysround.com) is an AI practice tool for system design interviews. It has a voice or typed AI interviewer and a shared whiteboard. At the end, you get feedback on each skill and a model answer. The AI feedback can be wrong.