# Design an ML experiment tracking platform: system design interview question

> Concurrent experiments produce long metric histories and large artifacts. Reliable comparisons require exact records of code, data, and environments.

Source: https://sysround.com/questions/design-ml-experiment-tracking-platform
Updated: 2026-11-05


Difficulty: hard. Levels: Senior, Staff+.

## How the interviewer opens it

"Design a platform that records configurations, metrics, datasets, and artifacts from machine learning experiments. Many teams run experiments at the same time and need reproducible comparisons."

## What to clarify first

- Scale
- Metric traffic and read:write ratio
- Latency
- Consistency and availability
- Durability and retention
- Reproducibility
- Artifacts and datasets
- Comparisons
- Team access
- Scope boundaries

## Scope of a strong answer

Core requirements:

- Create runs with configurations and exact references to code, environments, and datasets
- Record scalar metrics throughout each run
- Store artifacts and link them to runs
- Filter runs and compare their configurations and metrics

Usually out of scope:

- Schedule or execute training jobs
- Select model parameters automatically
- Create datasets or label data
- Deploy models or serve predictions

## What interviewers probe

### Metrics from concurrent runs

Retries, resumed runs, and delayed messages can duplicate points or change the apparent order of a metric history.

### Repeat an experiment

A configuration alone cannot reproduce a run when code, dependencies, or dataset contents change.

### Experiment comparisons

Users need fast comparisons across long histories. Metric names alone do not establish that results have the same meaning.

### Large artifacts and incomplete runs

Metadata and artifact bytes live in separate systems. Failures can leave incomplete uploads, missing files, or objects without references.

## Common mistakes

- Treat the platform as a training scheduler instead of a record of experiments
- Preserve a source commit but omit local edits or required untracked files
- Use mutable dataset paths or environment tags as exact input records
- Ignore duplicate metrics, resumed runs, or delayed messages
- Scan complete metric histories before reducing chart resolution
- Declare artifacts available before the service verifies their bytes

## The bar by level

- **Senior:** Defines metric identity, attempt semantics, and durable acknowledgement. Preserves exact source and environment records. Handles artifact publication failures and bounds comparison reads.
- **Staff+:** Defines guarantees across metadata, metrics, and artifacts. Quantifies query capacity, aggregate maintenance, backlog recovery, and storage cost. Resolves retention and isolation trade-offs with explicit failure behavior.

Practice this question with an AI interviewer: https://sysround.com/practice/new?problem=ml-experiment-tracking


---
SysRound (https://sysround.com) is an AI practice tool for system design interviews. It has a voice or typed AI interviewer and a shared whiteboard. At the end, you get feedback on each skill and a model answer. The AI feedback can be wrong.