Design an ML experiment tracking platform: system design interview question

How the interviewer opens it

“Design a platform that records configurations, metrics, datasets, and artifacts from machine learning experiments. Many teams run experiments at the same time and need reproducible comparisons.”

The prompt is vague on purpose. The first minutes are for you. Reduce the prompt to the core features and agree on the numbers before you draw.

What to clarify first

Scale
Metric traffic and read:write ratio
Latency
Consistency and availability
Durability and retention
Reproducibility
Artifacts and datasets
Comparisons
Team access
Scope boundaries

The interviewer has the answers. In a SysRound round, Alex gives these answers, as a real interviewer does.

Scope of a strong answer

Core requirements

Create runs with configurations and exact references to code, environments, and datasets
Record scalar metrics throughout each run
Store artifacts and link them to runs
Filter runs and compare their configurations and metrics

Usually out of scope

Schedule or execute training jobs
Select model parameters automatically
Create datasets or label data
Deploy models or serve predictions

What interviewers ask about

Metrics from concurrent runs: Retries, resumed runs, and delayed messages can duplicate points or change the apparent order of a metric history.
Repeat an experiment: A configuration alone cannot reproduce a run when code, dependencies, or dataset contents change.
Experiment comparisons: Users need fast comparisons across long histories. Metric names alone do not establish that results have the same meaning.
Large artifacts and incomplete runs: Metadata and artifact bytes live in separate systems. Failures can leave incomplete uploads, missing files, or objects without references.

Common mistakes

Treat the platform as a training scheduler instead of a record of experiments
Preserve a source commit but omit local edits or required untracked files
Use mutable dataset paths or environment tags as exact input records
Ignore duplicate metrics, resumed runs, or delayed messages
Scan complete metric histories before reducing chart resolution
Declare artifacts available before the service verifies their bytes

The bar by level

Senior (L5/E5): Defines metric identity, attempt semantics, and durable acknowledgement. Preserves exact source and environment records. Handles artifact publication failures and bounds comparison reads.
Staff (L6+/E6+): Defines guarantees across metadata, metrics, and artifacts. Quantifies query capacity, aggregate maintenance, backlog recovery, and storage cost. Resolves retention and isolation trade-offs with explicit failure behavior.

Sources · 1 reference

These records explain where the question appears. A company tag or candidate report does not guarantee a future interview question.

SysRound · 1 record

No company association
Design an ML experiment tracking platform
Reported date not specified
Source snapshot: Nov 12, 2026
View source

How the interviewer opens it

“Design a platform that records configurations, metrics, datasets, and artifacts from machine learning experiments. Many teams run experiments at the same time and need reproducible comparisons.”

The prompt is vague on purpose. The first minutes are for you. Reduce the prompt to the core features and agree on the numbers before you draw.

Scope of a strong answer

Core requirements

Create runs with configurations and exact references to code, environments, and datasets
Record scalar metrics throughout each run
Store artifacts and link them to runs
Filter runs and compare their configurations and metrics

Usually out of scope

Schedule or execute training jobs
Select model parameters automatically
Create datasets or label data
Deploy models or serve predictions

What interviewers ask about

Metrics from concurrent runs

Retries, resumed runs, and delayed messages can duplicate points or change the apparent order of a metric history.

Repeat an experiment

A configuration alone cannot reproduce a run when code, dependencies, or dataset contents change.

Experiment comparisons

Users need fast comparisons across long histories. Metric names alone do not establish that results have the same meaning.

Large artifacts and incomplete runs

Metadata and artifact bytes live in separate systems. Failures can leave incomplete uploads, missing files, or objects without references.

Common mistakes

Treat the platform as a training scheduler instead of a record of experiments

Preserve a source commit but omit local edits or required untracked files

Use mutable dataset paths or environment tags as exact input records

Ignore duplicate metrics, resumed runs, or delayed messages

Scan complete metric histories before reducing chart resolution

Declare artifacts available before the service verifies their bytes

The bar by level

Senior (L5/E5)

Defines metric identity, attempt semantics, and durable acknowledgement. Preserves exact source and environment records. Handles artifact publication failures and bounds comparison reads.

Staff (L6+/E6+)

Defines guarantees across metadata, metrics, and artifacts. Quantifies query capacity, aggregate maintenance, backlog recovery, and storage cost. Resolves retention and isolation trade-offs with explicit failure behavior.