How the interviewer opens it
“Design a system that tracks how datasets and columns depend on upstream data. Users must explore dependencies and assess the effect of a change as pipelines evolve.”
The prompt is vague on purpose. The first minutes are for you. Reduce the prompt to the core features and agree on the numbers before you draw.
What to clarify first
- Catalog size
- Traffic and read:write mix
- Latency and freshness
- Source capabilities
- Meaning of impact
- Current and historical views
- Identity
- Availability and consistency
- Durability and retention
- Scope limits
The interviewer has the answers. In a SysRound round, Alex gives these answers, as a real interviewer does.
Scope of a strong answer
Core requirements
- Record dataset and column dependencies from source metadata.
- Explore upstream sources and downstream dependencies.
- Identify datasets and columns that a proposed change can affect.
- Query current dependencies and dependencies at a past time.
Usually out of scope
- Track individual rows or data values.
- Build an analyzer for arbitrary program code.
- Execute pipelines or enforce deployment rules.
- Support multiple organizations with separate access rules.
What interviewers ask about
- Changes to pipelines
- Reports can arrive out of order. A new pipeline definition can remove dependencies or restore old logic.
- Column dependencies
- Column names can change. Filters and joins affect results without directly supplying output values. Some sources cannot describe their logic.
- Large dependency reports
- Shared columns can reach huge dependency sets. Pipeline loops and concurrent changes make complete reports difficult.
Common mistakes
- Treat the latest received event as the latest deployed definition.
- Keep old edges active forever and report dependencies that no longer exist.
- Treat absent column detail as proof of no dependency.
- Match columns only by name.
- Claim a complete impact result after a query reaches its work limit.
- Build a full SQL interpreter before defining the metadata contract.
The bar by level
- Senior (L5/E5)
- Defines deployment-time history and atomic revision publication. Handles late reports, removals, and unknown column detail. Bounds report work and calculates storage costs.
- Staff (L6+/E6+)
- Defines source authority and limits on completeness. Separates deployment history from execution provenance. Sets snapshot rules for large reports. Explains recovery and store partition choices.