# Design a cloud VM service like Amazon EC2: system design interview question

> VM control across zones: request order, scarce capacity, uncertain command results, and complete resource cleanup.

Source: https://sysround.com/questions/design-cloud-vm-service-like-amazon-ec2
Updated: 2026-11-05


Difficulty: hard. Levels: Senior, Staff+.

## How the interviewer opens it

"Design a control plane that creates, stops, and terminates virtual machines across availability zones. The service must recover from partial failures without resource leaks."

## What to clarify first

- Scale and request mix
- Latency
- VM lifecycle
- Placement and capacity
- Tenant quotas
- Correctness and retention
- Infrastructure command contract
- Infrastructure discovery and health
- Recovery deadline and capacity
- Scope boundaries

## Scope of a strong answer

Core requirements:

- Create a VM in a suitable availability zone
- Stop a VM and release its compute capacity
- Terminate a VM and delete its owned resources
- Report operation status and recover from partial failures

Usually out of scope:

- Build a hypervisor or an image service
- Purchase servers or expand physical capacity
- Support live migration or automatic recovery of guest workloads
- Implement restart, billing, public IP addresses, or shared disks

## What interviewers probe

### Operation completion

Acceptance precedes physical completion. A client retry or controller failure must not lose the request or create another VM.

### Zone and host capacity

Host reports become stale while concurrent requests consume capacity. Tenant quotas and physical capacity need separate guarantees.

### Overlapping infrastructure commands

Old commands can finish after newer requests. Database order alone cannot prevent an old command from recreating a resource.

### Recovery after controller failures

Unknown outcomes can leave resources behind. A fixed cleanup deadline requires bounded work, complete resource discovery, and reserved recovery capacity.

## Common mistakes

- The API treats VM creation as one synchronous call.
- The queue is the only record of an accepted request.
- The service releases capacity after a timeout without proof that compute is absent.
- The design assumes a worker lease blocks stale infrastructure commands.
- The design promises ten-minute cleanup without limits on backlog, retry delay, and recovery throughput.
- The service places a VM on another host while its original host may still run it.

## The bar by level

- **Senior:** Handles overlapping requests and unknown outcomes with durable resource intents and receiver generations. Separates quotas from host capacity. Produces a bounded cleanup plan with explicit throughput and retry delays.
- **Staff+:** Defines invariants across database failover, stale workers, and host partitions. Closes the resource set before terminal commands. Proves the recovery deadline from admission limits, audit rates, stage capacity, and retry schedules. Identifies which guarantees depend on infrastructure contracts.

Practice this question with an AI interviewer: https://sysround.com/practice/new?problem=virtual-machine-provisioning


---
SysRound (https://sysround.com) is an AI practice tool for system design interviews. It has a voice or typed AI interviewer and a shared whiteboard. At the end, you get feedback on each skill and a model answer. The AI feedback can be wrong.