How it works
An S3 object contains data and metadata and is addressed by a bucket and key, optionally with a version ID. In general purpose buckets, prefixes organize a flat key namespace rather than real directories. S3 is not a block device or a POSIX file system: applications normally replace an object rather than modify arbitrary bytes in place.
A regional endpoint accepts authenticated requests, evaluates access policies, and routes operations to the storage service. S3 manages the underlying placement, redundancy, and repair. Explain these responsibilities in an interview without claiming a specific internal hash ring, consensus protocol, or replica count that AWS does not publicly guarantee.
S3 Standard stores data redundantly across multiple devices in at least three Availability Zones within a Region. Other storage classes make different availability and retrieval trade-offs, including single-AZ placement or delayed retrieval. Regional redundancy does not automatically create a copy in another Region.
Successful object writes, overwrites, and deletes have strong read-after-write consistency, and LIST operations reflect completed changes. Updates to a single key are atomic, so readers do not see a partially overwritten object. This does not provide transactions across keys or an atomic transaction spanning S3 and your application database.
A common application flow is to authorize an upload, create a pending database record, and issue a presigned URL for direct upload to S3. A worker verifies the uploaded object, processes it, and marks the record ready. Keep the bucket private and deliver authorized content through signed access or a CDN rather than routing every byte through application servers.
What it gives you in a design
- Direct and multipart uploads
- Presigned requests let clients transfer data without giving them AWS credentials. Multipart upload supports parallel parts and retrying failed parts; the object becomes readable after successful completion.
- Strong consistency and conditional writes
- Strong reads and listings simplify workflows after an upload. Conditional writes using preconditions such as If-None-Match or If-Match can prevent accidental replacement or reject an update based on an outdated object.
- Versioning and retention controls
- Versioning preserves prior object versions and supports recovery from overwrites and ordinary deletes. Object Lock adds retention and legal-hold controls, but neither feature replaces correct authorization or a recovery plan.
- Storage classes and lifecycle policies
- Choose storage classes based on access frequency, availability needs, and retrieval latency. Lifecycle rules can transition or expire objects and remove abandoned multipart uploads; include retrieval charges and minimum storage durations in the trade-off.
- Event-driven processing
- Object events can trigger processing through services such as SQS, SNS, Lambda, and EventBridge. Treat event handling as asynchronous and retryable: S3 Event Notifications are designed for at-least-once delivery and are not guaranteed to arrive in order.
- Access control and encryption
- IAM policies, bucket policies, and S3 Block Public Access control exposure. New object uploads are encrypted at rest by default; SSE-KMS adds key-policy control and introduces KMS permissions, quotas, and costs.
- CDN integration and range reads
- S3 works as an origin for a CDN such as CloudFront, reducing repeated origin reads and moving delivery closer to users. HTTP range reads let clients fetch portions of large objects without downloading the entire object.
Scaling, replication and failure
S3 automatically scales storage and request handling, so applications do not manage storage shards or replica placement. AWS documents request-rate baselines per partitioned prefix, but scaling to a sudden new rate can take time and produce temporary 503 Slow Down responses. Use bounded concurrency, exponential backoff with jitter, and workload testing rather than assuming unlimited instantaneous throughput.
Separate byte storage from discovery and coordination. Store object keys, ownership, processing state, and searchable attributes in a database, and use S3 for the payloads. For popular media, put a CDN in front of S3 and use immutable, versioned object keys to avoid cache invalidation races.
Cross-Region Replication is asynchronous and must be explicitly configured. It does not make the source and destination a synchronously consistent global store, and existing objects require a separate replication approach such as Batch Replication. Define replication lag, destination access, failover routing, and acceptable data loss before presenting it as disaster recovery.
Distinguish durable storage from application availability. A regional outage, an authorization error, a KMS dependency failure, or an unavailable database can still make stored objects inaccessible to users. Retry transient failures, reconcile incomplete workflows, and configure cleanup for abandoned multipart uploads and unreferenced objects.
Limits and defaults worth knowing
Check the current documentation before relying on an exact value; many of these are configurable.
| Object key length | An object key can be up to 1,024 bytes in UTF-8, including prefixes and delimiters. |
|---|---|
| LIST page size | ListObjectsV2 returns at most 1,000 keys per response. Use continuation tokens for larger listings. |
| Multipart part count | A multipart upload supports up to 10,000 parts. Part sizing and upload concurrency must account for the expected final object size. |
| Published request-rate baselines | For general purpose buckets, AWS documents at least 3,500 PUT/COPY/POST/DELETE requests or 5,500 GET/HEAD requests per second per partitioned prefix. These are scaling baselines, not universal ceilings or latency guarantees. |
| Presigned URL expiration | SigV4 presigned URLs created with SDKs or the CLI can be valid for up to seven days when using suitable long-lived credentials. URLs signed with temporary credentials expire when those credentials expire, and policies can impose shorter limits. |
| Consistency boundary | Object updates are atomic per key, with strong consistency for object reads and listings. There are no atomic multi-key transactions, and caches or asynchronous replicas can still serve older data. |
Use it when
- The design stores large immutable or replace-as-a-whole payloads such as photos, videos, documents, exports, or backups.
- Clients should upload or download directly without consuming application-server bandwidth.
- The workload needs durable storage that grows independently of the metadata database.
- Media delivery benefits from a CDN, range requests, and versioned object URLs.
- Raw events or analytical files need inexpensive long-term retention with lifecycle-based tiering.
Avoid it when
- The primary workload needs low-latency updates to small records, secondary indexes, or transactions across related records.
- Applications require POSIX file locking, directory operations, or frequent in-place file modifications.
- The design requires synchronous cross-Region writes with zero replication lag.
- Archived data must always be immediately readable, but the proposed storage class has a restore step or retrieval delay.
What interviewers ask about it
Follow-ups you can expect once S3 is on your whiteboard, and the core of a strong answer.
- How do you keep database metadata and an S3 upload consistent?
- Use an explicit pending-to-ready state machine because a database transaction cannot atomically commit an S3 upload. Upload to a unique key, verify completion, and conditionally publish the metadata; use reconciliation and cleanup for uploads or records left behind by failures.
- Does S3 still have eventual consistency?
- Object reads, overwrites, deletes, and listings are strongly consistent after successful operations. Cross-Region Replication, event delivery, CDN caches, and some configuration changes have different propagation behavior, so identify the boundary rather than labeling all of S3 eventual or strongly consistent.
- How would you handle a large, unreliable client upload?
- Use multipart upload with retryable parts and retain the upload ID and completed-part information needed to resume. Verify checksums, complete the upload only after all required parts succeed, and configure lifecycle cleanup for abandoned uploads.
- How do you process object events without duplicate results?
- Make processing idempotent and identify the exact object version or use an immutable upload key. Record processing state with conditional updates, retry through a durable queue, and send persistent failures to a dead-letter queue; event arrival alone should not establish business readiness.
- How do you serve private files and revoke access?
- Keep the bucket private and authorize users before issuing short-lived presigned URLs or CDN signed access. A signed URL is a bearer capability until it expires or an applicable policy blocks it, so immediate per-user revocation may require a request-time authorization layer.
- What happens when the same object is overwritten concurrently?
- Each individual update is atomic, but blind concurrent writes can overwrite one another. Use immutable keys or conditional writes against the expected ETag, and use a database transaction if the conflict involves multiple objects or business records.
Common mistakes
- Calling S3 eventually consistent for ordinary object reads and listings, or assuming its strong consistency creates transactions across S3 and a database.
- Using bucket listings as the application's primary searchable index instead of maintaining metadata suited to its queries.
- Assuming an ETag is always an MD5 checksum, including for multipart uploads and all encryption configurations; use explicit checksum support when integrity matters.
- Treating regional durability, versioning, and asynchronous cross-Region replication as interchangeable guarantees of immediate recovery and availability.
Alternatives
- Google Cloud Storage or Azure Blob Storage
- Choose a comparable managed object store when the surrounding infrastructure, identity system, regional footprint, or commercial constraints favor another cloud. Compare consistency, replication, and lifecycle semantics explicitly.
- Amazon EFS or another managed file system
- Use a file system when applications need shared file access, directory semantics, and file-oriented updates rather than object API operations.
- Amazon EBS or another block store
- Use block storage for a database or application that needs an attached volume and frequent low-level updates. The application must manage its file system and data replication requirements.
- DynamoDB or PostgreSQL
- Use a database for small structured records, indexed queries, conditional business updates, or transactions. A common design combines the database for metadata with S3 for large payloads.
Where it comes up
Interview questions where S3 is a natural fit, and why.
- Design a video streaming service like YouTubeS3 stores original uploads and transcoded segments while asynchronous workers prepare content for CDN delivery.
- Design Google PhotosS3 holds private media objects while a metadata database coordinates upload readiness, image processing, and authorized sharing.
- Design Dropbox or Google DriveMultipart transfers and immutable object versions support large file uploads while a separate metadata layer handles conflicts and permissions.
- Design a chat app like WhatsAppAttachments belong in object storage with authorized delivery rather than inside the message database or live connection path.
- Design TikTokS3 supports the upload and media-processing pipeline, with publication gated on playable outputs and CDN readiness.
- Design a large data migration to the cloudS3 can stage bulk data or serve as a migration destination, requiring resumable transfer, integrity verification, and a separate cutover protocol.
Questions people ask
What is Amazon S3 architecture?
S3 exposes regional object storage through APIs organized around buckets and keys. AWS manages request routing, storage placement, redundancy, and repair, while applications choose access policies, storage classes, lifecycle rules, and replication settings. Its public guarantees matter more in an interview than guessing its proprietary internals.
Is Amazon S3 a database or a file system?
S3 is an object store, not a relational database or a POSIX file system. It supports operations on objects but does not provide general secondary indexes, joins, or multi-object transactions. Most application designs pair it with a database for metadata and coordination.
How do you use S3 in a system design interview?
Place large payloads in S3, keep searchable metadata and ownership in a database, and use direct uploads plus a CDN where appropriate. Explain how uploads become visible, how workers handle duplicate events, and how permissions apply to delivery. Discuss regional failure and replication only to the extent required by the system's recovery goals.