# Delivery guarantees

> Delivery guarantees describe whether messages can be lost, repeated, or processed with exactly one committed effect when failures occur. In system design interviews, the key is to define what counts as delivery, explain when acknowledgments happen, and separate receiving a message from completing its business action.

Source: https://sysround.com/concepts/delivery-guarantees
Updated: 2026-11-05


## How delivery guarantees works

Start by naming the boundary: producer to broker, broker to consumer, or consumer to an external service. A broker accepting a notification does not mean the provider accepted it, and provider acceptance does not mean the recipient saw it. Each boundary needs its own success condition.

At-most-once delivery allows loss but avoids duplicate delivery within the stated boundary. For example, a consumer can acknowledge a message before processing it; if it crashes afterward, the work is lost. This can suit disposable live updates, but it does not mean every message will arrive.

At-least-once delivery uses durable storage, acknowledgments, and retries to avoid silently losing accepted work under the stated failure assumptions. A consumer acknowledges only after completing its work. If it commits the work and crashes before acknowledging, the broker can deliver the message again.

Timeouts create uncertainty, not proof of failure. With a visibility timeout, a queue makes unacknowledged work available again after a lease expires; the original worker might still be running. Set realistic timeouts, renew leases for long jobs, and assume duplicate processing can overlap.

Exactly-once processing means one committed effect within a defined scope, not necessarily one network transmission or one execution attempt. For a database-backed consumer, insert a stable event ID into a table with a unique constraint and apply the business update in the same transaction. A retry then finds the committed ID and skips the update; a separate deduplication check would leave a crash window.

End-to-end guarantees require every handoff to be reliable. Use an outbox when a database update must publish an event, and use a provider-supported idempotency key for retrying an external operation. If an external service supports neither idempotency nor outcome lookup, a timeout can leave an unresolved choice between risking duplication and risking omission.

## Worked example: A worker crashes after crediting an account

A queue carries event credit-742, which adds $25 to an account. A worker receives it under a 30-second visibility timeout. In one database transaction, it inserts credit-742 into a processed-events table with a unique constraint and adds $25 to the balance. The transaction commits, but the worker crashes before acknowledging the message. After the timeout, another worker receives credit-742. Its insert encounters the existing ID, so it skips the balance change and acknowledges the message. There were two deliveries but one committed credit. If the credit instead called an external payment provider, the database transaction alone would not protect it; the provider would need an idempotency key or a way to resolve uncertain outcomes.

## Trade-offs

- **At-most-once accepts missing work.** Avoiding retries after ambiguous outcomes reduces duplication risk but can discard work that never completed.
- **At-least-once requires duplicate-safe consumers.** Retries recover from failures but can repeat or concurrently execute business operations unless effects are idempotent.
- **Exactly-once effects require coordination.** Atomically committing progress and output, or deduplication state and effects, adds storage work and limits the scope of the guarantee.
- **Deduplication has a retention cost.** Event IDs must remain available for the supported retry and replay horizon, or an old duplicate can produce a new effect.
- **Retries consume capacity.** Backoff, jitter, and backpressure are needed so an outage does not turn accumulated retries into another overload.
- **Retry limits qualify the guarantee.** Retention expiry, retry caps, and dead-letter queues mean accepted work may stop progressing without explicit recovery or operator action.

## When to use it in a system design interview

- A queue or event pipeline must survive worker crashes without silently dropping accepted work.
- Client or service retries could create duplicate charges, notifications, jobs, or counter increments.
- A chat or command system needs durable delivery across disconnects and reconnects.
- An interviewer asks what happens if an operation succeeds but its acknowledgment is lost.
- Different data deserves different policies, such as durable chat messages versus disposable typing indicators.

## How to bring it up in the interview

- State the boundary and success condition first: “The queue provides at-least-once delivery to workers; the database transaction makes the account update duplicate-safe.”
- Walk through crashes before the effect, after the effect but before acknowledgment, and after acknowledgment; interviewers probe these windows to test whether the guarantee actually holds.
- Explain the stable operation ID, its uniqueness scope, and how the deduplication record commits atomically with the effect; expect questions about concurrent duplicates and replay after expiration.
- Separate broker durability from consumer processing: explain what a producer acknowledgment survives, then describe consumer acknowledgments or offset commits.
- Name the external-system limit explicitly, including provider idempotency, outcome lookup, and reconciliation rather than claiming a queue makes the whole workflow exactly once.

## Common mistakes

- Calling at-least-once delivery “exactly once” because duplicate delivery is uncommon.
- Acknowledging before durable completion while claiming accepted work cannot be lost.
- Writing a deduplication record separately from the business update, leaving a crash window that can lose or repeat the effect.
- Assuming a timeout means an operation failed, or that an expired worker lease stops the old worker.
- Treating delivery guarantees as ordering guarantees, or assuming a dead-letter queue means the business operation eventually succeeded.

## Interview questions where it comes up

- [Design a distributed message queue like RabbitMQ](https://sysround.com/questions/design-message-queue): Acknowledgments, visibility leases, durable queue state, and duplicate processing define the queue's delivery contract.
- [Design a notification system](https://sysround.com/questions/design-notification-system): Retries and uncertain provider outcomes force an explicit choice between missing notifications and duplicate sends.
- [Design a chat app like WhatsApp](https://sysround.com/questions/design-whatsapp): Durable messages, reconnect replay, and receipts require clear delivery boundaries and duplicate suppression.
- [Design a distributed message log like Kafka](https://sysround.com/questions/design-kafka): Producer acknowledgments and consumer progress commits determine loss and replay behavior across failures.
- [Design a distributed job scheduler](https://sysround.com/questions/design-job-scheduler): Worker crashes and lease expiry can repeat execution, so reliable dispatch must be separated from duplicate-safe job effects.
- [Design an ad click aggregator](https://sysround.com/questions/design-ad-click-aggregator): Duplicate delivery and replay must not inflate counts or billing totals.
- [Design a payment system](https://sysround.com/questions/design-payment-system): Ambiguous charge outcomes require retry-safe operations and reconciliation beyond queue-level delivery guarantees.
- [Design a system to run commands across a fleet](https://sysround.com/questions/design-fleet-command-system): Offline delivery, repeated commands, and uncertain machine outcomes make the distinction between delivery and execution central.

## FAQ

### What is at-least-once delivery?

At-least-once delivery retries unacknowledged messages, so a message can arrive more than once. Avoiding loss depends on durable storage and the system's failure, retention, and recovery assumptions. It does not guarantee that a permanently failing business operation will eventually succeed.

### What is the difference between at-most-once and at-least-once delivery?

At-most-once permits missing messages but avoids duplicate delivery within its scope. At-least-once retries uncertain deliveries, reducing loss at the cost of possible duplicates. Choose based on whether missing work or repeating it is more harmful, and whether repeated effects can be made safe.

### Is exactly-once delivery possible in distributed systems?

Exactly-once effects are possible within a defined transactional scope, even when messages are transmitted or processed repeatedly. For example, a transaction can commit both a database update and its deduplication record. That guarantee does not automatically extend to an unrelated external API.

## Related concepts

- [Message queues](https://sysround.com/concepts/message-queues)
- [Idempotency and idempotency keys](https://sysround.com/concepts/idempotency)
- [Exactly-once processing](https://sysround.com/concepts/exactly-once)
- [Transactional outbox pattern](https://sysround.com/concepts/outbox-pattern)


---
SysRound (https://sysround.com) is an AI practice tool for system design interviews: a voice or typed AI interviewer, a shared whiteboard, then feedback on every skill and a model answer. Its feedback is AI-generated and can be wrong.
