How it works
A database maintains both its main data files and a log of changes. For example, setting account:42 to 900 produces a log record describing the update; the exact record format depends on the storage engine.
The key rule is about persistence order: the relevant log records must reach durable storage before the changed data pages do. Updating an in-memory buffer first is allowed; writing that buffer to persistent data files before its WAL is durable is not.
For a durable transaction commit, the database flushes the required log records, including its commit record, before acknowledging success. Writing to the operating system's page cache alone is not enough to survive a machine or power failure; the database needs a durability operation such as fsync, backed by storage that honors it.
The database writes changed data pages in the background and periodically creates checkpoints that advance the recovery starting point. After a crash, recovery uses the checkpoint and log to restore a valid state. Depending on the engine, this can involve redoing committed changes and undoing or otherwise excluding incomplete transactions.
Log records have ordered positions, often called log sequence numbers. Recovery uses these positions to track progress and avoid incorrectly applying changes twice. Old log segments become reclaimable only when recovery no longer needs them and any replicas, backups, or change-data-capture readers have met their retention requirements.
A worked example
Recovering an acknowledged key-value write
Suppose a key-value store receives 100 writes of roughly 1 KB each during a 5 ms batching window. It appends their records to its WAL and performs one durable flush for the batch before acknowledging the writes. Its in-memory table contains the new values, but the persistent table files have not yet been updated. The process then crashes. On restart, the store loads its last durable table state and replays the required WAL records, restoring the acknowledged values. If the whole disk is lost, that local WAL cannot help; surviving that failure requires another durable copy. A client that missed an acknowledgment still needs a retry-safe request because its write may already have committed.
Trade-offs to say out loud
- Commit latency versus throughput
- Waiting for a durable flush adds latency, while group commit lets several transactions share one flush at the cost of some batching delay.
- Additional writes
- Changes are recorded in the log and later incorporated into data files, adding storage traffic even though sequential log appends are efficient.
- Checkpoint work versus recovery time
- More aggressive checkpointing can reduce recovery work but competes with foreground traffic for storage bandwidth.
- Local durability versus failover durability
- A locally flushed WAL can survive a process crash but does not guarantee that a promoted replica contains every acknowledged write.
- Retention and disk pressure
- Slow replicas or change-data-capture readers can prevent log reclamation, so retention limits, monitoring, and recovery procedures matter.
- Weaker durability settings
- Acknowledging before the required flush can improve latency but allows recently acknowledged writes to disappear after a crash.
When to reach for it
- You are designing a durable key-value store or queue and need to explain how accepted writes survive crashes.
- The interviewer asks what happens if the server crashes after acknowledging a write but before updating its data files.
- You need to distinguish local crash recovery from safe replica promotion and machine-loss protection.
- You are discussing database change capture, replay, or migration and need an ordered change stream with explicit retention requirements.
How to bring it up in the interview
- Start with the acknowledgment contract: name the failures a successful write must survive, then state whether success requires a local flush, durable replica copies, or both.
- Describe the persistence ordering precisely: WAL before persistent data pages, not necessarily before an in-memory update. Expect a follow-up about what fsync actually guarantees.
- Walk through crashes before log flush, after durable commit but before acknowledgment, and after acknowledgment but before data-page flush. Explain why the middle case leaves the client uncertain and requires retry-safe operations.
- Separate WAL from replication and concurrency control. Interviewers may probe how replicas establish a safe commit position, or how transactions prevent conflicting updates; WAL alone answers neither question.
- Mention checkpointing and log retention when discussing operations. Be ready to explain recovery time, stalled consumers, and what happens when the log disk fills.
Common mistakes
- Treating an append to an application buffer or the operating system's page cache as a durable write.
- Saying WAL requires flushing all changed data pages before returning success, which defeats a major reason for using it.
- Assuming a local WAL prevents data loss when the entire machine or disk is lost.
- Claiming WAL provides isolation, exactly-once external side effects, or protection against duplicate client requests.
- Calling every append-only log a WAL: a message log may be the primary stored data, while a WAL exists to recover another storage structure.
Where it comes up
Interview questions where write-ahead log is part of a strong answer, and why.
- Design a distributed key-value storeWAL is central to explaining durable writes, storage-engine recovery, and the difference between local persistence and replicated durability.
- Design a distributed message queue like RabbitMQA durable queue needs a crash-recoverable record of accepted messages and queue-state changes before promising that work will not be lost.
- Design a distributed message log like KafkaDurable append, flush boundaries, and replay are central here, including distinguishing the primary message log from a separate storage-engine WAL.
Questions people ask
What is a write-ahead log in a database?
A write-ahead log is a sequence of change records persisted before the corresponding changes reach the database's main data files. It gives crash recovery enough information to restore a valid state even when those files were only partly updated.
What is the difference between WAL and replication?
WAL supports recovery from a durable log, while replication maintains copies across nodes. A database can ship WAL records to replicas, but the acknowledgment and failover rules determine whether those replicas preserve every acknowledged write.
Is write-ahead logging the same as event sourcing?
No. WAL is usually an internal storage-engine mechanism whose records can be reclaimed once recovery and retention requirements are satisfied. Event sourcing keeps application-level events as the authoritative history, with event schemas and replay behavior designed around business meaning.