Write it down before you do it

Your app calls save(). It returns. The user sees “Saved.” Somewhere in that half-second, a promise got made: if the power dies right now, this survives. The interesting question is when, exactly, that promise became true, because it is almost never the moment you think.

Here’s the naive mental model: to save a change, the database walks to the right spot in the big data file on disk and writes it there. Simple, and slow, and, worse, dangerous. That “right spot” might be a random location in a multi-gigabyte file, and updating a real record often means touching several places at once: the row, an index, maybe a free-space map. If the machine loses power halfway through touching those several places, you don’t get “the old data” or “the new data.” You get a structure that’s half-updated and internally contradictory, a torn write, a corrupt index. The database can’t even trust itself when it comes back up.

Write-ahead logging is the fix, and the name says the whole thing: write the intention to a log before you touch the real data. Before the database modifies a single data page, it appends a compact record to the end of a sequential log file — “here’s the change I’m about to make” — and forces that to disk. Only after the log record is safely down does the database consider the change durable and tell you “Saved.” The actual data pages can be updated lazily, in the background, minutes later, in whatever order is efficient.

Two things make this work, and they’re worth holding onto. First, appending to the end of a sequential file is dramatically cheaper than scattering writes across a big file. The disk head (or the SSD’s controller) isn’t jumping around; it’s writing one growing stream. So you pay the “make it durable” cost once, on a cheap sequential write, instead of many times on expensive random ones. Second, a crash is now survivable by construction. When the database restarts, it reads the log from the last known-good checkpoint forward and replays it, re-applying any changes that made it into the log but hadn’t yet been written to the data pages. Changes that never reached the log are gone, but gone cleanly, as if they never happened, which is exactly the promise you wanted. Half-applied corruption isn’t on the menu.

Now the part that bites people. The whole guarantee rests on one system call actually reaching the physical medium: fsync. Writing to the log isn’t enough — a plain write usually just lands in the operating system’s page cache, in volatile memory, and returns “success” while the bytes are still in RAM. If the power dies there, your durable log wasn’t durable. fsync is the call that says don’t come back until it’s really on the disk. It is slow precisely because it’s honest, it’s waiting on physics.

And so the most common durability bug isn’t exotic. It’s someone, somewhere, deciding that fsync on every commit is too slow (it is slow!) and quietly relaxing it, batching flushes, or turning them off, to make the benchmark look better. That’s not a free speed-up. It’s a change to the meaning of “Saved.” You’ve moved the durability point from “the moment we acknowledged” to “sometime in the next few hundred milliseconds, probably,” and bought yourself a small window where the database will happily tell a user their data is safe when it’s still only in RAM. This can be a completely legitimate trade — plenty of systems accept losing the last fraction of a second of writes on a hard crash in exchange for throughput. The sin isn’t making the trade. The sin is making it without saying so, so that “Saved” means one thing to the user and another to the disk.

There’s a legitimate way to have both, and it’s worth knowing the name: group commit. Instead of flushing once per transaction, the database gathers all the transactions that piled up in the last tiny slice of time and flushes them together in a single fsync. Each individual writer waits a hair longer, but the expensive honest call gets amortized across many of them. You keep the real guarantee and reclaim most of the speed, which is usually the trade you actually wanted, made on purpose.

The takeaway that outlives the specifics: durability is a promise about a specific instant, and that instant is the flush, not the write. Any system that returns “Saved” faster than it can honestly flush has moved the promise without telling you where. Write-ahead logging is how you keep the promise cheaply, write the intention down, force that, and let the real work happen whenever. But it’s only as true as the fsync you didn’t skip.

Concepts, to look up: write-ahead logging and the ARIES recovery algorithm (Mohan et al.); the “D” (durability) in ACID; the OS page cache and what fsync/fdatasync actually guarantee; group commit as the amortization of flush cost; checkpointing and log replay on recovery.