Redis Persistence Modes, in depth: RDB snapshots, AOF and fsync policies, multi-part AOF, fork costs, WAITAOF and choosing what you can afford to lose

Four modes and their loss windows

Redis offers no persistence, RDB snapshots, an append-only file (AOF), or both together. The useful way to compare them is the loss window: how many acknowledged writes can disappear if the process or machine dies at the worst moment.

Note the qualifier on everysec. The documentation says you may lose about one second of data; if the disk stalls so badly that fsync falls behind, the window can stretch. And appendfsync no hands flushing to the operating system, which on Linux normally writes dirty pages within about 30 seconds, depending on kernel tuning.

RDB: fork, copy-on-write and save points

An RDB snapshot is a compact binary file, dump.rdb by default, containing the whole dataset at one instant. Redis produces it by calling fork. The child process gets a copy of the parent's address space and writes every key to a temporary file, then atomically renames it over the old snapshot. The parent keeps serving clients and never does disk I/O for the snapshot.

Fork is cheap in principle because of copy-on-write: parent and child share physical pages until one of them modifies a page, at which point the kernel copies that page. Two costs remain. First, fork must copy the page tables, which takes time proportional to the dataset size and blocks the main thread; Redis reports the last one as latest_fork_usec in INFO. On a large instance on slow virtualised hardware, that pause can reach hundreds of milliseconds or more. Second, every page the parent writes during the snapshot is duplicated. A write-heavy workload can touch most pages during a long snapshot, so in the worst case memory approaches double the dataset size.

Snapshots are triggered by save points, lines of the form save seconds changes, meaning snapshot if at least that many changes happened within that many seconds; several lines can coexist. You can also run BGSAVE manually. SAVE does the same synchronously on the main thread and blocks every client, so avoid it on a live server. Check what your instance actually does with CONFIG GET save rather than assuming the shipped defaults.

If a background save fails, for example because the disk is full, Redis by default stops accepting writes (stop-writes-on-bgsave-error yes). That is deliberate: it turns a silent loss of durability into a loud error. Do not switch it off to make an alert go away; fix the disk.

AOF: a log of every write

With appendonly yes, Redis appends every command that changes the dataset to the append-only file, in the same RESP format clients send. On restart it replays the log to rebuild memory. The question is when the appended bytes become durable. Writing to a file only places them in the operating system's page cache; they survive a process crash but not a power loss or kernel panic until fsync forces them to the device.

The appendfsync setting chooses the trade-off. With always, Redis fsyncs before replying to the clients whose commands were in the batch; because it writes and fsyncs once per event-loop batch, concurrent writes share one fsync, a form of group commit, but throughput is bounded by the device's fsync latency. With everysec, the default and the recommended setting, a background thread fsyncs once a second. With no, the kernel decides. The same durability concepts apply to any write-ahead log; write-ahead logging covers them in general.

# redis.conf: a common durable-but-fast baseline
appendonly yes
appendfsync everysec
appenddirname "appendonlydir"        # Redis 7+: directory for the multi-part AOF
aof-use-rdb-preamble yes             # base file written in RDB format: faster rewrite and load
auto-aof-rewrite-percentage 100      # rewrite when the AOF doubles since the last rewrite
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes               # on restart, drop a half-written last command and continue
no-appendfsync-on-rewrite no         # keep fsyncing during rewrites (yes trades safety for latency)
save 3600 1 300 100 60 10000         # keep RDB snapshots too, for backups and fast restores
stop-writes-on-bgsave-error yes

Rewrites and the multi-part AOF

The log grows forever unless compacted. Incrementing a counter a hundred times leaves a hundred entries for one key. An AOF rewrite, triggered automatically by the size thresholds above or manually with BGREWRITEAOF, forks a child that writes the minimal set of data needed to rebuild the current state. It does not read the old log; it walks memory, like a snapshot.

Since Redis 7.0 the AOF is multi-part: a directory, named by appenddirname, holds at most one base file, one or more incremental files and a manifest that lists them. When a rewrite starts, the parent opens a new incremental file and keeps appending there, while the child writes a new base file. When the child finishes, Redis writes a temporary manifest naming the new base and the new incremental file and atomically swaps it in, then deletes the old files. If the rewrite fails, the old base and incrementals plus the new incremental still describe the full dataset. Before 7.0, the parent buffered writes in memory during the rewrite and wrote them twice, which cost memory and could stall at the end; the multi-part design removed that. With aof-use-rdb-preamble yes, the base file is in RDB format, giving fast loading with the AOF's small loss window for the tail.

Which file wins on restart, and the classic data-loss trap

When both modes are enabled, Redis loads the AOF on restart, because it is the most complete record. That rule creates the best-known way to lose a whole dataset. Suppose an instance has run with RDB only and has a good dump.rdb. Someone edits redis.conf to set appendonly yes and restarts. Redis now looks for the AOF, finds none, and starts with an empty dataset, and the next snapshot may then overwrite the good dump.

The documented procedure avoids this: back up dump.rdb, enable AOF on the live server with CONFIG SET appendonly yes so it builds the AOF from memory, wait until INFO persistence shows aof_rewrite_in_progress and aof_rewrite_scheduled at 0 with aof_last_bgrewrite_status ok, persist the change to the config file with CONFIG REWRITE, and only then restart and compare key counts.

Persistence is not replication, and vice versa

Replication copies writes to other servers asynchronously; it protects against losing a machine, not against a bug that deletes data, which replicates too. Persistence protects against process and power loss on one machine. Most production deployments want both. The replication mechanics are in database replication.

Two commands let an individual client ask for stronger guarantees on writes it cares about. WAIT blocks until a number of replicas have acknowledged receiving the client's previous writes. WAITAOF, added in Redis 7.2, takes numlocal, numreplicas and a timeout in milliseconds, and blocks until the client's previous writes have been fsynced to the local AOF and to the AOF of that many replicas, returning the counts actually reached. Numlocal can only be non-zero if AOF is enabled locally, and the documentation is explicit that neither command makes Redis strongly consistent: a failover can still lose writes.

import redis

r = redis.Redis(host="primary", port=6379)

def durable_set(key: str, value: str, replicas: int = 1, timeout_ms: int = 200) -> None:
    r.set(key, value)
    local, remote = r.execute_command("WAITAOF", 1, replicas, timeout_ms)
    if local < 1 or remote < replicas:
        # The write happened but is not as durable as requested: surface it, do not ignore it.
        raise RuntimeError(f"WAITAOF reached local={local} replicas={remote}")

Worked example: sizing a 24 GB instance

Consider an illustrative session-and-cart store with 24 GB of data on a 32 GB virtual machine, 20,000 writes per second, AOF everysec plus hourly RDB snapshots. Is it safe?

Memory first. A snapshot or rewrite of 24 GB takes a while to write; if writes touch a quarter of the pages during that time, copy-on-write duplicates about 6 GB, so peak usage approaches 30 GB plus fragmentation and client buffers. That is too close to 32 GB. Options: a larger machine, smaller shards, or fewer forks. Kernel settings matter too. Redis logs warnings when vm.overcommit_memory is not 1, because with strict overcommit accounting fork can fail on a large process even though copy-on-write would never need the full copy, and when transparent huge pages are enabled, because copying 2 MB pages instead of 4 KB pages inflates both the copy cost and the latency of writes during a fork. Follow both warnings.

Latency next. Measure latest_fork_usec under realistic load. If forks take 400 ms, every snapshot and rewrite is a 400 ms pause for all clients. Spreading data across more, smaller instances shortens each fork. Disk last: everysec needs a device that can absorb 20,000 writes per second of appends plus a full base file on rewrite; put the AOF on local SSD, not a network volume with unpredictable fsync latency.

Monitoring, backups and recovery

INFO persistence is the single most useful view. Alert on rdb_last_bgsave_status and aof_last_bgrewrite_status not ok, on aof_last_write_status not ok, on rdb_changes_since_last_save growing without bound, on latest_fork_usec beyond your latency budget, and on aof_delayed_fsync increasing, which counts times an fsync was still in progress when the next was due.

For backups, the RDB file is safe to copy while Redis runs, because it is written to a temporary name and renamed into place only when complete. Copy it off the machine regularly and test restores. Copying the AOF directory is safe only when no rewrite is in progress; the documented approach is to pause automatic rewrites with auto-aof-rewrite-percentage 0, confirm aof_rewrite_in_progress is 0, copy or hard-link the files, then restore the setting. The current redis.io documentation also describes a BACKUP command family, introduced in Redis 8.10, that produces a restorable base, incremental and manifest set without pausing rewrites; check that your version has it before planning around it.

On recovery, a truncated AOF, where the last command was half-written during a crash or a full disk, loads anyway with aof-load-truncated yes and a warning in the log. A corrupted AOF, with bad bytes in the middle, stops startup. Back up the file, run redis-check-aof without --fix first to see where the problem is, and only then decide whether to let it truncate everything after the bad offset, because that can discard a large amount of data.

Choosing a mode

For data structures and access patterns that shape these choices, see Redis data structures in practice, and for Redis as a stream backbone see Redis for streaming.

What to do next

1. Run CONFIG GET save, CONFIG GET appendonly and CONFIG GET appendfsync on every instance and write down the loss window each implies.
2. Decide the acceptable loss per dataset and change the configuration to match, enabling AOF on live servers with CONFIG SET and CONFIG REWRITE, never by editing the file and restarting.
3. Set vm.overcommit_memory to 1, disable transparent huge pages, and confirm peak memory during a fork stays well inside the machine.
4. Add alerts on the INFO persistence status fields, rdb_changes_since_last_save, latest_fork_usec and aof_delayed_fsync.
5. Copy RDB snapshots off the host on a schedule and run a timed restore test at least quarterly.
6. Use WAITAOF on the few writes that must survive a crash, and handle the case where it returns fewer acknowledgements than requested.