Stax Digital Garden

TND-1003 - Store-and-Forward on Local Disk

4 min read

Living Document Notice
Published 2026-10-28. The evolving architecture, revisions, and connected notes for this dispatch live in the Stax Digital Garden.

Store-and-Forward on Local Disk

Store-and-Forward on Local Disk: Deep crimson P22R ruby red vector CRT macro showing spiral queue buffer with segmented block sectors

Summary

Distributed synchronization protocols frequently assume that network connectivity is reliable or that offline changes can wait in RAM. Storing uncommitted mutations in volatile application memory exposes user data to power cuts, operating system panics, and process crashes. The Tender daemon relies on an on-disk write-ahead log structured for atomic append operations.

By pairing POSIX append semantics with binary segment files and validation checksums, the daemon commits state changes to local storage before processing them further. This guarantees deterministic recovery across unexpected hardware reboots without corrupting the local knowledge archive.

Write-Ahead Logging for Plain-Text Vaults

Database management systems use write-ahead logs to survive crashes. In contrast, personal knowledge management tools often leave synchronization state in memory or rewrite monolithic JSON tracking files on exit. When the host operating system terminates abruptly, unsaved edits and sync metadata vanish.

The Tender daemon applies database journaling principles to markdown vaults. The daemon records every file modification as a fixed-header binary entry in a rolling write-ahead log (wal.bin) before signaling external sync services.

+-----------------------------------------------------------------------+
|                       Binary Journal Entry Layout                     |
|                                                                       |
| 0x00: [ Magic Bytes: 4B ("TJRN") ]                                    |
| 0x04: [ Log Sequence Number: 8B (uint64) ]                            |
| 0x0C: [ Timestamp UTC: 8B (int64 microseconds) ]                      |
| 0x14: [ File Path Length: 2B (uint16) ]                               |
| 0x16: [ Payload Byte Length: 4B (uint32) ]                            |
| 0x1A: [ Operation Code: 2B (1=Put, 2=Delete, 3=Move) ]                |
| 0x1C: [ Checksum: 4B (CRC-32C of Path + Payload) ]                    |
| 0x20: [ UTF-8 File Path: Variable ]                                   |
| 0x??: [ Binary / Text Payload: Variable ]                             |
+-----------------------------------------------------------------------+

Each log entry begins with a 32-byte preamble. The fixed layout allows recovery scanners to inspect record boundaries without decoding the full payload.

POSIX Append Guarantees and fdatasync

Opening the journal file with the O_APPEND flag ensures atomic write operations at the operating system kernel level. When multiple internal worker threads write to the log, POSIX guarantees that the write offset updates atomically during the write system call:

int log_fd = open(
    "/var/lib/tender/wal.bin",
    O_WRONLY | O_CREAT | O_APPEND,
    0600
);

Simply calling write() does not guarantee that bytes reside on physical storage media; operating system page caches delay writes until dirty pages flush to disk. To guarantee durability without sacrificing unnecessary performance, the Tender daemon invokes fdatasync() instead of fsync():

SyscallFlushes Data BlocksFlushes Inode MetadataLatency Impact on NVMe
write()No (cached in RAM)No< 1 microsecond
fdatasync()YesOnly if file size changes0.2 - 1.5 milliseconds
fsync()YesYes (atime, mtime, ctime)1.0 - 4.0 milliseconds

Because the journal file length increases monotonically, fdatasync() flushes new data blocks and the required length metadata while omitting redundant file access timestamp updates.

Recovery Sequence After Crash

When the Tender daemon starts, it executes a deterministic crash recovery sweep before opening its IPC interface. It reads the journal from the beginning, validating record headers and checksums sequentially:

pub fn recover_journal(file: &mut File) -> Result<u64, JournalError> {
    let mut reader = BufReader::new(file);
    let mut last_valid_seq = 0;
    let mut valid_byte_offset = 0;
 
    loop {
        match read_record(&mut reader) {
            Ok(record) => {
                last_valid_seq = record.header.sequence_id;
                valid_byte_offset = reader.stream_position()?;
            }
            Err(JournalError::IncompleteWrite) | Err(JournalError::ChecksumMismatch) => {
                // Truncate torn write at the last clean boundary
                reader.get_mut().set_len(valid_byte_offset)?;
                break;
            }
            Err(JournalError::Eof) => break,
        }
    }
    Ok(last_valid_seq)
}

If the host machine lost power during an append operation, the file may terminate with a torn write. The recovery sweep detects the corrupt trailing record via CRC-32C verification failure, truncates the file back to the last valid byte offset, and marks that sequence number as the authoritative local high-water mark.

Replaying the validated entries reconstructs uncommitted transport tasks without requiring expensive full-vault re-indexing:

journal_replay: verified 142 valid records, truncated 18 torn bytes at offset 0x004A2F80

The daemon then transitions to live monitoring, appending new mutations immediately following the recovered offset.

  • Directus Target: tender
  • Garden Source Reference: Crash Consistent Logging, Write Ahead Logs, Local Journal Format, MOC - Ingestion & Capture, MOC - Local-First Systems and Synchronization, MOC - Bosun PKM Tools

Graph View