Stax Digital Garden

BSN-1019 - The Uncomfortable Trade-offs of Local-First Systems

4 min read

Living Document Notice
Published 2026-09-28. The evolving architecture, revisions, and connected notes for this dispatch live in the Stax Digital Garden.

The Uncomfortable Trade-offs of Local-First Systems

The Uncomfortable Trade-offs of Local-First Systems: Abstract monochrome amber phosphor CRT tripartite geometric tension triangle balancing metadata mesh locking polygon and file strata

Summary

Local-first architectures guarantee personal data ownership and offline accessibility, but introduce complex engineering penalties. Eliminating centralized database authorities forces client runtimes to handle distributed conflict resolution, file-locking contention, and directory traversal bottlenecks locally.

Proponents of local-first software often overlook these operational trade-offs. Storing knowledge graphs across local plain-text files creates measurable performance costs in CRDT metadata amplification, filesystem lock serialization, and directory scan overhead.

The Three Fundamental Local-First Penalties

Operating without a central database coordinator shifts synchronization and query burdens onto the user device. Systems engineering in this domain requires confronting three distinct technical limitations.

+-------------------------------------------------------------+
|               Local-First Engineering Penalties             |
+-------------------------------------------------------------+
                               |
        +----------------------+----------------------+
        |                      |                      |
        v                      v                      v
+---------------+      +---------------+      +---------------+
| CRDT Metadata |      | POSIX Locking |      | Flat-File I/O |
| Amplification |      |  Contention   |      |  Degradation  |
+---------------+      +---------------+      +---------------+
| State log can |      | Multi-process |      | Traversing    |
| grow 10x past |      | file access   |      | 50,000 files  |
| raw text size |      | blocks threads|      | exhausts disk |
+---------------+      +---------------+      +---------------+

First, conflict-free replicated data types (CRDTs) accumulate tombstone records and causal history identifiers. Over months of active editing, metadata volume can exceed the actual text document payload by an order of magnitude.

Second, POSIX file systems lack fine-grained row-level locking. When an editor UI and a background indexing daemon access the same Markdown file concurrently, process contention introduces thread stalls.

Third, querying flat-file directory trees lacks the optimization paths of clustered B-trees. Resolving global references across 50,000 text files requires reading tens of thousands of directory entries from disk.

Synchronization and Conflict Resolution Comparison

Managing concurrent revisions across disconnected devices requires selecting an appropriate synchronization primitive. The table below compares four conflict resolution approaches.

MechanismMemory FootprintNetwork Payload OverheadOffline Partition SafetyImplementation Complexity
State-based CRDTHigh (Retains tombstones)Large (Full state vectors)Deterministic convergenceHigh (Complex pruning rules)
Operation-based CRDTModerate (Causal log history)Compact (Delta operations)Requires reliable causal deliveryHigh (Reorder buffer logic)
Last-Write-Wins (LWW)Minimal (Single wall-clock stamp)Tiny (Timestamp + payload)Risk of silent data lossLow (Clock skew vulnerabilities)
Git Three-Way MergeLow (Snapshot tree deltas)Variable (Compressed packfiles)Manual conflict markersModerate (Process invocation cost)

While CRDTs offer mathematical convergence guarantees, garbage collection of dead tombstones in offline-capable systems remains an open distributed systems challenge.

POSIX File Locking Implementation

To avoid race conditions between background indexers and active user typing sessions, Bosun uses non-blocking advisory file locks via the fcntl system call.

#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
 
int acquire_read_lock(int fd) {
    struct flock fl;
    fl.l_type = F_RDLCK;
    fl.l_whence = SEEK_SET;
    fl.l_start = 0;
    fl.l_len = 0; // Lock entire file
 
    // Non-blocking lock attempt
    if (fcntl(fd, F_SETLK, &fl) == -1) {
        if (errno == EACCES || errno == EAGAIN) {
            return -1; // File locked by concurrent writer
        }
        return -2; // System call error
    }
    return 0; // Lock acquired
}
 
int release_lock(int fd) {
    struct flock fl;
    fl.l_type = F_UNLCK;
    fl.l_whence = SEEK_SET;
    fl.l_start = 0;
    fl.l_len = 0;
    return fcntl(fd, F_SETLK, &fl);
}

When an indexer encounters an active lock (EAGAIN), it postpones indexing that specific note and retries during a later idle cycle rather than blocking reader threads.

An Open Architectural Problem

A critical unsolved challenge in peer-to-peer local-first architectures is tombstone garbage collection across partitioned networks. If a device remains disconnected for six months while other nodes prune deleted records, reconnecting the stale device can cause deleted notes to resurrect as new documents.

Inspect active advisory locks across open note files in Linux:

cat /proc/locks | grep -E "(POSIX.*READ|POSIX.*WRITE)"
  • Directus Target: bosunpkm-blog
  • Garden Source Reference: MOC - Bosun PKM Engine, MOC - Bosun PKM Tools, MOC - Local-First Systems and Synchronization

Graph View