Stax Digital Garden

EMB-1004 - Deterministic Directory Indexes Without Machine Learning

2 min read

Living Document Notice
Published 2026-09-13. The evolving architecture and revisions for this dispatch live in the Stax Digital Garden.

Deterministic Directory Indexes Without Machine Learning

Deterministic Directory Indexes Without Machine Learning: Warm sepia-white phosphor P4/P20 vector CRT macro showing a hierarchical deterministic folder and file directory tree index

Commercial photo clouds scan family libraries with facial recognition algorithms and scene categorization classifiers that consume server resources and compromise privacy. Embers rejects biometric indexing in favor of fast, deterministic queries anchored in timestamp indices and geographic EXIF headers.

The Privacy Cost of Automated Vision Models

Modern photo management applications execute multi-gigabyte neural network models on host servers. These vision pipelines build vector embeddings of facial geometry and run automated object detection sweeps across every uploaded snapshot. In household environments, running persistent machine learning inference saturates host CPU cycles, drives up thermal output, and introduces biometric databases into private storage vaults.

Embers eliminates machine learning subsystems entirely. Media organization relies exclusively on verifiable metadata: UTC timestamp indexes, camera hardware tags, user-defined album relations, and physical EXIF coordinates.

Host System Architecture:
[Uploaded Media] ---> [EXIF Parser (libexif/sharp)] ---> [SQLite B-Tree Index]
                                                   |
(Zero Face Scanners) <------------------------------+
(Zero Neural Weights)
(Zero Cloud Inferences)

Chronological and Spatial Query Indexes

Organizing media chronologically requires composite indices on creation timestamps and album identifiers. The SQLite schema avoids opaque vector tables:

-- Schema definition for deterministic photo indexing
CREATE TABLE media_items (
    id TEXT PRIMARY KEY,
    album_id TEXT NOT NULL,
    original_filename TEXT NOT NULL,
    captured_at INTEGER NOT NULL,
    latitude REAL,
    longitude REAL,
    width INTEGER NOT NULL,
    height INTEGER NOT NULL,
    mime_type TEXT NOT NULL,
    file_size INTEGER NOT NULL,
    sha256 TEXT NOT NULL UNIQUE,
    FOREIGN KEY(album_id) REFERENCES albums(id) ON DELETE CASCADE
);
 
CREATE INDEX idx_media_album_chronology ON media_items (album_id, captured_at DESC);
CREATE INDEX idx_media_captured_at ON media_items (captured_at DESC);

Performance Comparison: Deterministic Versus Neural Ingest

MetricPyTorch / FaceNet PipelineEmbers Deterministic Pipeline
Ingest Speed (1,000 photos)14 minutes 30 seconds8.2 seconds
Peak RAM Consumption2,840 MiB48 MiB
Host CPU Core Utilization100% across 8 cores12% on 1 core
External Model Weight Dependencies1.8 GiB ONNX models0 bytes

The deterministic approach processes incoming archives at the raw I/O throughput speed of the storage device.

# Query chronological distribution for an album directly from terminal
$ sqlite3 /var/lib/embers/data.sqlite3     "SELECT strftime('%Y-%m', datetime(captured_at, 'unixepoch')) as month, count(*)      FROM media_items WHERE album_id = 'album-2026' GROUP BY month;"
2026-06|142
2026-07|318
2026-08|204
  • Directus Target: embers
  • Garden Source Reference: MOC - The Digital Necropolis and Cold Decadal Storage, MOC - Bosun PKM Tools
  • Garden Source Reference: deterministic-directory-indexes-without-machine-learning, exif-metadata, sqlite-indexing, zero-biometrics

Graph View