Living Document Notice
Published 2026-10-30. The evolving architecture, revisions, and connected notes for this dispatch live in the Stax Digital Garden.
Asset Shuttling and Blob Management
Summary
Storing high-resolution photography, audio recordings, and large PDF attachments inside Git repositories destroys repository performance. As binary files accumulate, packfiles inflate, clone times degrade, and workstation memory fills during routine diffing operations. The Tender daemon intercepts binary attachments dropped into vault folders, hashes their payloads, and offloads them to content-addressable storage.
The daemon swaps raw binary references in markdown documents with deterministic content hashes and cloud asset pointers. This keeps the text repository lightweight while ensuring that media assets remain available across distributed devices.
The Problem of Binary Assets in Git
Git tracks changes by calculating deltas between file revisions. While plain-text markdown files compress efficiently into Git packfiles, binary formats such as JPEG images or compressed PDF documents contain uncompressible byte streams. Adding or modifying binary files forces Git to store complete file copies in .git/objects/pack, rapidly expanding repository size.
A vault containing 1,000 markdown notes requires less than 15 megabytes of storage. Adding 200 mobile photographs can inflate that same repository beyond 800 megabytes. Clones across fresh workstations become slow, and mobile sync clients experience memory exhaustion when parsing large git trees.
+--------------------------------------------------------------------+
| Asset Shuttling Pipeline |
| |
| [ User Drops image.png ] ---> [ Vault /assets/image.png ] |
| | |
| v |
| ( Tender Interceptor ) |
| | |
| +----------------+----------------+ |
| | | |
| v v |
| [ Compute BLAKE3 ] [ Rewrite Note ] |
| Hash: a8f9...3b21 bosun-blob://... |
| | |
| v |
| [ Store in Blob Staging ] |
| (~/.cache/tender/blobs/) |
| | |
| v |
| ( Asynchronous Upload ) |
| | |
| v |
| [ Harbor Object Storage ] |
+--------------------------------------------------------------------+
The Tender daemon intercepts media files at ingestion, separating text revision history from binary blob storage.
BLAKE3 Content Hashing and Ingestion
When a user drops an asset into the vault directory, the Tender daemon detects the write via IN_CLOSE_WRITE. Instead of letting the binary file remain permanently in the Git tracking path, the daemon computes a BLAKE3 content digest:
use blake3::Hasher;
use std::fs::File;
use std::io::{Read, BufReader};
pub fn hash_asset_payload(path: &Path) -> Result<String, std::io::Error> {
let file = File::open(path)?;
let mut reader = BufReader::new(file);
let mut hasher = Hasher::new();
let mut buffer = [0u8; 65536];
loop {
let bytes_read = reader.read(&mut buffer)?;
if bytes_read == 0 {
break;
}
hasher.update(&buffer[..bytes_read]);
}
Ok(hasher.finalize().to_hex().to_string())
}BLAKE3 processes multi-megabyte payloads in parallel using SIMD vector instructions, computing digests at storage interface speeds. The resulting 256-bit hash serves as the immutable identifier for the blob.
The daemon stores the binary payload in a local staging directory outside the repository root:
~/.cache/bosun-pkm/blobs/<hash_prefix>/<full_hash>.bin
Path Rewriting and Markdown Hygiene
Once the binary payload is secured in local staging, the Tender daemon rewrites the corresponding markdown note link:
<!-- Before Ingestion -->
<!-- After Tender Path Rewriting -->
The custom URI scheme (bosun-blob://) isolates note content from physical storage paths. When rendering notes, local client plugins resolve the URI against local disk cache or fetch the object from remote Harbor edge runtime storage:
| Storage Tier | Location | Retention Policy | Access Latency |
|---|---|---|---|
| L1 Cache | Local Vault (.cache/blobs/) | Hot working set (LRU, 5GB max) | < 2 milliseconds |
| L2 Cache | Workstation Shared Cache | Pinned project assets | < 5 milliseconds |
| Cold Storage | Harbor / MinIO Object Store | Durable archive | 50 - 250 milliseconds |
This architecture keeps the primary Git repository clean of binary bloat. Team members clone repositories in seconds without downloading historical image revisions.
Local background workers upload staged blobs to remote object stores using presigned S3 PUT requests:
PUT /vault-assets/b3a8f9c2d7e10842e45f91c6e4e590218a42b10a9c8f6e1d2c3b4a5f6e7d8c9a HTTP/1.1
Host: harbor.internal.net
Content-Type: image/png
Content-Length: 4194304
x-amz-content-sha256: b3a8f9c2d7e10842e45f91c6e4e590218a42b10a9c8f6e1d2c3b4a5f6e7d8c9a
[ Binary Stream ]The object storage endpoint confirms ingestion with HTTP status code 200 OK or 204 No Content, prompting the local daemon to mark the staging record as synchronized.
- Directus Target: tender
- Garden Source Reference: Blob Offloading Engine, Directus Media Pipeline, Asset Shuttling Spec, MOC - Ingestion & Capture, MOC - Local-First Systems and Synchronization, MOC - Bosun PKM Tools, MOC - Harbor Ecosystem