QTM-1006 - Secret Distribution via Plain Files (age asymmetric encryption, POSIX permission baselines)

Living Document Notice
Published 2026-09-15. The evolving architecture, revisions, and connected notes for this dispatch live in the Stax Digital Garden.

Secret Distribution via Plain Files (age asymmetric encryption, POSIX permission baselines)

Summary

Centralized secret management daemons introduce continuous network dependencies, complex cluster quorum consensus, and significant memory footprints. For small server fleets and edge nodes, asymmetric file encryption using age combined with POSIX filesystem permissions provides secure credential injection without operational daemon overhead.

The Operational Overhead of Secret Daemons

Enterprise infrastructure architectures frequently mandate central secret stores such as HashiCorp Vault, AWS Secrets Manager, or Doppler. While suitable for large cloud environments, running dedicated secret managers on modest bare-metal or single-node VPS installations creates significant operational hazards.

A distributed secret manager requires active consensus clusters, high resident memory, and uninterrupted network egress. If an edge node reboots during an upstream network partition, secret retrieval fails and dependent services cannot initialize. Furthermore, managing dynamic tokens, rotation leases, and unseal keys introduces extensive administrative complexity.

Quartermaster replaces secret daemons with flat, asymmetric encrypted files using age encryption. Node secrets are encrypted offline at deploy time using the node’s public key, committed directly into version-controlled repositories or deployment bundles, and decrypted on the node into an in-memory tmpfs volume upon boot.

Asymmetric Encryption Model with age

The encryption model relies on X25519 asymmetric cryptography. Each node generates a local host identity during initial provisioning:

1. The private key resides strictly on the node at /etc/quartermaster/keys/host.agekey with 0400 permissions.
2. The public key is recorded in the central repository inside the node manifest.
3. Operators encrypt configuration secrets using the target node’s public key. The encrypted .age file is safe for storage in plain directories.

# Generate node identity keypair on target node
age-keygen -o /etc/quartermaster/keys/host.agekey
chmod 0400 /etc/quartermaster/keys/host.agekey
 
# Extract public key for node manifest
age-keygen -y /etc/quartermaster/keys/host.agekey
# Output: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

Encrypting credentials for the node requires only the public key:

# Encrypt environment credentials for edge-lon-02
age -r age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p   -o secrets.env.age secrets.env

Filesystem Permission Baseline Matrix

Decrypted secrets never touch persistent solid-state storage. Quartermaster decrypts secrets directly into an ephemeral RAM disk mounted with strict filesystem mount flags.

Path	Filesystem Type	Mount Flags	Ownership	Octal Mode	Security Objective
/run/secrets	tmpfs	noexec,nosuid,nodev	root:root	0700	In-memory only; blocks execution and device creation
/run/secrets/*.env	tmpfs	Inherited	bosun:bosun	0400	Read-only access restricted strictly to service user
/etc/quartermaster/keys	ext4 / xfs	Standard	root:root	0700	Protects local asymmetric identity private keys
/etc/quartermaster/keys/host.agekey	ext4 / xfs	Standard	root:root	0400	Host private key immutable to non-root users

Host Key Revocation and Decommissioning

When a node experiences physical tampering, hardware retirement, or cluster eviction, operators must revoke its cryptographic identity immediately. Because secrets are encrypted to specific host public keys, decommissioning requires two atomic steps: securely destroying local disk keys and updating repository manifests to omit the retired recipient.

# Securely overwrite and delete host private key on decommissioned node
shred -u -z -n 5 /etc/quartermaster/keys/host.agekey
 
# Verify absence of key material in memory buffers
sync && echo 3 > /proc/sys/vm/drop_caches

Secret Injection and Decryption Pipeline

Quartermaster initializes node secrets via a systemd one-shot service executing prior to application service launch.

#!/bin/sh
set -eu
 
SECRETS_DIR="/run/secrets"
KEY_FILE="/etc/quartermaster/keys/host.agekey"
ENCRYPTED_SRC="/etc/quartermaster/secrets"
 
if [ ! -f "$KEY_FILE" ]; then
  echo "FATAL: Host private key $KEY_FILE not found" >&2
  exit 1
fi
 
# Ensure RAM-backed tmpfs exists
if ! mountpoint -q "$SECRETS_DIR"; then
  mkdir -m 0700 -p "$SECRETS_DIR"
  mount -t tmpfs -o noexec,nosuid,nodev,size=16M tmpfs "$SECRETS_DIR"
fi
 
# Decrypt each secret file into RAM
for FILE in "$ENCRYPTED_SRC"/*.age; do
  [ -e "$FILE" ] || continue
  BASENAME=$(basename "$FILE" .age)
  TARGET="$SECRETS_DIR/$BASENAME"
  
  age -d -i "$KEY_FILE" -o "$TARGET" "$FILE"
  chown bosun:bosun "$TARGET"
  chmod 0400 "$TARGET"
done
 
echo "SUCCESS: Secrets decrypted into $SECRETS_DIR"

Service Credential Ingestion

Applications load secrets directly through systemd environment file directives, bypassing shell environment leaks:

[Service]
Type=simple
User=bosun
Group=bosun
EnvironmentFile=/run/secrets/directus.env
ExecStart=/usr/local/bin/directus start

---

• Directus Target: quartermaster
• Garden Source Reference: MOC - Fleet Operations
• Garden Source Reference: MOC - Bosun PKM Tools
• Garden Source Reference: [QTM-1006 - Secret Distribution via Plain Files (age asymmetric encryption, POSIX permission baselines)](QTM-1006 - Secret Distribution via Plain Files (age asymmetric encryption, POSIX permission baselines))