Stax Digital Garden

OUT-1006 - Resident Memory Caps - Preventing RSS Bloat in Batch Vault Ingestion

3 min read

Living Document Notice
Published 2026-09-15. The evolving architecture, revisions, and connected notes for this dispatch live in the Stax Digital Garden.

Resident Memory Caps - Preventing RSS Bloat in Batch Vault Ingestion

Resident Memory Caps - Preventing RSS Bloat in Batch Vault Ingestion: Abstract monochrome emerald green phosphor CRT tiered polygon core with radial allocation sectors bounded beneath circular threshold ceiling ring

Summary

Parsing massive note archives often causes unbounded heap expansion when streaming allocators buffer entire document trees in memory. Monolithic parsers reading nested XML trees or recursive JSON structures can exhaust available system RAM, triggering operating system out-of-memory killers that crash adjacent batch workers.

Outrigger enforces strict resident set size (RSS) ceiling limits via operating system cgroups and job objects. Setting explicit memory boundaries isolates worker allocations, forcing parsers to stream document tokens through bounded memory buffers rather than materializing entire vaults into heap memory.

Enforcing Cgroup V2 Limits on Linux Workers

On Linux runners, Outrigger places each conversion worker into a dedicated cgroup v2 slice with strict limits configured on memory.max and memory.high. The high watermark triggers background kernel page reclamation before the hard maximum is reached, allowing memory-constrained parsers to flush transient string allocations without terminating.

# Provision dedicated cgroup slice for Outrigger worker
WORKER_CGROUP="/sys/fs/cgroup/outrigger/worker-$$"
mkdir -p "${WORKER_CGROUP}"
 
# Set hard ceiling at 256MB and throttle warning threshold at 200MB
echo "268435456" > "${WORKER_CGROUP}/memory.max"
echo "209715200" > "${WORKER_CGROUP}/memory.high"
 
# Bind worker process to the cgroup
echo $$ > "${WORKER_CGROUP}/cgroup.procs"

If a parser attempts to allocate beyond 256MB due to recursive entity expansion or deeply nested DOM trees, the Linux kernel invokes the cgroup out-of-memory killer targeting solely that specific worker process. The parent supervisor catches the SIGKILL termination, logs the byte offset of the offending note, and isolates the source file into quarantine storage without affecting neighboring workers.

POSIX Resource Limits via setrlimit

In containerized environments lacking root permissions to mount cgroup slices, Outrigger falls back to POSIX setrlimit system calls to cap address space (RLIMIT_AS) and data segment size (RLIMIT_DATA).

#include <sys/resource.h>
#include <stdio.h>
#include <stdlib.h>
 
void enforce_worker_memory_boundary(rlim_t max_bytes) {
    struct rlimit mem_limit;
    mem_limit.rlim_cur = max_bytes;
    mem_limit.rlim_max = max_bytes;
 
    if (setrlimit(RLIMIT_AS, &mem_limit) != 0) {
        perror("setrlimit RLIMIT_AS failed");
        exit(EXIT_FAILURE);
    }
}

When RLIMIT_AS is exceeded, subsequent malloc or mmap syscalls return ENOMEM. The Outrigger memory allocator intercepts ENOMEM, flushes open file buffers, and reports a structured exhaustion error code back to the orchestrator.

Memory Allocation Metrics Across Parsing Strategies

The table below compares heap allocation behavior between unconstrained whole-file parsing and Outrigger stream-bounded ingestion when processing a 12GB export archive containing 45,000 notes.

Processing ModelPeak RSS per WorkerPage Fault CountIngestion ThroughputWorker OOM Terminations
Unconstrained DOM Parser1,842 MB472,11014.2 MB/s18 crashes
Chunked Token Stream (1MB)48 MB12,43038.6 MB/s0 crashes
Outrigger Cgroup Cap (256MB)62 MB15,89037.1 MB/s0 crashes
Fallback RLIMIT_AS (256MB)58 MB14,21036.8 MB/s0 crashes

Stream-bounded parsing maintains flat RSS curves regardless of archive size, keeping memory footprints predictable during sustained batch execution.

Windows Job Object Memory Configuration

For Windows runtimes, Outrigger binds child processes to an anonymous Job Object with JOBOBJECT_EXTENDED_LIMIT_INFORMATION.

use windows::Win32::System::JobObjects::{
    SetInformationJobObject, JobObjectExtendedLimitInformation,
    JOBOBJECT_EXTENDED_LIMIT_INFORMATION, JOB_OBJECT_LIMIT_PROCESS_MEMORY,
};
use windows::Win32::Foundation::HANDLE;
use std::mem::size_of;
 
pub unsafe fn apply_windows_memory_ceiling(job: HANDLE, limit_bytes: usize) -> bool {
    let mut limits = JOBOBJECT_EXTENDED_LIMIT_INFORMATION::default();
    limits.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_PROCESS_MEMORY;
    limits.ProcessMemoryLimit = limit_bytes;
 
    SetInformationJobObject(
        job,
        JobObjectExtendedLimitInformation,
        &limits as *const _ as _,
        size_of::<JOBOBJECT_EXTENDED_LIMIT_INFORMATION>() as u32,
    ).is_ok()
}
  • Directus Target: outrigger
  • Garden Source Reference: [Failure Isolation - Why Conversion Pipelines Must Fail Closed on Ambiguous Syntax](Failure Isolation - Why Conversion Pipelines Must Fail Closed on Ambiguous Syntax), The Task Manifest Protocol – Explicit Authority in CI, MOC - Outrigger Protocol, MOC - Agentic Containment and Sandbox Boundaries, MOC - Adversarial Agent Containment, MOC - Bosun PKM Tools

Graph View