Stax Digital Garden

REC-1009 - Sandboxing Headless Scrapers with Outrigger

4 min read

Living Document Notice
Published 2026-09-18. The evolving architecture, revisions, and connected notes for this dispatch live in the Stax Digital Garden.

Sandboxing Headless Scrapers with Outrigger

Sandboxing Headless Scrapers with Outrigger: Warm golden amber P20 vector CRT macro showing hexagonal isolation containment cell enclosing turbulent signal while passing calm laminar data streams

Summary

When scraping unstructured websites that rely entirely on client-side JavaScript rendering, static HTTP byte stream extraction is insufficient. Scrapers must spin up headless browser runtimes such as Chromium via Playwright. Running untrusted third-party JavaScript from commercial recipe sites inside an automated scraper introduces severe security risks, including browser sandbox escape vulnerabilities and local network discovery probes.

FreeMyRecipes uses the Outrigger Protocol isolation runtime to encapsulate headless scraping sessions. By restricting network egress, dropping Linux capabilities, and applying strict seccomp system call filters, Outrigger confines untrusted browser execution to temporary ephemeral containers.

Attack Vectors in Automated Web Scraping

Commercial food blogs host scripts from dozens of unverified advertising networks and header-bidding partners. These third-party script tags change dynamically on every page view.

An automated scraping worker that visits thousands of recipe URLs daily is exposed to malicious scripts attempting to fingerprint local systems, access cloud instance metadata services (such as 169.254.169.254), or exploit zero-day bugs in browser rendering engines.

Untrusted Food Blog URL
           │
           ▼
┌──────────────────────────────────────────────┐
│ Outrigger Isolation Runtime                  │
│                                              │
│ - Linux cgroups: Memory limit 512MB          │
│ - Linux namespaces: Network, PID, Mount      │
│ - Seccomp filter: Block raw sockets          │
│ - iptables: Block local LAN & 169.254.0.0/16 │
└──────────────────────────────────────────────┘
           │
           ▼
Playwright Chromium Instance (Ephemeral)

By enforcing strict namespace isolation, the scraper treats every target URL as untrusted hostile input.

Isolation Mechanism Comparison

The table below outlines the isolation properties provided by Outrigger compared to standard bare-metal execution and basic container setups.

Isolation StrategyLocal Network AccessFile System PersistenceSystem Call FilteringProcess Lifetime
Bare Metal PlaywrightUnrestricted (can scan LAN)Full host user accessUnrestrictedPersistent
Docker Container (--privileged)Host network access possibleEphemeral container rootStandard Docker profileContainer duration
Docker Container (Standard)Outbound bridge networkEphemeral container rootDefault seccomp profileContainer duration
Outrigger Isolation RuntimeBlocked; DNS filtered via proxyRead-only root with tmpfsWhitelist-only BPF seccompDestroyed immediately on page close

Outrigger Sandbox Configuration

Outrigger defines security boundaries using declarative JSON runtime profiles. The profile below restricts Chromium processes spawned for DOM extraction.

{
  "profile": "recipe-scraper-isolation",
  "limits": {
    "max_memory_mb": 512,
    "max_cpu_percent": 50,
    "timeout_seconds": 15
  },
  "network": {
    "allow_outbound_ports": [443],
    "deny_subnets": [
      "10.0.0.0/8",
      "172.16.0.0/12",
      "192.168.0.0/16",
      "169.254.169.254/32"
    ],
    "dns_servers": ["1.1.1.1"]
  },
  "filesystem": {
    "read_only": true,
    "tmpfs": ["/tmp", "/dev/shm"]
  },
  "seccomp": {
    "default_action": "SCMP_ACT_ERRNO",
    "allow_syscalls": [
      "read", "write", "openat", "close", "fstat",
      "mmap", "mprotect", "munmap", "brk", "rt_sigaction",
      "poll", "select", "futex", "epoll_wait", "epoll_ctl"
    ]
  }
}

Running Playwright Under Outrigger

When the FreeMyRecipes scraper encounters an SPA that cannot be parsed via raw HTTP streams, it invokes Playwright inside the Outrigger sandbox:

import { chromium } from 'playwright';
import { OutriggerSandbox } from '@outrigger/core';
 
export async function scrapeIsolatedRecipe(url: string): Promise<string> {
  const sandbox = new OutriggerSandbox({
    configPath: './outrigger-scraper.json'
  });
 
  const sandboxContext = await sandbox.spawnContext();
 
  try {
    const browser = await chromium.launch({
      args: [
        '--no-sandbox',
        '--disable-setuid-sandbox',
        '--disable-gpu',
        '--disable-dev-shm-usage',
        `--net-ns=${sandboxContext.netNsPath}`
      ]
    });
 
    const page = await browser.newPage();
    await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 10000 });
    
    // Extract JSON-LD script content or rendered DOM
    const jsonLdContent = await page.evaluate(() => {
      const script = document.querySelector('script[type="application/ld+json"]');
      return script ? script.textContent : null;
    });
 
    await browser.close();
    return jsonLdContent || '';
  } finally {
    await sandboxContext.destroy();
  }
}

Executing this sandbox ensures that any malicious exploit payload delivered by a rogue advertising network fails to break out of the temporary process boundary.

outrigger-cli run --profile recipe-scraper-isolation -- freemyrecipes extract https://example.com/spa-recipe
# [OUTRIGGER] Sandbox initialized in 8ms (PID: 49120)
# [OUTRIGGER] Scrape completed in 1.4s. Memory high-water: 142MB.
# [OUTRIGGER] Teardown complete. Cgroups destroyed.
  • Directus Target: freemyrecipes
  • Garden Source Reference: ops-1-scraper-sandboxing, outrigger-manifest, MOC - Data Liberation Workbenches, MOC - Culinary & Domain Workspaces, MOC - Outrigger Protocol

Graph View