From 469df382b2a95cbda144080da9c278c785f6f2a2 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Thu, 23 Jul 2026 22:51:06 -0700 Subject: [PATCH 1/5] feat(groom): bwrap sandbox harness + key-broker with deterministic CI proofs (BE-4302) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 1 of the groom auto-builder sandbox: trusted assets that confine the untrusted agent step, plus a no-API-spend CI job proving every confinement property. No changes to groom.yml's agent steps (that is phase 2), so this is mergeable independently of PR #65. - .github/groom/agent-sandbox.sh — bwrap wrapper (fail-loud preflight; never runs the command unsandboxed) that jails an arbitrary command with a read-only /usr+/etc, tmpfs /tmp+HOME, the clone bound ro or rw-git-ro, explicit ro-files, one writable out-dir, a cleared env, and its own pid namespace. - .github/groom/broker.mjs — zero-dep node reverse proxy that holds the real ANTHROPIC_API_KEY on the host, strips inbound credentials, injects the real key, forwards only /v1/* to api.anthropic.com, and streams responses (SSE). - sandbox-tests job in test-groom-scripts.yml (the PR-gating test workflow) + tests/fake-upstream.mjs: asserts env scrub, FS confinement + tmpfs shadowing, both clone modes, pid isolation, and the broker contract — no claude, no key. - README: sandbox contract + loud-preflight guarantee under the groom docs. --- .github/groom/README.md | 72 +++++++++ .github/groom/agent-sandbox.sh | 173 ++++++++++++++++++++ .github/groom/broker.mjs | 80 ++++++++++ .github/groom/tests/fake-upstream.mjs | 48 ++++++ .github/groom/tests/sandbox-tests.sh | 192 +++++++++++++++++++++++ .github/workflows/test-groom-scripts.yml | 23 +++ 6 files changed, 588 insertions(+) create mode 100755 .github/groom/agent-sandbox.sh create mode 100644 .github/groom/broker.mjs create mode 100644 .github/groom/tests/fake-upstream.mjs create mode 100755 .github/groom/tests/sandbox-tests.sh diff --git a/.github/groom/README.md b/.github/groom/README.md index 22de4ee..3b4cec8 100644 --- a/.github/groom/README.md +++ b/.github/groom/README.md @@ -172,3 +172,75 @@ python3 .github/groom/ledger.py --repo owner/name --check "" ```bash python3 -m unittest discover -s .github/groom/tests -p 'test_*.py' -v ``` + +## The agent sandbox — `agent-sandbox.sh` + `broker.mjs` (BE-4302) + +The auto-builder (phase 3) runs an untrusted agent that writes code. These two +trusted assets confine that agent so a prompt-injected or misbehaving run cannot +read the runner's secrets, touch anything outside its clone, or exfiltrate the +API key — while still letting it edit its worktree and reach Anthropic. + +- **[`agent-sandbox.sh`](agent-sandbox.sh)** — a [bubblewrap](https://github.com/containers/bubblewrap) + (`bwrap`) wrapper that runs an arbitrary command inside an unprivileged jail: + + ```bash + agent-sandbox.sh --clone --clone-mode ro|rw-git-ro --out-dir \ + [--ro-file ...] [--env KEY=VALUE ...] -- + ``` + +- **[`broker.mjs`](broker.mjs)** — a ~50-line node-stdlib reverse proxy + (`node broker.mjs `) that holds the real key on the host and forwards the + jail's requests to it. + +### The sandbox contract (what the agent can and cannot see) + +| Surface | Inside the jail | +|---|---| +| `/usr`, `/etc` | read-only | +| `/tmp`, `/home/agent` (`HOME`) | fresh tmpfs — host `/tmp` is **shadowed**, not shared | +| the clone (`--clone`) | bound **at its real path**; `ro` = read-only, `rw-git-ro` = worktree writable but `.git` read-only | +| explicit `--ro-file`s | read-only, at their real paths | +| the out-dir (`--out-dir`) | the **only** writable host location (created on the host first) | +| host `$HOME` / `$RUNNER_TEMP` / `$GITHUB_WORKSPACE` / other repos | **invisible** | +| host process table | **invisible** (own pid namespace) | +| environment | **cleared** — only `HOME`, `PATH`, `TERM`, and each `--env KEY=VALUE`; nothing inherited from the host | +| network | shared (so the agent can reach the broker on loopback) | + +The `rw-git-ro` worktree write is exactly how the builder's patch is captured: the +agent edits tracked files, the wrapper's caller reads them back on the host +afterward, but the agent can never rewrite git history or `.git/config`. + +**The real API key never enters the jail.** The broker reads +`ANTHROPIC_API_KEY` from *its own* (host) environment, **deletes** any inbound +`x-api-key` / `authorization` header, injects the real key, and forwards only +`/v1/*` paths to `api.anthropic.com` — streaming the response through unbuffered +so SSE works. `GET /healthz` answers locally; anything not under `/v1/` is `404`. +It listens on `127.0.0.1` only, refuses to start with an empty key, and logs one +line per request — method + path + status, never headers or body. + +### The loud-preflight guarantee + +Before it runs anything, `agent-sandbox.sh` **proves the sandbox works or exits +non-zero** — it will **never** fall back to running the command unsandboxed. The +preflight installs `bubblewrap` if missing, installs an unconfined AppArmor +profile for `bwrap` when the runner sets +`kernel.apparmor_restrict_unprivileged_userns=1` (mirroring the runner image's own +podman workaround), and self-tests a real `bwrap` invocation. If that still fails +it drops the userns restriction and retests; if it *still* fails it emits +`::error::bwrap sandbox unavailable …` and exits non-zero. A broken sandbox stops +the run — it never silently degrades to no sandbox. + +### Tests — deterministic, no API spend + +[`tests/sandbox-tests.sh`](tests/sandbox-tests.sh) (run by the `sandbox-tests` job +in [`test-groom-scripts.yml`](../workflows/test-groom-scripts.yml)) asserts every +row of the contract above with `bash -c` as the sandboxed command — env scrub, FS +confinement + tmpfs shadowing, both clone modes, pid isolation — and points the +broker at a local fake upstream ([`tests/fake-upstream.mjs`](tests/fake-upstream.mjs)) +to prove key injection/stripping, the `/healthz` + non-`/v1` behavior, and SSE +pass-through. No `claude`, no API key, no spend. + +```bash +shellcheck -x .github/groom/agent-sandbox.sh .github/groom/tests/sandbox-tests.sh +bash .github/groom/tests/sandbox-tests.sh # Linux + unprivileged userns only +``` diff --git a/.github/groom/agent-sandbox.sh b/.github/groom/agent-sandbox.sh new file mode 100755 index 0000000..f370c8f --- /dev/null +++ b/.github/groom/agent-sandbox.sh @@ -0,0 +1,173 @@ +#!/usr/bin/env bash +# +# agent-sandbox.sh — run an arbitrary command inside a bubblewrap (bwrap) jail. +# +# This is the confinement harness for the groom auto-builder's agent step +# (BE-4302, phase 1). It gives an untrusted agent a network-connected shell that +# can ONLY see: a read-only /usr + /etc, ephemeral /tmp + $HOME, the target clone +# (read-only, or read-write worktree with a read-only .git), an explicit set of +# read-only files, and one writable out-dir. Everything else on the host — other +# repos, the runner's secrets, $HOME, $RUNNER_TEMP, $GITHUB_WORKSPACE, the host +# process table — is invisible. The real API key never enters the jail; the agent +# reaches Anthropic only through the broker (broker.mjs) on host loopback. +# +# Usage: +# agent-sandbox.sh --clone --clone-mode ro|rw-git-ro --out-dir \ +# [--ro-file ...] [--env KEY=VALUE ...] -- +# +# The preflight FAILS LOUD: if a working bwrap sandbox cannot be established on +# this runner image, the script exits non-zero and the command is NEVER run. It +# never falls back to running the command unsandboxed. + +set -euo pipefail + +die() { + echo "agent-sandbox: $*" >&2 + exit 2 +} + +# The base mounts used by BOTH the preflight self-test and the real invocation, +# minus the caller-supplied clone/ro-file/out-dir/env. `true` runs as the probe. +selftest() { + bwrap \ + --unshare-all --share-net \ + --ro-bind /usr /usr \ + --symlink usr/bin /bin \ + --symlink usr/lib /lib \ + --symlink usr/lib64 /lib64 \ + --symlink usr/sbin /sbin \ + --proc /proc \ + --dev /dev \ + --tmpfs /tmp \ + true 2>/dev/null +} + +# Establish a working unprivileged-userns bwrap sandbox or exit non-zero. Mirrors +# the runner image's own podman AppArmor workaround +# (actions/runner-images: images/ubuntu/scripts/build/install-container-tools.sh): +# Ubuntu 23.10+ ships kernel.apparmor_restrict_unprivileged_userns=1, which blocks +# the unprivileged user namespaces bwrap needs unless an unconfined AppArmor +# profile is installed for /usr/bin/bwrap. +preflight() { + # Fast path: already usable, do nothing (keeps repeated invocations quiet). + if command -v bwrap >/dev/null 2>&1 && selftest; then + return 0 + fi + + if ! command -v bwrap >/dev/null 2>&1; then + sudo apt-get install -y bubblewrap + fi + + local restrict=/proc/sys/kernel/apparmor_restrict_unprivileged_userns + if [[ -r "$restrict" && "$(cat "$restrict")" == "1" ]]; then + sudo tee /etc/apparmor.d/bwrap >/dev/null <<'PROFILE' +abi , +include +profile bwrap /usr/bin/bwrap flags=(unconfined) { + userns, + include if exists +} +PROFILE + sudo apparmor_parser -r -W /etc/apparmor.d/bwrap || true + fi + + if selftest; then + return 0 + fi + + # Last resort: drop the unprivileged-userns restriction outright and retest. + sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 || true + if selftest; then + return 0 + fi + + echo "::error::bwrap sandbox unavailable on this runner image — refusing to run the agent unsandboxed" + exit 1 +} + +main() { + local clone="" clone_mode="" out_dir="" + local ro_files=() envs=() cmd=() + + while [[ $# -gt 0 ]]; do + case "$1" in + --clone) [[ $# -ge 2 ]] || die "--clone needs a value"; clone="$2"; shift 2 ;; + --clone-mode) [[ $# -ge 2 ]] || die "--clone-mode needs a value"; clone_mode="$2"; shift 2 ;; + --out-dir) [[ $# -ge 2 ]] || die "--out-dir needs a value"; out_dir="$2"; shift 2 ;; + --ro-file) [[ $# -ge 2 ]] || die "--ro-file needs a value"; ro_files+=("$2"); shift 2 ;; + --env) [[ $# -ge 2 ]] || die "--env needs a value"; envs+=("$2"); shift 2 ;; + --) shift; cmd=("$@"); break ;; + *) die "unknown argument: $1" ;; + esac + done + + [[ -n "$clone" ]] || die "--clone is required" + [[ -n "$out_dir" ]] || die "--out-dir is required" + [[ ${#cmd[@]} -gt 0 ]] || die "a -- is required" + case "$clone_mode" in + ro | rw-git-ro) ;; + *) die "--clone-mode must be 'ro' or 'rw-git-ro' (got '${clone_mode:-}')" ;; + esac + [[ -d "$clone" ]] || die "clone path is not a directory: $clone" + + preflight + + # out-dir must exist on the host before it can be bound rw into the jail. + mkdir -p "$out_dir" + + local bwrap_args=( + --unshare-all --share-net --die-with-parent --new-session --clearenv + --ro-bind /usr /usr + --symlink usr/bin /bin + --symlink usr/lib /lib + --symlink usr/lib64 /lib64 + --symlink usr/sbin /sbin + --ro-bind /etc /etc + --proc /proc + --dev /dev + --tmpfs /tmp + --tmpfs /home/agent + --setenv HOME /home/agent + --setenv PATH /usr/local/bin:/usr/bin:/bin + --setenv TERM dumb + ) + + local e key val + if [[ ${#envs[@]} -gt 0 ]]; then + for e in "${envs[@]}"; do + [[ "$e" == *=* ]] || die "--env expects KEY=VALUE (got '$e')" + key="${e%%=*}" + val="${e#*=}" + bwrap_args+=(--setenv "$key" "$val") + done + fi + + # The clone is bound AT ITS REAL PATH so tool output paths match the host. + case "$clone_mode" in + ro) + bwrap_args+=(--ro-bind "$clone" "$clone") + ;; + rw-git-ro) + # Read-write worktree, but .git stays read-only: the agent may edit + # tracked files (the patch we capture) yet can never rewrite history + # or git config. Ordering matters — the rw clone bind first, then the + # ro .git overlay on top. + bwrap_args+=(--bind "$clone" "$clone" --ro-bind "$clone/.git" "$clone/.git") + ;; + esac + + local f + if [[ ${#ro_files[@]} -gt 0 ]]; then + for f in "${ro_files[@]}"; do + bwrap_args+=(--ro-bind "$f" "$f") + done + fi + + bwrap_args+=(--bind "$out_dir" "$out_dir" --chdir "$clone") + + # stdout/stderr pass through to the host shell; the caller redirects stdout + # on the HOST side to capture any exec JSON out of the agent's reach. + exec bwrap "${bwrap_args[@]}" -- "${cmd[@]}" +} + +main "$@" diff --git a/.github/groom/broker.mjs b/.github/groom/broker.mjs new file mode 100644 index 0000000..30897c6 --- /dev/null +++ b/.github/groom/broker.mjs @@ -0,0 +1,80 @@ +// broker.mjs — a zero-dependency reverse proxy that holds the real Anthropic key +// so a sandboxed agent never sees it (BE-4302, phase 1). +// +// node broker.mjs +// +// The agent inside the bwrap jail (agent-sandbox.sh) talks to this broker on host +// loopback with NO key of its own; the broker strips any inbound credential, +// injects the real ANTHROPIC_API_KEY (read from its own env, never the jail's), +// and forwards to api.anthropic.com. The key lives only in the host process; the +// jail can spend against it but can never read it. +// +// Contract: listens on 127.0.0.1 only; forwards only /v1/* paths; deletes inbound +// x-api-key / authorization before adding the real one; streams responses through +// unbuffered so SSE works; logs method + path + status ONLY (never headers/body). +// +// BROKER_UPSTREAM_HOST / BROKER_UPSTREAM_PORT override the upstream target for +// tests only; production leaves them unset and pins api.anthropic.com:443. + +import http from 'node:http'; +import https from 'node:https'; + +const port = Number(process.argv[2]); +if (!Number.isInteger(port) || port <= 0 || port > 65535) { + console.error('broker: usage: node broker.mjs '); + process.exit(1); +} + +const KEY = process.env.ANTHROPIC_API_KEY; +if (!KEY) { + console.error('broker: ANTHROPIC_API_KEY is empty — refusing to start'); + process.exit(1); +} + +const UPSTREAM_HOST = process.env.BROKER_UPSTREAM_HOST || 'api.anthropic.com'; +const UPSTREAM_PORT = Number(process.env.BROKER_UPSTREAM_PORT || 443); + +const server = http.createServer((req, res) => { + // Local health check — never touches upstream, never spends. + if (req.method === 'GET' && req.url === '/healthz') { + console.log(`${req.method} ${req.url} 200`); + res.writeHead(200, { 'content-type': 'text/plain' }); + res.end('ok\n'); + return; + } + + // Only the Messages/Anthropic API surface is proxied; everything else is denied. + if (!req.url.startsWith('/v1/')) { + console.log(`${req.method} ${req.url} 404`); + res.writeHead(404, { 'content-type': 'text/plain' }); + res.end('not found\n'); + return; + } + + const headers = { ...req.headers }; + delete headers['x-api-key']; + delete headers['authorization']; + headers.host = UPSTREAM_HOST; + headers['x-api-key'] = KEY; + + const upstream = https.request( + { host: UPSTREAM_HOST, port: UPSTREAM_PORT, method: req.method, path: req.url, headers }, + (upRes) => { + console.log(`${req.method} ${req.url} ${upRes.statusCode}`); + res.writeHead(upRes.statusCode, upRes.headers); + upRes.pipe(res); // stream through — no buffering, so SSE passes intact + }, + ); + + upstream.on('error', () => { + console.log(`${req.method} ${req.url} 502`); + if (!res.headersSent) res.writeHead(502, { 'content-type': 'text/plain' }); + res.end('upstream error\n'); + }); + + req.pipe(upstream); // forward the request body streaming too +}); + +server.listen(port, '127.0.0.1', () => { + console.log(`broker listening on 127.0.0.1:${port} -> ${UPSTREAM_HOST}:${UPSTREAM_PORT}`); +}); diff --git a/.github/groom/tests/fake-upstream.mjs b/.github/groom/tests/fake-upstream.mjs new file mode 100644 index 0000000..ee83bdc --- /dev/null +++ b/.github/groom/tests/fake-upstream.mjs @@ -0,0 +1,48 @@ +// fake-upstream.mjs — a tiny local HTTPS server standing in for api.anthropic.com +// in the broker tests (BE-4302). It never talks to the real API. The broker points +// at it via BROKER_UPSTREAM_HOST/PORT; the broker process runs with +// NODE_TLS_REJECT_UNAUTHORIZED=0 so it accepts this server's self-signed cert +// (test-only — production talks real TLS to api.anthropic.com). +// +// node fake-upstream.mjs +// +// It echoes back the x-api-key header it RECEIVED so the test can prove the broker +// injected the real key and stripped the caller's dummy one. On /v1/stream it emits +// a chunked SSE response to prove streaming survives the proxy. + +import https from 'node:https'; +import fs from 'node:fs'; + +const port = Number(process.argv[2]); +const key = fs.readFileSync(process.argv[3]); +const cert = fs.readFileSync(process.argv[4]); + +const server = https.createServer({ key, cert }, (req, res) => { + const received = req.headers['x-api-key'] || ''; + + if (req.url === '/v1/stream') { + // Chunked SSE: multiple writes with a gap so a buffering proxy would collapse + // them; a streaming proxy delivers all three data frames intact. + res.writeHead(200, { + 'content-type': 'text/event-stream', + 'x-echo-received-key': received, + }); + res.write('data: one\n\n'); + setTimeout(() => { + res.write('data: two\n\n'); + res.write('data: [DONE]\n\n'); + res.end(); + }, 50); + return; + } + + res.writeHead(200, { + 'content-type': 'text/plain', + 'x-echo-received-key': received, + }); + res.end(`received_x_api_key=${received}\n`); +}); + +server.listen(port, '127.0.0.1', () => { + console.log(`fake-upstream listening on 127.0.0.1:${port}`); +}); diff --git a/.github/groom/tests/sandbox-tests.sh b/.github/groom/tests/sandbox-tests.sh new file mode 100755 index 0000000..7033c7d --- /dev/null +++ b/.github/groom/tests/sandbox-tests.sh @@ -0,0 +1,192 @@ +#!/usr/bin/env bash +# +# sandbox-tests.sh — deterministic proofs of the agent-sandbox.sh confinement +# contract and the broker.mjs credential proxy (BE-4302, phase 1). +# +# Runs the wrapper's preflight and then asserts every confinement property with +# `bash -c` as the sandboxed command — NO claude, NO API key, NO spend. Requires a +# Linux host with unprivileged user namespaces (a GitHub `ubuntu-latest` runner); +# the wrapper's preflight installs bubblewrap + the AppArmor profile as needed. +# +# Each assertion's inside-command is written to exit 0 on success, so a green +# `bwrap` exit means the property held; the driver fails loud on any non-zero. +# +# shellcheck disable=SC2016 # inside-command snippets intentionally keep $VAR literal for the jail + +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/../../.." && pwd)" +SANDBOX="$ROOT/.github/groom/agent-sandbox.sh" +BROKER="$ROOT/.github/groom/broker.mjs" +FAKE_UPSTREAM="$ROOT/.github/groom/tests/fake-upstream.mjs" + +# Runnable outside GitHub Actions too — synthesize the runner env vars if absent. +: "${RUNNER_TEMP:=$(mktemp -d)}" +: "${GITHUB_WORKSPACE:=$ROOT}" + +work="$(mktemp -d "${RUNNER_TEMP%/}/sandbox-tests.XXXXXX")" +clone="$work/clone" +outdir="$work/out" +rofile="$work/allowed-ro.txt" + +# Host canaries that must be INVISIBLE from inside the jail. +home_canary="$HOME/agent-sandbox-canary-home.$$" +temp_canary="${RUNNER_TEMP%/}/agent-sandbox-canary-temp.$$" +ws_canary="${GITHUB_WORKSPACE%/}/agent-sandbox-canary-ws.$$" + +fake_pid="" +broker_pid="" +host_sleep_pid="" + +cleanup() { + [[ -n "$fake_pid" ]] && kill "$fake_pid" 2>/dev/null || true + [[ -n "$broker_pid" ]] && kill "$broker_pid" 2>/dev/null || true + [[ -n "$host_sleep_pid" ]] && kill "$host_sleep_pid" 2>/dev/null || true + rm -f "$home_canary" "$temp_canary" "$ws_canary" /tmp/canary + rm -rf "$work" +} +trap cleanup EXIT + +fail() { + echo "FAIL: $*" >&2 + exit 1 +} + +pass() { + echo "PASS: $*" +} + +# --- fixtures ---------------------------------------------------------------- + +mkdir -p "$clone" +( + cd "$clone" + git init -q + git config user.email t@t.local + git config user.name tester + echo tracked > tracked.txt + git add -A + git commit -qm init +) +echo "read-only-content" > "$rofile" +echo secret > "$home_canary" +echo secret > "$temp_canary" +echo secret > "$ws_canary" +echo secret > /tmp/canary + +# --- 1. environment scrub ---------------------------------------------------- +# Only FOO/HOME/PATH (+ TERM) inside; a host-exported canary is NOT injected. + +export HOSTSECRET=leaked-host-value +if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ + --env FOO=bar -- bash -c ' + dump=$(tr "\0" "\n" < /proc/self/environ) + echo "$dump" | grep -qx "FOO=bar" || { echo "FOO missing"; exit 1; } + echo "$dump" | grep -qx "HOME=/home/agent" || { echo "HOME wrong"; exit 1; } + echo "$dump" | grep -qx "PATH=/usr/local/bin:/usr/bin:/bin" || { echo "PATH wrong"; exit 1; } + if echo "$dump" | grep -q "HOSTSECRET"; then echo "HOSTSECRET leaked into jail"; exit 1; fi + exit 0 +'; then fail "environment scrub"; fi +unset HOSTSECRET +pass "environment scrub (FOO/HOME/PATH present, HOSTSECRET absent)" + +# --- 2. filesystem confinement + tmpfs shadowing ----------------------------- +# Host $HOME / $RUNNER_TEMP / $GITHUB_WORKSPACE canaries unreadable; host +# /tmp/canary shadowed by tmpfs; writes fail except to --out-dir. + +if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ + --ro-file "$rofile" \ + --env HOME_CANARY="$home_canary" --env TEMP_CANARY="$temp_canary" \ + --env WS_CANARY="$ws_canary" --env OUTDIR="$outdir" \ + --env CLONE="$clone" --env ROFILE="$rofile" -- bash -c ' + for f in "$HOME_CANARY" "$TEMP_CANARY" "$WS_CANARY" /tmp/canary; do + if cat "$f" >/dev/null 2>&1; then echo "leaked readable: $f"; exit 1; fi + done + grep -q "read-only-content" "$ROFILE" || { echo "explicit --ro-file not readable"; exit 1; } + if echo x > /usr/should-fail 2>/dev/null; then echo "wrote read-only /usr"; exit 1; fi + if echo x > "$CLONE/should-fail" 2>/dev/null; then echo "wrote read-only clone"; exit 1; fi + if echo x > "$ROFILE" 2>/dev/null; then echo "wrote read-only --ro-file"; exit 1; fi + echo captured > "$OUTDIR/proof.txt" || { echo "out-dir not writable"; exit 1; } + exit 0 +'; then fail "filesystem confinement"; fi +test -f "$outdir/proof.txt" || fail "out-dir write not visible on host" +pass "filesystem confinement (host canaries hidden, /tmp shadowed, writes gated to out-dir)" + +# --- 3. clone rw-git-ro: worktree write lands on host; .git stays read-only --- + +if ! "$SANDBOX" --clone "$clone" --clone-mode rw-git-ro --out-dir "$outdir" \ + --env CLONE="$clone" -- bash -c ' + echo "builder patch" > "$CLONE/patch-from-agent.txt" || { echo "worktree write failed"; exit 1; } + if echo x >> "$CLONE/.git/config" 2>/dev/null; then echo "wrote read-only .git/config"; exit 1; fi + exit 0 +'; then fail "clone rw-git-ro"; fi +grep -q "builder patch" "$clone/patch-from-agent.txt" 2>/dev/null \ + || fail "rw-git-ro worktree write not visible on host" +pass "clone rw-git-ro (worktree write captured on host, .git read-only)" + +# --- 4. pid isolation -------------------------------------------------------- + +sleep 300 & +host_sleep_pid=$! +if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ + --env HOSTPID="$host_sleep_pid" -- bash -c ' + if ls /proc | grep -qx "$HOSTPID"; then echo "host pid $HOSTPID visible in jail"; exit 1; fi + exit 0 +'; then fail "pid isolation"; fi +kill "$host_sleep_pid" 2>/dev/null || true +host_sleep_pid="" +pass "pid isolation (host pids invisible in jail /proc)" + +# --- 5. broker credential proxy ---------------------------------------------- +# Fake local HTTPS upstream stands in for api.anthropic.com; the broker runs with +# the real (fake) key and a test-only TLS bypass for the self-signed upstream. + +certdir="$work/certs" +mkdir -p "$certdir" +openssl req -x509 -newkey rsa:2048 -nodes \ + -keyout "$certdir/key.pem" -out "$certdir/cert.pem" \ + -days 1 -subj "/CN=localhost" >/dev/null 2>&1 || fail "could not generate test cert" + +real_key="sk-ant-TESTFAKE-broker-forwarding-proof" +up_port=8791 +broker_port=8790 + +node "$FAKE_UPSTREAM" "$up_port" "$certdir/key.pem" "$certdir/cert.pem" & +fake_pid=$! +ANTHROPIC_API_KEY="$real_key" \ + BROKER_UPSTREAM_HOST=127.0.0.1 \ + BROKER_UPSTREAM_PORT="$up_port" \ + NODE_TLS_REJECT_UNAUTHORIZED=0 \ + node "$BROKER" "$broker_port" & +broker_pid=$! + +ready="" +for _ in $(seq 1 50); do + if curl -fsS "http://127.0.0.1:$broker_port/healthz" >/dev/null 2>&1; then ready=1; break; fi + sleep 0.2 +done +[[ -n "$ready" ]] || fail "broker did not come up on 127.0.0.1:$broker_port" + +if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ + --env BROKERPORT="$broker_port" --env REALKEY="$real_key" -- bash -c ' + base="http://127.0.0.1:$BROKERPORT" + # real key injected, caller-supplied dummy stripped + body=$(curl -s "$base/v1/messages" -H "x-api-key: dummy") + echo "$body" | grep -q "$REALKEY" || { echo "real key not forwarded upstream: $body"; exit 1; } + if echo "$body" | grep -q "dummy"; then echo "caller dummy key leaked upstream"; exit 1; fi + # healthz served locally with 200 + code=$(curl -s -o /dev/null -w "%{http_code}" "$base/healthz") + [ "$code" = "200" ] || { echo "healthz not 200: $code"; exit 1; } + # non-/v1 path denied + code=$(curl -s -o /dev/null -w "%{http_code}" "$base/not-v1") + [ "$code" = "404" ] || { echo "non-/v1 not 404: $code"; exit 1; } + # chunked/SSE response streams through intact + stream=$(curl -sN "$base/v1/stream") + echo "$stream" | grep -q "data: one" || { echo "sse frame one missing"; exit 1; } + echo "$stream" | grep -q "data: two" || { echo "sse frame two missing"; exit 1; } + echo "$stream" | grep -q "\[DONE\]" || { echo "sse [DONE] missing"; exit 1; } + exit 0 +'; then fail "broker credential proxy"; fi +pass "broker credential proxy (key injected+stripped, healthz local, non-/v1 404, SSE streams)" + +echo "ALL SANDBOX TESTS PASSED" diff --git a/.github/workflows/test-groom-scripts.yml b/.github/workflows/test-groom-scripts.yml index 3cc9ae9..09a14b9 100644 --- a/.github/workflows/test-groom-scripts.yml +++ b/.github/workflows/test-groom-scripts.yml @@ -37,3 +37,26 @@ jobs: - name: Run groom ledger unit tests run: python3 -m unittest discover -s .github/groom/tests -p 'test_*.py' -v + + # Deterministic proofs of the agent-sandbox.sh confinement contract and the + # broker.mjs credential proxy (BE-4302, phase 1). No claude, no API key, no + # spend — every property is asserted with `bash -c` as the sandboxed command. + # Runs here (a PR-gating test workflow), NOT in ci-groom.yml: ci-groom.yml is the + # scheduled groom CALLER that deliberately never fires on a PR, so a job there + # could never be seen green on this change. shellcheck (repo convention) + node + + # openssl + curl are preinstalled on ubuntu-latest; the wrapper's preflight + # installs bubblewrap + the AppArmor profile the runner image needs. + sandbox-tests: + name: sandbox-tests + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6 + with: + persist-credentials: false + + - name: Shellcheck the sandbox scripts + run: shellcheck -x .github/groom/agent-sandbox.sh .github/groom/tests/sandbox-tests.sh + + - name: Prove sandbox confinement + broker proxy (no API spend) + run: bash .github/groom/tests/sandbox-tests.sh From b59cd67365892b4cf3e9c3c43f3e733127941179 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Thu, 23 Jul 2026 22:53:17 -0700 Subject: [PATCH 2/5] fix(groom): make sandbox-tests cleanup SC2015-clean (explicit if over A && B || C) --- .github/groom/tests/sandbox-tests.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/groom/tests/sandbox-tests.sh b/.github/groom/tests/sandbox-tests.sh index 7033c7d..5e22559 100755 --- a/.github/groom/tests/sandbox-tests.sh +++ b/.github/groom/tests/sandbox-tests.sh @@ -39,9 +39,9 @@ broker_pid="" host_sleep_pid="" cleanup() { - [[ -n "$fake_pid" ]] && kill "$fake_pid" 2>/dev/null || true - [[ -n "$broker_pid" ]] && kill "$broker_pid" 2>/dev/null || true - [[ -n "$host_sleep_pid" ]] && kill "$host_sleep_pid" 2>/dev/null || true + if [[ -n "$fake_pid" ]]; then kill "$fake_pid" 2>/dev/null || true; fi + if [[ -n "$broker_pid" ]]; then kill "$broker_pid" 2>/dev/null || true; fi + if [[ -n "$host_sleep_pid" ]]; then kill "$host_sleep_pid" 2>/dev/null || true; fi rm -f "$home_canary" "$temp_canary" "$ws_canary" /tmp/canary rm -rf "$work" } From 8ca79f332900f3c32a91e8d58cb791bc30d0e8e3 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Thu, 23 Jul 2026 23:01:37 -0700 Subject: [PATCH 3/5] fix(groom): read /proc/self/environ via cat, not < redirect (empty in jail) The `< /proc/self/environ` redirect resolves /proc/self in the pre-exec forked shell and reads empty inside the bwrap jail, so the env-scrub assertion saw no FOO. Reading it with `cat` (which opens post-exec, /proc/self = cat) returns the real environ. Verified end-to-end in a privileged ubuntu:24.04 container: all sandbox assertions + broker proxy pass, shellcheck clean. --- .github/groom/tests/sandbox-tests.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/groom/tests/sandbox-tests.sh b/.github/groom/tests/sandbox-tests.sh index 5e22559..6b9cb5e 100755 --- a/.github/groom/tests/sandbox-tests.sh +++ b/.github/groom/tests/sandbox-tests.sh @@ -80,7 +80,9 @@ echo secret > /tmp/canary export HOSTSECRET=leaked-host-value if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ --env FOO=bar -- bash -c ' - dump=$(tr "\0" "\n" < /proc/self/environ) + # NB: read via `cat` not `< /proc/self/environ` — the `<` redirect resolves + # /proc/self in the pre-exec forked shell and reads empty inside the jail. + dump=$(cat /proc/self/environ | tr "\0" "\n") echo "$dump" | grep -qx "FOO=bar" || { echo "FOO missing"; exit 1; } echo "$dump" | grep -qx "HOME=/home/agent" || { echo "HOME wrong"; exit 1; } echo "$dump" | grep -qx "PATH=/usr/local/bin:/usr/bin:/bin" || { echo "PATH wrong"; exit 1; } From 3ed4386052a6e4e4f06aee505559aa6f9629de66 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Thu, 23 Jul 2026 23:15:25 -0700 Subject: [PATCH 4/5] fix(groom): harden sandbox path validation + broker stream resilience (BE-4302) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address CodeRabbit review on PR #66: - agent-sandbox.sh: require --clone/--out-dir/--ro-file to be absolute paths (bwrap binds them at their real path; a relative value would resolve against an unexpected CWD instead of failing loud, per the script's fail-closed design). - agent-sandbox.sh: rw-git-ro now fails loud when .git is a gitdir pointer file (git-worktree checkout) rather than a directory — the pointed-to metadata isn't mounted, which would silently break git instead of protecting it. - broker.mjs: handle upstream-response stream errors (previously unhandled → crashed the broker mid-stream), tear down on client abort/close, and add a 120s upstream idle timeout so a hung upstream can't pin a request open. Co-Authored-By: Claude Opus 4.8 --- .github/groom/agent-sandbox.sh | 12 ++++++++++++ .github/groom/broker.mjs | 17 +++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/.github/groom/agent-sandbox.sh b/.github/groom/agent-sandbox.sh index f370c8f..a09e60e 100755 --- a/.github/groom/agent-sandbox.sh +++ b/.github/groom/agent-sandbox.sh @@ -104,6 +104,10 @@ main() { [[ -n "$clone" ]] || die "--clone is required" [[ -n "$out_dir" ]] || die "--out-dir is required" [[ ${#cmd[@]} -gt 0 ]] || die "a -- is required" + # bwrap binds each of these at its REAL path; a relative value would resolve + # against an unexpected CWD instead of failing loud, so require absolute paths. + [[ "$clone" = /* ]] || die "--clone must be an absolute path (got '$clone')" + [[ "$out_dir" = /* ]] || die "--out-dir must be an absolute path (got '$out_dir')" case "$clone_mode" in ro | rw-git-ro) ;; *) die "--clone-mode must be 'ro' or 'rw-git-ro' (got '${clone_mode:-}')" ;; @@ -152,6 +156,13 @@ main() { # tracked files (the patch we capture) yet can never rewrite history # or git config. Ordering matters — the rw clone bind first, then the # ro .git overlay on top. + # + # This assumes .git is a real directory. In a git-worktree checkout it + # is instead a file holding a `gitdir:` pointer to metadata elsewhere on + # the host — which would NOT be mounted into the jail, silently breaking + # git rather than protecting it. Fail loud instead (callers pass plain + # clones; worktree checkouts are unsupported here). + [[ -d "$clone/.git" ]] || die "--clone-mode rw-git-ro needs a plain .git directory (got a gitdir pointer file — git worktree checkouts are unsupported): $clone/.git" bwrap_args+=(--bind "$clone" "$clone" --ro-bind "$clone/.git" "$clone/.git") ;; esac @@ -159,6 +170,7 @@ main() { local f if [[ ${#ro_files[@]} -gt 0 ]]; then for f in "${ro_files[@]}"; do + [[ "$f" = /* ]] || die "--ro-file must be an absolute path (got '$f')" bwrap_args+=(--ro-bind "$f" "$f") done fi diff --git a/.github/groom/broker.mjs b/.github/groom/broker.mjs index 30897c6..ae88dbc 100644 --- a/.github/groom/broker.mjs +++ b/.github/groom/broker.mjs @@ -63,6 +63,13 @@ const server = http.createServer((req, res) => { console.log(`${req.method} ${req.url} ${upRes.statusCode}`); res.writeHead(upRes.statusCode, upRes.headers); upRes.pipe(res); // stream through — no buffering, so SSE passes intact + // A mid-stream upstream failure (after headers) can't become a 502 anymore; + // handle its 'error' so it tears down the client response instead of + // bubbling to an uncaughtException that takes the whole broker down. + upRes.on('error', () => { + console.log(`${req.method} ${req.url} upstream-stream-error`); + res.destroy(); + }); }, ); @@ -72,6 +79,16 @@ const server = http.createServer((req, res) => { res.end('upstream error\n'); }); + // Don't let a hung upstream pin a request open forever; tear it down on idle. + upstream.setTimeout(120_000, () => upstream.destroy(new Error('upstream timeout'))); + + // If the client goes away (aborted/dropped), stop talking to upstream so we + // don't leave a half-open request spending against the key. + res.on('close', () => { + if (!res.writableFinished) upstream.destroy(); + }); + req.on('error', () => upstream.destroy()); + req.pipe(upstream); // forward the request body streaming too }); From 7942e58a18672a421ab36c9a45a14309fdced825 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Thu, 23 Jul 2026 23:54:10 -0700 Subject: [PATCH 5/5] fix(groom): close residual sandbox+broker review gaps (BE-4302) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the still-valid cursor-review panel findings on top of 3ed4386: - broker: reject dot-segment paths (`/v1/../v2/...`) that pass a bare `/v1/` prefix check but normalize off /v1 upstream — fail closed. - broker: on an upstream error AFTER the response began streaming, tear the response down instead of splicing 'upstream error' bytes into the SSE body. - broker: add a `res.on('error')` handler — `.pipe()` installs none, so a client socket error mid-stream would crash the whole broker via uncaughtException. - agent-sandbox: reject an `--out-dir` that overlaps `--clone` (equal / nested / ancestor). It is bound rw LAST, so bwrap's last-wins ordering would otherwise shadow the read-only clone/.git mounts and silently un-protect them. Tests: overlap rejection (3 cases), dot-segment 404, and broker crash-resilience on a mid-stream client disconnect added to sandbox-tests.sh. Co-Authored-By: Claude Opus 4.8 --- .github/groom/agent-sandbox.sh | 20 ++++++++++++++--- .github/groom/broker.mjs | 22 ++++++++++++++++--- .github/groom/tests/sandbox-tests.sh | 33 ++++++++++++++++++++++++++++ 3 files changed, 69 insertions(+), 6 deletions(-) diff --git a/.github/groom/agent-sandbox.sh b/.github/groom/agent-sandbox.sh index a09e60e..1180a78 100755 --- a/.github/groom/agent-sandbox.sh +++ b/.github/groom/agent-sandbox.sh @@ -114,11 +114,25 @@ main() { esac [[ -d "$clone" ]] || die "clone path is not a directory: $clone" - preflight - - # out-dir must exist on the host before it can be bound rw into the jail. + # out-dir must exist on the host before it can be bound rw into the jail; create + # it here so we can canonicalize it for the overlap check below. mkdir -p "$out_dir" + # The writable out-dir is bound LAST, and bwrap's last-bind-wins ordering means + # an out-dir that overlaps the clone would shadow the read-only clone/.git + # mounts and silently make protected content (including .git under rw-git-ro) + # writable — defeating the read-only contract. Canonicalize both and require + # them to be disjoint (neither equal nor an ancestor of the other). Fail here, + # before the (slow) preflight, so a bad invocation is rejected fast. + local clone_real out_real + clone_real="$(realpath "$clone")" || die "cannot resolve --clone path: $clone" + out_real="$(realpath "$out_dir")" || die "cannot resolve --out-dir path: $out_dir" + if [[ "$out_real" == "$clone_real" || "$out_real" == "$clone_real"/* || "$clone_real" == "$out_real"/* ]]; then + die "--out-dir must not overlap --clone (out-dir '$out_real' vs clone '$clone_real'): a writable bind over the clone would defeat its read-only mounts" + fi + + preflight + local bwrap_args=( --unshare-all --share-net --die-with-parent --new-session --clearenv --ro-bind /usr /usr diff --git a/.github/groom/broker.mjs b/.github/groom/broker.mjs index ae88dbc..5793878 100644 --- a/.github/groom/broker.mjs +++ b/.github/groom/broker.mjs @@ -44,7 +44,10 @@ const server = http.createServer((req, res) => { } // Only the Messages/Anthropic API surface is proxied; everything else is denied. - if (!req.url.startsWith('/v1/')) { + // Reject dot-segments too: a raw target like `/v1/../v2/foo` passes a bare + // prefix check yet normalizes to a non-/v1 route upstream, so fail it closed. + const reqPath = req.url.split('?')[0]; + if (!req.url.startsWith('/v1/') || reqPath.includes('..')) { console.log(`${req.method} ${req.url} 404`); res.writeHead(404, { 'content-type': 'text/plain' }); res.end('not found\n'); @@ -75,8 +78,16 @@ const server = http.createServer((req, res) => { upstream.on('error', () => { console.log(`${req.method} ${req.url} 502`); - if (!res.headersSent) res.writeHead(502, { 'content-type': 'text/plain' }); - res.end('upstream error\n'); + if (!res.headersSent) { + res.writeHead(502, { 'content-type': 'text/plain' }); + res.end('upstream error\n'); + } else { + // Response already began streaming (routine for SSE): a 502 body can't be + // sent anymore, and appending 'upstream error' here would splice stray + // bytes into the middle of the real body. Tear it down instead so the + // client sees a truncated stream rather than a corrupted one. + res.destroy(); + } }); // Don't let a hung upstream pin a request open forever; tear it down on idle. @@ -87,6 +98,11 @@ const server = http.createServer((req, res) => { res.on('close', () => { if (!res.writableFinished) upstream.destroy(); }); + // .pipe() forwards neither errors nor a listener onto the client response, so a + // client socket error mid-stream (ECONNRESET, write-after-close) would emit an + // unhandled 'error' on res and take the whole broker down. Absorb it and tear + // down the upstream leg — same crash-proofing as the upRes/upstream handlers. + res.on('error', () => upstream.destroy()); req.on('error', () => upstream.destroy()); req.pipe(upstream); // forward the request body streaming too diff --git a/.github/groom/tests/sandbox-tests.sh b/.github/groom/tests/sandbox-tests.sh index 6b9cb5e..e797562 100755 --- a/.github/groom/tests/sandbox-tests.sh +++ b/.github/groom/tests/sandbox-tests.sh @@ -126,6 +126,24 @@ grep -q "builder patch" "$clone/patch-from-agent.txt" 2>/dev/null \ || fail "rw-git-ro worktree write not visible on host" pass "clone rw-git-ro (worktree write captured on host, .git read-only)" +# --- 3b. out-dir/clone overlap rejection ------------------------------------- +# The out-dir is bound rw LAST, so bwrap's last-wins ordering would let it shadow +# the read-only clone/.git mounts if it overlapped them. The wrapper must reject +# an out-dir equal to, inside, or an ancestor of the clone — failing loud before +# it ever builds the jail. (This validation runs before preflight, so it holds +# even on a host without bwrap.) + +if "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$clone" -- true 2>/dev/null; then + fail "out-dir == clone was accepted (rw bind would un-protect the read-only clone)" +fi +if "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$clone/nested/out" -- true 2>/dev/null; then + fail "out-dir inside clone was accepted (rw bind would shadow the read-only clone)" +fi +if "$SANDBOX" --clone "$clone/.git" --clone-mode ro --out-dir "$clone" -- true 2>/dev/null; then + fail "out-dir as an ancestor of clone was accepted (rw bind would shadow the read-only clone)" +fi +pass "out-dir/clone overlap rejected (equal, nested, and ancestor cases)" + # --- 4. pid isolation -------------------------------------------------------- sleep 300 & @@ -182,6 +200,10 @@ if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ # non-/v1 path denied code=$(curl -s -o /dev/null -w "%{http_code}" "$base/not-v1") [ "$code" = "404" ] || { echo "non-/v1 not 404: $code"; exit 1; } + # dot-segment path denied (raw target, sent un-normalized via --path-as-is): + # a bare prefix check would pass "/v1/.." yet it normalizes off /v1 upstream. + code=$(curl -s --path-as-is -o /dev/null -w "%{http_code}" "$base/v1/../not-v1") + [ "$code" = "404" ] || { echo "dot-segment /v1/../ not 404: $code"; exit 1; } # chunked/SSE response streams through intact stream=$(curl -sN "$base/v1/stream") echo "$stream" | grep -q "data: one" || { echo "sse frame one missing"; exit 1; } @@ -191,4 +213,15 @@ if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ '; then fail "broker credential proxy"; fi pass "broker credential proxy (key injected+stripped, healthz local, non-/v1 404, SSE streams)" +# --- 6. broker crash-resilience: client disconnect mid-stream ---------------- +# .pipe() puts no error listener on the client response, so a client that drops +# mid-SSE would emit an unhandled 'error' on res and kill the whole broker. Force +# that: /v1/stream sends one frame, waits 50ms, then the rest — abort inside the +# gap, then prove the broker is still serving. +curl -sN --max-time 0.02 "http://127.0.0.1:$broker_port/v1/stream" >/dev/null 2>&1 || true +sleep 0.2 +curl -fsS "http://127.0.0.1:$broker_port/healthz" >/dev/null 2>&1 \ + || fail "broker crashed after a client disconnected mid-stream" +pass "broker crash-resilience (survives a client mid-stream disconnect)" + echo "ALL SANDBOX TESTS PASSED"