diff --git a/docs/gvisor-integration.md b/docs/gvisor-integration.md index 82966154f..aa6f2526c 100644 --- a/docs/gvisor-integration.md +++ b/docs/gvisor-integration.md @@ -129,6 +129,48 @@ if (config.containerRuntime) { } ``` +### Volume mounting & filesystem access (the Gofer) + +gVisor uses AWF's **standard compose agent** — the same service definition as the +default Docker runtime, only with `runtime: runsc`. So the *mount set is +identical to default Docker mode*: AWF applies selective bind mounts under +`/host` (see `src/services/agent-volumes/system-mounts.ts`) and the agent +`chroot`s into `/host` before running: + +- `/usr`, `/bin`, `/sbin`, `/lib`, `/lib64`, `/opt`, `/sys`, `/dev` → `/host/*` + read-only (system libraries and toolchains come from the host, unlike sbx). +- `` → `/host` read-write; `/tmp` → `/host/tmp` + read-write. +- When `chroot.binariesSourcePath` is configured, that tool directory is additionally + mounted at **`/host/tmp/awf-runner-bin`** (ro), and `entrypoint.sh` prepends it + to `PATH`. This is especially useful for DinD/ARC staged filesystems where the + runner-installed binaries are not present in the mounted `/usr` tree. +- An empty home volume exposes only whitelisted `$HOME` subdirs; select `/etc` + files (SSL certs, `passwd`, `group`, `hosts`, …) are mounted individually. + +The gVisor-specific twist is **how those mounts are accessed**. A `runsc` +sandbox uses a separate host-side **Gofer** process to mediate path-based +filesystem operations over LISAFS. With directfs (enabled by default in current +`runsc`), the Gofer can pass an opened host file descriptor to the Sentry, which +then performs data I/O directly and avoids repeated Gofer round trips. (9P was +the legacy Sentry-Gofer protocol.) Practical consequences: + +- **Isolation:** the sandbox is restricted to the filesystem tree assembled for + it; the Gofer controls path resolution and opening, while directfs constrains + subsequent access to the descriptors it passes to the Sentry. +- **Compatibility:** filesystem behavior still follows gVisor's implementation; + operations such as some `mmap` sharing and exotic `ioctl`s can differ from a + native bind mount. This is the filesystem analogue of the syscall shims noted + below and should be validated when a tool works in Docker but not under gVisor. + +:::note +Because gVisor reuses the compose agent, there is **no sbx-style host-path == +guest-path** behavior here: the agent sees files under `/host` (pre-chroot) and +at their normal paths (post-chroot), exactly as in default Docker mode. A new +compose-model runtime inherits this mount set for free; a new *microVM* runtime +(like sbx) must define its own sharing scheme instead. +::: + ### The netstack DNS problem (and the fix) gVisor's userspace netstack has an isolated sandbox loopback that **cannot reach diff --git a/docs/sbx-integration.md b/docs/sbx-integration.md index 2b31e4b71..77e43fa7c 100644 --- a/docs/sbx-integration.md +++ b/docs/sbx-integration.md @@ -146,6 +146,61 @@ not-yet-ready Squid) and `XDG_CONFIG_HOME` (so the sbx CLI finds credentials in `$HOME/.config`, not wherever the Copilot harness pointed it). ::: +### Volume mounting & toolchain sharing + +The most important structural difference from AWF's Docker/gVisor path is that +the microVM does **not** use a chroot. In compose mode the agent is bind-mounted +under `/host/...` and then `chroot`s into it; sbx instead uses **positional path +mounts where the host path maps to the identical path inside the VM** +(`/home/runner/work/...` on the host is `/home/runner/work/...` in the guest). +The generated command is: + +```text +sbx create --name shell [extraMount...] /tmp /usr/local/bin $HOME +``` + +(`shell` is sbx's generic agent image, which supplies the guest base OS.) + +What `createSandbox()` shares, in order: + +1. **Workspace** — `workspaceDir = $GITHUB_WORKSPACE || process.cwd()`, the first + positional mount, read-write. This is the repo checkout the agent edits. +2. **Extra mounts** — `config.volumeMounts` (from `--volume`). AWF stores these + Docker-style (`host:container:mode`), but sbx only accepts a positional + `hostPath` with an optional `:ro` suffix, so the manager parses out the host + path and mode and **discards the container-path segment** (host path = guest + path). Default is read-write; `ro` mode becomes `hostPath:ro`. +3. **Three always-added system mounts** that carry the toolchain and runtime state: + - **`/usr/local/bin`** — the toolchain seam. The Copilot CLI and other + host-installed tools live here and are mounted straight in. Note it is + *narrow*: only `/usr/local/bin`, **not** `/usr`, `/lib`, `/lib64`, or `/opt`. + - **`/tmp`** — agent runtime files (rendered prompts, logs). + - **`$HOME`** (`$HOME || /home/runner`) — writable agent dirs (`.cache`, + `.config`, `.local`, `.anthropic`, `.copilot`, …). + +A `seenPaths` set deduplicates so no path is mounted twice, and +`execInSandbox(..., { workDir })` passes `--workdir` so commands run inside the +mounted workspace. + +| Aspect | sbx (microVM) | Docker / gVisor (compose agent) | +| --- | --- | --- | +| Path model | Host path **==** guest path | Bind-mounted under `/host`, then `chroot /host` | +| System libraries | From the sbx `shell` **guest image** | Host `/usr`,`/bin`,`/lib`,`/lib64`,`/opt` mounted read-only | +| Toolchain binaries | Host `/usr/local/bin` mounted in | `/usr` from the host or sysroot; optional `chroot.binariesSourcePath` overlay at `/host/tmp/awf-runner-bin` (ro) | +| Workspace | `workspaceDir` positional (rw) | `:/host:rw` | +| Home | Whole `$HOME` mounted (rw) | Empty home volume with only whitelisted subdirs | + +:::caution Toolchain portability +Because host system libraries are **not** shared into the VM, a binary in +`/usr/local/bin` that dynamically links against host-specific libraries — or +expects an interpreter/runtime under `/usr` — can fail inside the microVM unless +the sbx guest image already provides a compatible base. This is the sharpest +contrast with compose mode's broad read-only `/usr`+`/lib` mounts, and the single +most important detail to plan for when building a **KVM-based microVM backend**: +you must either ship a guest image whose base matches the tools you mount in, or +widen the mount set to include the libraries those tools need. +::: + ### Wiring into the main workflow (`src/commands/main-action.ts`) `main-action.ts` decides `useSbx = !runtimeUsesComposeAgent(config.containerRuntime)`