Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions docs/gvisor-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,49 @@ if (config.containerRuntime) {
}
```

### Volume mounting & filesystem access (the Gofer)

gVisor uses AWF's **standard compose agent** — the same service definition as the
default Docker runtime, only with `runtime: runsc`. So the *mount set is
identical to default Docker mode*: AWF applies selective bind mounts under
`/host` (see `src/services/agent-volumes/system-mounts.ts`) and the agent
`chroot`s into `/host` before running:

- `/usr`, `/bin`, `/sbin`, `/lib`, `/lib64`, `/opt`, `/sys`, `/dev` → `/host/*`
read-only (system libraries and toolchains come from the host, unlike sbx).
- `<workspaceDir>` → `/host<workspaceDir>` read-write; `/tmp` → `/host/tmp`
read-write.
- Host-installed tool binaries are mounted at **`/host/tmp/awf-runner-bin`** (ro)
rather than `/host/usr/local/bin`, because `/host/usr` is mounted read-only and
Docker cannot then create `local/bin` under it (matters in DinD/ARC staged
filesystems); `entrypoint.sh` prepends that dir to `PATH`.
Comment thread
Copilot marked this conversation as resolved.
Outdated
- An empty home volume exposes only whitelisted `$HOME` subdirs; select `/etc`
files (SSL certs, `passwd`, `group`, `hosts`, …) are mounted individually.

The gVisor-specific twist is **how those mounts are accessed**. In a `runsc`
sandbox the application never touches host files directly. All filesystem I/O is
proxied by the **Gofer** — a separate host-side process, one per sandbox, that
the Sentry talks to over an internal protocol (LISAFS/9P). The Sentry holds no
host file descriptors for the bind mounts; it asks the Gofer, which enforces that
only the explicitly mounted paths are reachable. Practical consequences:

- **Isolation:** the mount set is the *only* host filesystem the sandbox can
reach, and even those pass through the Gofer boundary rather than raw host FDs —
an extra containment layer on top of AWF's selective mounts.
- **Compatibility:** operations the Gofer models imperfectly (some `mmap`
sharing, exotic `ioctl`s, dense small-file I/O) can behave differently or run
slower than a native bind mount. This is the filesystem analogue of the syscall
shims noted below, and the thing to validate when a tool "works in Docker but
not under gVisor."
Comment thread
Copilot marked this conversation as resolved.
Outdated

:::note
Because gVisor reuses the compose agent, there is **no sbx-style host-path ==
guest-path** behavior here: the agent sees files under `/host` (pre-chroot) and
at their normal paths (post-chroot), exactly as in default Docker mode. A new
compose-model runtime inherits this mount set for free; a new *microVM* runtime
(like sbx) must define its own sharing scheme instead.
:::

### The netstack DNS problem (and the fix)

gVisor's userspace netstack has an isolated sandbox loopback that **cannot reach
Expand Down
55 changes: 55 additions & 0 deletions docs/sbx-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,61 @@ not-yet-ready Squid) and `XDG_CONFIG_HOME` (so the sbx CLI finds credentials in
`$HOME/.config`, not wherever the Copilot harness pointed it).
:::

### Volume mounting & toolchain sharing

The most important structural difference from AWF's Docker/gVisor path is that
the microVM does **not** use a chroot. In compose mode the agent is bind-mounted
under `/host/...` and then `chroot`s into it; sbx instead uses **positional path
mounts where the host path maps to the identical path inside the VM**
(`/home/runner/work/...` on the host is `/home/runner/work/...` in the guest).
The generated command is:

```text
sbx create --name <name> shell <workspaceDir> [extraMount...] /tmp /usr/local/bin $HOME
```

(`shell` is sbx's generic agent image, which supplies the guest base OS.)

What `createSandbox()` shares, in order:

1. **Workspace** — `workspaceDir = $GITHUB_WORKSPACE || process.cwd()`, the first
positional mount, read-write. This is the repo checkout the agent edits.
2. **Extra mounts** — `config.volumeMounts` (from `--volume`). AWF stores these
Docker-style (`host:container:mode`), but sbx only accepts a positional
`hostPath` with an optional `:ro` suffix, so the manager parses out the host
path and mode and **discards the container-path segment** (host path = guest
path). Default is read-write; `ro` mode becomes `hostPath:ro`.
3. **Three always-added system mounts** that carry the toolchain and runtime state:
- **`/usr/local/bin`** — the toolchain seam. The Copilot CLI and other
host-installed tools live here and are mounted straight in. Note it is
*narrow*: only `/usr/local/bin`, **not** `/usr`, `/lib`, `/lib64`, or `/opt`.
- **`/tmp`** — agent runtime files (rendered prompts, logs).
- **`$HOME`** (`$HOME || /home/runner`) — writable agent dirs (`.cache`,
`.config`, `.local`, `.anthropic`, `.copilot`, …).

A `seenPaths` set deduplicates so no path is mounted twice, and
`execInSandbox(..., { workDir })` passes `--workdir` so commands run inside the
mounted workspace.

| Aspect | sbx (microVM) | Docker / gVisor (compose agent) |
| --- | --- | --- |
| Path model | Host path **==** guest path | Bind-mounted under `/host`, then `chroot /host` |
| System libraries | From the sbx `shell` **guest image** | Host `/usr`,`/bin`,`/lib`,`/lib64`,`/opt` mounted read-only |
| Toolchain binaries | Host `/usr/local/bin` mounted in | Host binaries mounted at `/host/tmp/awf-runner-bin` (ro) |
Comment thread
Copilot marked this conversation as resolved.
Outdated
| Workspace | `workspaceDir` positional (rw) | `<workspaceDir>:/host<workspaceDir>:rw` |
| Home | Whole `$HOME` mounted (rw) | Empty home volume with only whitelisted subdirs |

:::caution Toolchain portability
Because host system libraries are **not** shared into the VM, a binary in
`/usr/local/bin` that dynamically links against host-specific libraries — or
expects an interpreter/runtime under `/usr` — can fail inside the microVM unless
the sbx guest image already provides a compatible base. This is the sharpest
contrast with compose mode's broad read-only `/usr`+`/lib` mounts, and the single
most important detail to plan for when building a **KVM-based microVM backend**:
you must either ship a guest image whose base matches the tools you mount in, or
widen the mount set to include the libraries those tools need.
:::

### Wiring into the main workflow (`src/commands/main-action.ts`)

`main-action.ts` decides `useSbx = !runtimeUsesComposeAgent(config.containerRuntime)`
Expand Down
Loading