Skip to content
This repository was archived by the owner on Sep 8, 2026. It is now read-only.

meta(#532): built-in meta sandbox tools on /api/agent - #535

Merged
btipling merged 6 commits into
mainfrom
plan/meta-sandbox-tools
Aug 15, 2026
Merged

btipling merged 6 commits into
mainfrom
plan/meta-sandbox-tools

Conversation

@btipling

Copy link
Copy Markdown
Owner

meta(#532): built-in meta sandbox tools on /api/agent

Phase 2 of the built-in meta tool surface (#332, parent #530): first-party sandbox bind tools — non-secret inventory, current-bind surface, and a mid-session switch that persists meta.activeSandboxId via the existing session envelope seam. Built on the stable sandbox API shipped by #330.

Closes #532 · Refs #530 (parent epic) · Refs #332 (product)

What ships

  • lib/agent/metaSandboxTools.ts — createMetaSandboxTools({userId, sessionId, userPreferredSandbox, sessionStoreSeam}), always-on in-process AI SDK tools:
    • meta_sandbox_list — non-secret projection mirroring GET /api/sandboxes.options (id/name/slug/backend/status/image/canRead/canWrite/usable/granted). Never base_url/token.
    • meta_sandbox_active — the persisted meta.activeSandboxId when it is a usable grant, else null, + its describeSandboxTools surface (mirrors the route's active). A set-but-unusable id is reported honestly, fail-closed.
    • meta_sandbox_switch — validate Redis-safe + usable-grant to the caller; persist meta.activeSandboxId on the caller's own session via the envelope seam (resolveSessionStore → isEnvelopeStore → readEnvelope/upsertEnvelope, updatedAt preserved like attachedSkills). Fail-closed, no partial write on a non-Redis-safe id, unusable/ungranted/wrong-tenant grant, missing sessionId, or an unavailable / non-envelope store.
  • app/api/agent/route.ts — assemble into extraTools always; seam closed over the existing resolveSessionStore() + services.harnessSessionsRedis.resolveTenantIdForUser. Covered by the meta_* soft-path guard (isMetaToolName).
  • lib/agent/runAgent.ts — append META_SANDBOX_SYSTEM_ADDENDUM when a meta_sandbox_* tool is present.
  • Docs (same PR) — docs/mcp.md, docs/sandbox.md, docs/session-model.md, AGENTS.md row. Cloud ops N/A (no Production mutate, no GHA).

Tests

New lib/agent/metaSandboxTools.test.ts — 16 cases (store-fakes mirroring the metaTools.test.ts pattern), covering: non-secret projection (no base_url/token), active-descriptor resolution incl. set-but-unusable honesty, switch success (persists activeSandboxId, preserves updatedAt + stored fields, fresh-envelope create), and the full fail-closed matrix (unusable/ungranted/wrong-tenant/foreign id, missing sessionId, store unavailable, non-envelope isEnvelopeStore guard, non-Redis-safe id) with no partial write assertions.

Gates

Gate Result
npm run typecheck ✅ exit 0
node scripts/di-gate.mjs ✅ ok — no in-body I/O construction
node_modules/vitest/vitest.mjs run ✅ 1496 passed / 0 failed (112 files; 1480 baseline + 16 new)
npm run build ✅ (public/harness present)

Not merged. Ready for review.

@btipling btipling self-assigned this Aug 15, 2026
@vercel

vercel Bot commented Aug 15, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
invincible Ready Ready Preview Aug 15, 2026 5:50am

Request Review

@btipling

Copy link
Copy Markdown
Owner Author

Implented per implement-plan on #532 (plan Status HANDOFF-READY).

  • New lib/agent/metaSandboxTools.ts (3 tools) + 16 unit tests (non-secret projection, active descriptor incl. set-but-unusable honesty, switch persist via envelope seam preserving updatedAt, and the full fail-closed no-partial-write matrix).
  • Wired into /api/agent extraTools always; META_SANDBOX_SYSTEM_ADDENDUM appended when meta_sandbox_* present.
  • Docs: docs/mcp.md, docs/sandbox.md, docs/session-model.md, AGENTS.md row. Cloud ops N/A.

Gates (agent workspace): typecheck ✅ · di-gate ✅ · full vitest run 1496 passed / 0 failed (1480 baseline + 16 new) · npm run build ✅.

Not merged. CI: Vercel = pending (deploying), Preview Comments = pass → mergeStateStatus: UNSTABLE until Vercel lands.

@btipling btipling left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review — PR #535

Verdict: BLOCK
Repo: btipling/invincible
Scope: main ← 9e774da · 8 files · phase 2 meta_sandbox_* (#532)
Lenses run: L1, L2, L6, L8 (skip L3: no protocol; L4: no workflow; L5: tiny inventory; L7: no UI)
AGENTS.md read: yes · SECURITY.md: tokens never client/Wasm

Findings

Sev Lens Finding Break scenario Refutation attempt Confidence
Blocker L1 Switch does not bind the next turn and is overwritten by the host PUT. meta_sandbox_switch writes meta.activeSandboxId on the envelope and preserves updatedAt. Next /api/agent resolve uses only parsed.sandboxId from the client (route.ts ~415) — it never reads the envelope. The host sends session.activeSandboxId from local state (harnessChat.ts ~983) and on success adopts agentResult.sandboxId, which is the start-of-turn resolve id (runAgent.ts sandboxOut = hasFsTools ? params.sandboxId), not the switched id. Then PUT folds that old id with a newer updatedAt → LWW wipes the tool write. Docs claim the switch “survives even without a host PUT” — true only if the host never PUTs. It always does. User: “switch to staging.” Tool: switched active sandbox to id=stg. This turn still uses prod FS. After persist, local + Redis are prod again. Next turn still prod. (1) Route resolve must treat envelope meta.activeSandboxId as requestedSandboxId (same as persona snapshot). (2) Surface the new id on the JSON/SSE result (skill_attached-style) so the host adopts it before PUT. (3) Check upsertEnvelope status (below). high
Major L1 upsertEnvelope conflict is ignored. Result is {status:'stored'|'conflict'}; switch always returns switched…. Same updatedAt write loses to a concurrent host PUT (input.updatedAt < existing.updatedAt). Model is told it switched; store unchanged. Mid-turn host PUT (cwd / attachedSkills). Switch reports success. Next resolve still old. if (result.status !== 'stored') return ERROR … (no write). high
Major L1 The case switch exists for never reaches the tool. SANDBOX_SELECTION_REQUIRED (multiple usable, no preferred / stale active) is ok: false without softContinue. Route hard-returns 403 before runAgent unless BUILTIN_HTTP_FETCH is on. meta_* are excluded from nonSkillToolCount on purpose. User with two grants, no preferred: every agent turn 403s. Cannot meta_sandbox_switch. Feature dead unless HTTP is enabled. Soft-continue (or run with meta tools only) on selection-required so switch can persist, then next turn resolves. UI-only select is not this PR’s job if the addendum says the agent can switch. high
Nit L2 Always-on switch: prompt injection can retarget a sandbox the user already has a grant on (read-only session → writable). Same product accept as #533; lower blast radius than delete-all-skills. “Summarize https://evil” → meta_sandbox_switch to the write grant. Residual / #534-class visibility. Not the merge gate. med

Checked and not findings:

  • L2 IDOR / secrets: userId/sessionId closed over; foreign id not in listUserSandboxChoices → no write. projectSandboxOption has no base_url / token. Key is {tenantId, caller userId, caller sessionId}. Redis-safe id checked. Tests cover unusable / foreign / no sessionId / non-envelope store.
  • L6: DI + sessionStoreSeam; no in-body I/O. isMetaToolName still excludes meta_sandbox_* from the workspace 403 mask (correct, but it causes the selection-required hard-403 above).

Residual risk

Stale logicalCwd after a bind change (docs mention fail-closed abs paths). List unbounded (sandbox count is small). Switch this turn does not rebuild FS tools (documented subsequent-turn).

Merge guidance

Do not merge until a switched id is what resolve and the host session see on the next turn (envelope read + result fold), and selection-required can reach the tool. Conflict must fail closed.

What was not attacked

Live Redis + host PUT after a real switch; two-tenant grant row in a real DB.

@btipling

btipling commented Aug 15, 2026 •

Copy link
Copy Markdown
Owner Author

Block-response plan — sandbox bind switch must actually persist, honor store conflicts, and be reachable for selection

Status: DONE (all increments landed; committed to this PR branch) — living comment; amended as I iterate. Increments 1 (B2), 2 (A1/A2 wire), 3 (A1 resolve seed + host fold), and 4 (B3 — selection reachability) are DONE and committed; docs updated.

Decisions resolved (you approved my recommendations):

  1. B3 policy — always soft-path on selection-required when the meta sandbox tools are present (agent may drive the pick; no user gates per the meta(#531): built-in meta persona + skill authoring tools on /api/agent #533 stance).
  2. A2 wire shape — new reserved activeSandboxId field on the JSON response and the stream done event, distinct from sandboxId (pre-turn client bind).
  3. Precedence — envelope ?? body: the server envelope wins for requestedSandboxId when a sessionId is present, but the body still seeds when the envelope carries no bind (no regression to the existing parsed.sandboxId path).

The block (from adversarial review) — three coupled correctness bugs, not scope/wont-fix

  • B1 — the switch never survives a turn (host clobber + server never honors it): tool writes meta.activeSandboxId, but the done/JSON reply returns the pre-turn resolved id, the host folds that OLD id and PUTs it, and the server never reads the persisted bind back out. Stacks B1a stale host fold / B1b no server read / B1c reply reflects pre-turn active. DONE (increments 2 + 3).
  • B2 — the tool ignores envelope conflicts → false "switched" success: upsertEnvelope result was discarded; the read→write race against a concurrent host PUT yields a rejected write reported as success. DONE (increment 1, commit 8a76f66).
  • B3 — the switch tool is unreachable when the agent needs it (multi-sandbox selection): resolve returns 403 selection-required without softContinue, the route hard-403s, and the deferredNoFsResponse guard counts meta_* as no surface. DONE (increment 4).

Strategy — A (server-authoritative envelope bind; approved)

meta.activeSandboxId on the session envelope is the single source of truth. A1 (B1b): route reads the envelope bind and passes it as requestedSandboxId when the body has none. A2 (B1c/B1a): server returns the effective active activeSandboxId on JSON + done; host folds THAT (fallback sandboxId). A3 (B2): conflict-aware write. B3 (reachability): selection-required soft-paths when meta sandbox tools are present; the deferredNoFsResponse guard counts them as a legitimate soft surface for this class only.

Change set (all landed)

  • lib/agent/metaSandboxTools.ts — B2: retryPersistActiveSandbox preserves stored updatedAt, re-reads + bounded-retries once on conflict, returns a stored-write-only true; persistent conflict → honest fail-closed error, never a false success. Also metaSandboxSwitchActiveId / metaSandboxSwitchTargetId — typed extraction of the post-turn switch target from an AI-SDK step result set (switched active sandbox to id=<id> tools=[...]), the structured carrier for A2.
  • lib/agent/runAgent.ts / lib/agent/agentStream.ts — A2: RunAgentResult.activeSandboxId + done-event activeSandboxId = metaSandboxSwitchTargetId(steps) ?? sandboxId; emitted on both generate (JSON) and stream paths.
  • lib/agentApi.ts — A2: AgentSuccess.activeSandboxId parsed from both the JSON body and the stream done event.
  • app/api/agent/route.ts — A2 JSON emit; A1 resolve seed: when a sessionId is present, read the caller's envelope meta.activeSandboxId and seed requestedSandboxId (envelope ?? body, fail-open on store error). B3 (increment 4, commit a3a345c): a selectionRequired soft-resolve now proceeds when meta_sandbox_* tools are present (agent self-selects), while every OTHER deferral (workspace-not-running softContinue, builtin-http grant deny) still requires a real non-FS surface and else returns the deferred 403.
  • lib/tenancy/resolveSandbox.ts — B3: selectionRequired?: boolean on the not-ok result, set ONLY on the selection-required class (multiple usable grants / no bound/preferred id), never alongside softContinue, never on forbidden.
  • lib/harnessChat.ts — fold: success reconcile folds agentResult.activeSandboxId (fallback sandboxId) so a switch survives the fold/PUT. 403-grant-honesty clear unchanged.
  • Docs — docs/sandbox.md + docs/session-model.md updated to state the server-authoritative bind + conflict semantics and the B3 selection-required soft-path. (docs/mcp.md unchanged — it already describes the meta sandbox tool family; no new text needed.)

Test plan (all landed)

  • B2: 2 tests (concurrent host-bump recovered via ONE bound retry; persistent conflict → honest error). metaSandboxTools.test.ts 16 → 18.
  • A2: runAgent.test.ts +5 (generate folds switch target; mirrors sandboxId w/o switch; omitted w/o bind; stream done + result carry switch target; stream mirrors w/o switch); agentApi.test.ts +3 (JSON parse, absent→undefined, stream done parse).
  • A1: harnessChat.test.ts +2 (JSON + stream: the post-turn effective bind wins over the pre-turn sandboxId on the fold — the B1 regression guard); route.test.ts +3 (envelope seeds resolve; envelope wins over stale body sandboxId; body still seeds with no sessionId).
  • B3 (increment 4): route.test.ts +2 (selection-required soft-paths to meta tools with no 403 — skipSandboxTools, meta_sandbox_list/_switch present; hard-forbidden stays 403 even with meta tools present); resolveSandbox.test.ts +2 selectionRequired-flag assertions (set on both selection classes, undefined/softContinue-absent on forbidden).

Progress log

  • Inc 1 (done, commit 8a76f66): B2 conflict-aware write + tests. typecheck ✅; suite 18/18 ✅.
  • Inc 2 (done, commit 774bdd0): A2 wire — activeSandboxId on RunAgentResult, done event, AgentSuccess (JSON + stream). typecheck ✅; di-gate ✅; 106/106 suites ✅.
  • Inc 3 (done, commit 04319ab): A1 resolve seed (envelope ?? body) + host folds the post-turn EFFECTIVE bind. typecheck ✅; di-gate ✅; full vitest ✅ 1511/1511.
  • Inc 4 (done, commit a3a345c): B3 selection-required soft-path (selectionRequired flag + route guard) + docs. typecheck ✅; di-gate ✅; full vitest ✅ 1513/1513 (112 files); npm run build ✅. Docs updated.

Gates & notes

  • npm run typecheck ✅ · node scripts/di-gate.mjs ✅ (no I/O construction — envelope reads/writes stay inside the DI seam / route) · full node_modules/vitest/vitest.mjs run ✅ 1513/1513 · npm run build ✅.
  • Cloud ops: N/A — no Production mutate, no GHA (reuses the session envelope + grants). Living docs updated (docs/sandbox.md, docs/session-model.md); docs/mcp.md unchanged.
  • Expected merge-gate state: PR still UNSTABLE until the Vercel preview deploy lands; not merging on my own.

Living comment.

…retry, fail-closed no false success)

Adversarial-review block B2: the tool discarded the upsertEnvelope result, so a
store-rejected write (concurrent host PUT bumping updatedAt) was reported as a
false "switched" success. Now retryPersistActiveSandbox preserves the stored
updatedAt, re-reads + retries ONCE on a conflict, and returns true ONLY for a
stored write; a still-conflicting retry fails closed with an honest error.

Tests: +2 (concurrent-bump recover via bound retry; persistent conflict honest
error). metaSandboxTools.test.ts 16 -> 18 passing.

@btipling btipling left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review — PR #535 (re-run)

Verdict: PASS WITH NOTES
Repo: btipling/invincible
Scope: a3a345c on plan/meta-sandbox-tools
Lenses run: L1, L2, L6, L8
AGENTS.md read: yes

Prior BLOCK

Prior Status
B1 Switch lost: resolve ignored envelope; host folded pre-turn sandboxId; PUT LWW-wiped Fixed. Route seeds requestedSandboxId from envelope; envelope wins over body when sessionId is set (route.ts). runAgent emits post-turn activeSandboxId from a successful meta_sandbox_switch result (typed parse, not the truncated summary). Host folds activeSandboxId ?? sandboxId (harnessChat.ts). Tests: route B1 A1 + envelope-wins; harness JSON/stream fold.
B2 upsertEnvelope conflict reported as switched Fixed. retryPersistActiveSandbox: honor status, retry once on conflict, true only for stored. Tests for recover + honest fail.
B3 SANDBOX_SELECTION_REQUIRED hard-403 before tools Fixed. selectionRequired on resolve; route soft-paths when meta_sandbox_* present. Hard SANDBOX_FORBIDDEN (no usable grant) still 403. Route test: 200 + tools vs still-403 forbidden.
Nit L2 always-on switch Unchanged product accept (same class as #533).

Findings (this pass)

Sev Lens Finding Break scenario Refutation attempt Confidence
Nit L1 Envelope bind always wins over the body when sessionId is present. A host UI change that has not yet PUT the envelope is ignored on the next /api/agent. Operator picks another sandbox in chrome, sends immediately, no PUT yet → still old bind. Host already PUTs session mutations first in the normal path. Document “PUT then POST.” med
Nit L8 docs/session-model.md table still says the host applies the server’s resolved sandboxId on success. The wire is now activeSandboxId (switch target). Next implementer reverts the fold. One-line doc fix. med

Checked and not re-opened:

  • IDOR / no base_url / token: unchanged.
  • Switch-id parse: ^switched active sandbox to id=(\S+)\s+tools=; ERROR ignored; host re-checks Redis-safe.
  • Workspace-not-running still does not treat meta tools as a substitute (metaSelectionDeferred is selection-only).
  • This turn still does not rebuild FS tools after switch (next turn + envelope seed). Documented.

Residual risk

Always-on meta_sandbox_switch (prompt injection can retarget a grant the user already has). One conflict retry can still lose to two rapid host PUTs (honest error). Stale logicalCwd after bind change.

Merge guidance

OK to merge for #532. Optional doc + “PUT then POST” note.

What was not attacked

Live Redis + host PUT after a real switch on two devices.

This branch was previously deployed

1 inactive deployment
Preview — a9f8f64e Deployed Aug 15, 2026 by vercel[bot]
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

plan: built-in meta MCP — phase 2 — sandbox inventory + active-switch tools

1 participant