Skip to content

fix(server): finish Grok background subagents from subagent_finished - #13609

Merged
juliusmarminge merged 3 commits into
t3code/codex-turn-mappingfrom
v2/grok-subagent-finished
Sep 25, 2026
Merged

juliusmarminge merged 3 commits into
t3code/codex-turn-mappingfrom
v2/grok-subagent-finished

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

A Grok background subagent that outlived its root turn never finished in T3: its row stayed running and held the root run open indefinitely. Grok's spawn tool completes at launch ("Subagent started in background. subagent_id: …"). The subagent's end arrives only as a structured subagent_finished notification on the root session, and nothing in T3 handled it (only a doc comment in AcpAdapterV2.ts mentioned it).

Source

(xai-org/grok-build at f0e3be1)

  • SessionUpdate::SubagentFinished is sent on the parent session: crates/codegen/xai-grok-shell/src/extensions/notification.rs ~802. Fields: subagent_id, child_session_id, status, error? (the failure message), tool_calls, turns, duration_ms, tokens_used (defaults to 0; backward-compat tests ~1832/1862), output?, will_wake.
  • status is exactly "completed", "failed" or "cancelled": SubagentResult::status() in crates/codegen/xai-grok-tools/src/implementations/grok_build/task/types.rs ~526.
  • Grok's own leader consumes it by child_session_id: crates/codegen/xai-grok-shell/src/leader/server.rs ~591.
  • Grok then injects its own wake turn subagent-completed-<subagent id>: crates/codegen/xai-grok-shell/src/agent/subagent/spawn.rs ~524.

Fix

  • The Grok prompt runtime already owns the x.ai/session_notification handlers (for turn_completed). It now also routes subagent_finished to handlers registered through handleXAiSubagentFinished. The client allows one handler per extension method, so this can't be a second registration. handleXAiSubagentFinished dies for a runtime that isn't the Grok prompt runtime: such a runtime could never deliver the end, so its subagents would silently stay running.
  • Only the statuses Grok defines map to a notice; any other status is ignored rather than guessed. The result is Grok's output when the subagent completed and its error when it failed or was cancelled.
  • The ACP extension context gains finishSubagent({ childSessionId, status, result }). It finishes the subagent row with that result: in the turn held open for it, then re-arms deferred finalize; or, when the root turn already settled, in the carryover, projected while the completed root still owns the run. It is root-session only and dropped under Stop quarantine, like applyBackgroundTaskMutation.
  • Where Grok's own wake turn lands: as a provider continuation run, the same way Claude and Codex background wakes project. The continuation carries the "Background task completed." notification and Grok's reply, while the subagent and its node stay attributed to run 1.

Fixture: grok_background_subagent

Recorded live from Grok 1.0.41, using the gates and recorder from #13594. The root spawns a background subagent that runs sleep 20 and replies SUBAGENT_DONE; the root ends with ROOT_DONE right away. Replay asserts:

  • run 1 is held open for the subagent, the subagent completes (provider_native, attributed to run 1), and run 1 settles only after it
  • the subagent's SUBAGENT_DONE stays in its child thread, never the parent
  • Grok's subagent-completed-* reply (held at a gate until run 1 settled) is a provider continuation, run 2 (agent:provider)

Run 1 is finished with #13594's finish_held_run, on the adapter's finish receipt.

Without the handler, the fixture fails: the subagent stays running, so the adapter never arms run 1's finish. finish_held_run fails after its 60 s wait with OrchestratorV2ScenarioStepError … :actual=running:no_finish_armed, and #13594's teardown releases the held frames, so the test ends right there (71 s wall) instead of hanging.

Settled-root case

In the recorded fixture Grok keeps the root turn open, so subagent_finished always lands in the held turn. The other path, where the root turn already settled and the subagent is carryover, has an AcpAdapterV2 test: the root completes with the subagent still running, then:

  • a subagent_finished from another session is dropped (no row update; the run is still pinned)
  • a failed end from the root session finishes the carryover row at once with Grok's error as its result, and background work stops pinning the run

Each assertion fails when its branch is removed from the adapter.

Mock tests: kept, not deleted

I did not delete the mock-based monitor and subagent tests, because these fixtures don't reach their paths:

  • The <monitor-event task_id=…>, Monitor "…" ended and Background subagent "…" text forms are real Grok output: they're the reminder text Grok builds for its own wake prompts (crates/codegen/xai-grok-tools/src/reminders/task_completion.rs ~214/282/519). Live they show up only inside x.ai/queue/changed.runningText, never as user_message_chunk. Grok does persist them as host-turn user echoes (session/acp_session_impl/turn.rs ~421), so session/load history replay can still deliver them, and their parsers stay useful. Their XAiAcpExtension unit tests stay.
  • The AcpAdapterV2 tests on task-monitor-1 / tool-call-generic-1 drive the adapter through test flavors with their own extractBackgroundTaskId / extractSubagentUpdate, keyed by tool call id. They exercise the adapter's hold, carryover and wake machinery (including races a live agent can't produce on demand), not Grok parsing, so the mock frame shapes don't change what they prove.

Verification

  • vp test run on OrchestratorReplayFixtures.integration.test.ts, .contract.test.ts, AcpAdapterV2.test.ts, XAiAcpExtension.test.ts, GrokAdapterV2.test.ts: 273 passed. The Grok replays passed on 3 consecutive runs.
  • New tests: the subagent_finished parser (the three statuses, error as the failed result, an unknown status and a missing child id ignored), forwarding from the prompt runtime plus the die for any other runtime, and the settled-root carryover test above.
  • With the subagent_finished registration removed: grok_background_subagent fails as described above.
  • vp exec tsc --noEmit -p . in apps/server: 0 error TS / warning TS. vp run knip:check: clean. vp lint on touched files: no new warnings. Diff grepped for renamed service imports and raw data in errors: none.
  • Not run: repo-wide suites.

Stacked on #13594.

Model: Claude Opus 5.5 (Claude Code)

🤖 Generated with Claude Code


Devin Review

@macroscopeapp

macroscopeapp Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Macroscope skipped reviewing this pull request. Per-review cost limit exceeded (workspace setting).

This review would cost an estimated $15.74, which exceeds your per-review limit of $15.00.

The top 3 files driving up this estimate:

File Diff Size Estimate
apps/server/src/orchestration-v2/testkit/fixtures/grok_background_subagent/grok_transcript.ndjson 297.82KB $14.89
apps/server/src/provider/acp/XAiAcpExtension.ts 6.39KB $0.32
apps/server/src/orchestration-v2/testkit/fixtures/grok_background_subagent/output.ts 3.77KB $0.19

Tip

To get this pull request reviewed, you can:

  1. Comment @macroscope-app on this PR to request a manual review (monthly spend limits still apply).
  2. Exclude the file(s) above from review by adding a pattern to your .macroscope/ignore.md — note that creating this file replaces Macroscope's built-in default ignores rather than extending them.
  3. Raise your cost limit in your workspace billing settings.

Turn off this reminder going forward

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Sep 25, 2026
Comment on lines +1562 to +1565
},
);

const xAiSubagentFinishedRegistrations = new WeakMap<

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module-level WeakMap makes the runtime's notification registration an implicit global side channel: handleXAiSubagentFinished silently does nothing unless this exact wrapper previously populated the map. Please expose registration on the wrapped runtime's explicit service interface (or pass it as an explicit extension callback) so the dependency and unsupported-runtime behavior are visible without a module-global registry. The fix spans the runtime interface and consumer, so no single-hunk suggestion applies.

Posted via Macroscope — Effect Service Conventions

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire — 4.9 KiB — 6.8 KiB ✅
Codex Thread snapshot wire — 3.7 KiB — 4.9 KiB ✅
Codex Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Codex Live turn WebSocket decoded — 20.4 KiB — 29.3 KiB ✅
Codex Live turn messages — 2 — 8 ✅
Claude Total thread wire — 4.9 KiB — 6.8 KiB ✅
Claude Thread snapshot wire — 3.7 KiB — 4.9 KiB ✅
Claude Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Claude Live turn WebSocket decoded — 20.8 KiB — 29.3 KiB ✅
Claude Live turn messages — 2 — 8 ✅

Baseline: unavailable · PR result: ddb6f80 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 106.1 KiB
  • Claude decoded thread snapshot: 106.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@macroscopeapp

This comment has been minimized.

@macroscopeapp

This comment has been minimized.

@macroscopeapp

macroscopeapp Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This change modifies production Grok/ACP lifecycle handling across prompt completion, background tasks, subagent state, deferred finalization, and continuation runs. The cross-cutting timing and notification-dispatch changes, together with an unresolved design concern, warrant human review.

Not approved because:

  • Per-review cost limit exceeded (workspace setting). Approvability relies on correctness review in order to determine eligibility

Review your spending limits in Billing settings, or comment @macroscope-app review this PR to bypass the limit and review now. You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarminge force-pushed the v2/grok-subagent-finished branch 2 times, most recently from 7891911 to 47bd352 Compare September 25, 2026 17:03
Base automatically changed from v2/grok-gates to t3code/codex-turn-mapping September 25, 2026 22:35
juliusmarminge and others added 3 commits September 25, 2026 15:36
A background subagent that outlived its root turn never finished in T3.
Grok's spawn tool completes at launch ("Subagent started in background"),
and the subagent's end arrives only as a structured `subagent_finished`
notification on the root session. Nothing handled it, so the subagent
row stayed running and held the root run open indefinitely.

The Grok runtime now routes `subagent_finished` (child_session_id,
status, output) to the adapter, which finishes the subagent row with its
output: in the turn held open for it, or in the carryover of a root turn
that already settled. Grok then runs its own `subagent-completed-<id>`
wake turn, which replays as a provider continuation like Claude and
Codex background wakes.

grok_background_subagent is recorded live from Grok 1.0.41 and fails
without the handler.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…settled-root end

subagent_finished now yields a notice only for the statuses Grok defines
(completed, failed, cancelled) and carries Grok's `error` as the result of a
subagent that did not complete. handleXAiSubagentFinished dies for a runtime
that does not own Grok's session notifications instead of dropping ends.
Adds parser, forwarding and settled-root carryover tests, and finishes the
fixture's held runs on the adapter's finish receipt.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n handlers

The Grok adapter reached the subagent_finished forwarding through a
module-level WeakMap keyed by the wrapped runtime, a side channel that did
nothing visible unless the wrapper had filled it in.

Grok sends subagent_finished on the same session notification methods as
turn completion, and the ACP client keeps one handler per method. The
prompt-completion runtime now owns those methods openly: it settles prompts
from each notification, then passes it to whatever handler was registered
through its own handleExtNotification. registerXAiSubagentFinished
registers through that interface, the same way registerXAiBackgroundTaskTracking
registers task lifecycle handlers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@juliusmarminge
juliusmarminge force-pushed the v2/grok-subagent-finished branch from 241ce38 to ddb6f80 Compare September 25, 2026 22:41
@juliusmarminge
juliusmarminge merged commit 48b5111 into t3code/codex-turn-mapping Sep 25, 2026
24 checks passed
@juliusmarminge
juliusmarminge deleted the v2/grok-subagent-finished branch September 25, 2026 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL 500-999 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant