Skip to content

fix(nightly-v2): prevent false cancellation by concurrent runtimes - #87

Draft
saphid wants to merge 1 commit into
mainfrom
agent/delegated-task-false-cancel
Draft

saphid wants to merge 1 commit into
mainfrom
agent/delegated-task-false-cancel

Conversation

@saphid

@saphid saphid commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

What Changed

Pin the V2 runtime ownership fix into Fork Nightly. A second server using the same state directory now fails before constructing runtime services, so it cannot cancel work owned by the running server. The lock stays held through shutdown cleanup and the OS releases it on process exit.

Fork main carries release configuration rather than V2 source. This PR selects one source commit after the existing 14 patches. No merge or installation is performed by this PR preparation.

Why

The cancellations at 2026-09-22 15:49:33Z and 23:42:19Z came from command:runtime-reconcile:startup, not a user cancel or WebSocket disconnect. Recovery marked persisted runs cancelled and sessions stopped without contacting the process that owned them. The second batch happened while the standalone server and its Claude child were still running. Historical process output identifies PID 26669 as the desktop's Electron executable parenting provider CLIs, consistent with its embedded Node server. Its full script argument was truncated in the capture.

The fix holds an exclusive transaction on a separate SQLite ownership file for the server lifetime. It does not lock the conversation database, use expiring heartbeats, or signal another process. Codex, Claude, Cursor, Grok, OpenCode and the remaining V2 adapters all use the guarded server entry point. Web, desktop and mobile share the server behavior; wire contracts are unchanged.

Late completion remains unable to revive a cancelled run because finalization requires a current active attempt. Existing false cancellations are not repaired automatically, which avoids reviving deliberate cancellations or duplicating a relaunched lane. Both standalone and desktop-bundled servers must be upgraded because old builds do not acquire this lock. The sibling 429 fix can be sequenced independently: this patch does not edit provider terminal-status handling.

Verification

  • Real startup-recovery regression failed before the guard: expected running, received cancelled.
  • On the assembled V2 stack, vp test run apps/server/src/serverRuntimeLock.test.ts apps/server/src/serverRuntimeOwnership.test.ts apps/server/src/orchestration-v2/ProviderRuntimeOwnership.test.ts apps/server/src/orchestration-v2/ProviderRuntimeRecoveryService.test.ts apps/server/src/serverRuntimeStartup.test.ts: 26 passed.
  • Server vp exec tsc --noEmit -p apps/server/tsconfig.json: passed. Focused vp lint and formatting: passed.
  • Tests cover the real server entry point, recovery exclusion, independent homes, directory aliases, shutdown ordering, failed startup and cross-process lock release after killing only the test's captured child.
  • Complete 15-patch replay passed on OV2 060756de5adcda82fedace42d8c47b449bc1097b; the resulting source matches the tested tree apart from generated release metadata. node --test .github/scripts/downstream-nightly.test.mjs: 22 passed. V2 manifest parses successfully.
  • Claude Fable 5.1 at high effort reviewed source 0eca7a70b426321775e14fd82dbf77c1a10e0694 through delegate_task: no blocking findings. It independently passed the seven new tests, server typecheck, and focused lint.
  • Desktop retains its existing retry behavior when a bundled backend loses ownership. Showing the lock failure clearly in the desktop is a follow-up; this change prevents the rejected backend from altering live work.
  • CI is blocked on unavailable Blacksmith runners. Check, Test, Release Smoke, and Mobile Native Static Analysis remain queued with no assigned runner; fork main has an older run queued since 2026-09-22 08:01Z. Nightly manifest validation passed. The PR stays draft until CI can run.
  • No browser verification or live-state writes. No production server was killed, restarted or signalled. This is a backend lifecycle fix with no UI layout changes.

Checklist

  • This PR is small and focused
  • I explained what changed and why

Implementation: GPT-6 Astra in Codex through T3 Code. Independent review through delegate_task: Claude Fable 5.1, high effort.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XS labels Sep 23, 2026
@saphid

saphid commented Sep 23, 2026

Copy link
Copy Markdown
Owner Author

Cross-vendor review completed through delegate_task using claudeAgent / claude-fable-5-1, effort high, for pinned source 0eca7a70b426321775e14fd82dbf77c1a10e0694. Verdict: no blocking findings. The reviewer independently ran the seven new tests, server typecheck, and focused lint successfully, and checked lock acquisition, scope finalization, service-update handoff and packaged-runtime support.

Advisory dispositions:

  • Desktop retries a rejected backend startup with its existing capped backoff rather than displaying the ownership error clearly. Recorded as a follow-up in the PR description. The rejected backend cannot reconcile or shut down the active owner's work.
  • The event store proves startup reconciliation caused the cancellations, but does not record the writer PID. The historical process capture supports the embedded-server explanation. The PR preserves that evidence limit and the requirement to upgrade both runtime installations.

Delivery remains blocked by CI runner availability, not by a known code defect. All four CI jobs request Blacksmith labels and have no assigned runner; the repository runner endpoint reports zero runners. Fork main already has a CI run queued since 2026-09-22 08:01Z. The GitHub-hosted Nightly validation job passed. No workflow-runner changes are bundled into this lifecycle fix.

@github-actions

Copy link
Copy Markdown

Thread transfer impact

⚠️ The latest CI run did not produce a thread transfer result for 9bd270e.

This comment will update automatically after the next completed run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant