Skip to content

feat(desktop): add opt-in Cua Driver computer use (temporary testing PR) - #11709

Closed
juliusmarminge wants to merge 5 commits into
mainfrom
temp/cua-driver-10899
Closed

juliusmarminge wants to merge 5 commits into
mainfrom
temp/cua-driver-10899

Conversation

@juliusmarminge

Copy link
Copy Markdown
Member

Temporary draft for checking out and testing the rebased Cua Driver integration on another machine. Carries the original work by Francesco Bonacci from #10899, whose organization-owned fork does not permit our pushes.

Adds default-off Cua computer use for Codex sessions, with desktop and standalone host support. Rebased onto main, adapted the Settings and tool-capability conflicts, and fixed the Effect service conventions findings: dependency acquisition, namespace imports, structured failures with preserved causes, and documented diagnostic suppression.

Validation: 154 focused tests passed, including desktop lifecycle and telemetry unit tests; server and desktop typechecks, targeted lint, and formatting passed. Earlier rebase verification passed 372 server/contracts/packaging tests and web/contracts typechecks. Desktop unit tests used mocked native hosts and an Electron path override to avoid a download on a disk-constrained machine.

This PR is for checkout and testing, not a merge-readiness claim. Current UI screenshots and native end-to-end verification have not been refreshed after the rebase. Original integration evidence and remaining platform limitations are recorded in #10899.

Original implementation by Francesco Bonacci with Codex. Rebase and convention fixes by GPT-6 in Codex.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). labels Sep 14, 2026
@github-actions

github-actions Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 13.6 KiB 13.6 KiB −16 B (−0.1%) 15.1 KiB ✅
Codex Thread snapshot wire 7.0 KiB 7.0 KiB −2 B (−0.0%) 7.3 KiB ✅
Codex Live turn WebSocket wire 6.6 KiB 6.6 KiB −14 B (−0.2%) 7.8 KiB ✅
Codex Live turn WebSocket decoded 57.1 KiB 57.1 KiB 0 B (0.0%) 66.4 KiB ✅
Codex Live turn messages 10 10 0 (0.0%) 21 ✅
Claude Total thread wire 13.6 KiB 13.6 KiB −12 B (−0.1%) 15.1 KiB ✅
Claude Thread snapshot wire 7.1 KiB 7.1 KiB +1 B (+0.0%) 7.3 KiB ✅
Claude Live turn WebSocket wire 6.6 KiB 6.5 KiB −13 B (−0.2%) 7.8 KiB ✅
Claude Live turn WebSocket decoded 57.9 KiB 57.8 KiB −44 B (−0.1%) 66.4 KiB ✅
Claude Live turn messages 10 9 −1 (−10.0%) 21 ✅

Baseline: cba7dd7 · PR result: 468818d · Source CI: failure

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 113.9 KiB
  • Claude decoded thread snapshot: 114.6 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@juliusmarminge juliusmarminge added the preview:mac Build an Apple Silicon DMG for this PR on every push. label Sep 14, 2026
`build-exe` rejected the bundle because CuaDriver reached the SDK through
`import()`, which cannot load file-backed modules inside a Node SEA. Load it
through createRequire like node-pty and fff-node; the package only exposes an
`import` export condition, so resolve its directory and require the entry
file by absolute path. The web build also failed on third-party license
generation because @trycua/cua-driver and @ubjs/* ship no notice file; add
repository overrides for both upstreams.

Verified by building the linux-x64 CLI archive and running Codex with the
managed Cua MCP against a headless Xvfb display.

Claude Fable 5 via Claude Code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

macOS preview

The preview download was removed because this PR closed or the preview label was removed.

Drop the promise wrapper and node:fs/node:path usage from the SDK loader.
The loader is now a lazy Effect.fn over FileSystem and Path, fails with
CuaDriverSdkLoadError directly, and the standalone factory takes it as a
plain Effect so tests inject Effect.succeed / Effect.fail instead of async
thunks. createRequire stays because it is the only way a single-executable
reaches a file-backed package.

Claude Fable 5 via Claude Code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juliusmarminge

Copy link
Copy Markdown
Member Author

Testing done; the three follow-up commits now live on the fork branch behind #10899 (fb7e10f, b8781e4, b0d2ee7).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

preview:mac Build an Apple Silicon DMG for this PR on every push. size:XXL 1,000+ changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants