[bobleer]

Summary

Computer Use tasks were blocked by a manual application-sharing picker and a separate input grant, and the desktop-control settings page exposed a large diagnostic panel. Restore agent-directed application discovery, targeting, input, verification and recovery, and return settings to the standard three-row layout.

Type and Areas

Bug/regression fix. Desktop native adapters, ComputerUse/ControlHub orchestration, built-in agent prompts, Web UI settings and control-session contracts.

Motivation / Impact

• Explicit and implicit Agent startup no longer enter the optional manual-sharing flow. Existing manual-sharing clients and saved grants remain compatible.
• Allow supported background application launch on macOS/Windows. Resolve bundle IDs and explicit application paths without reporting the human's foreground app as the target. Linux reports its unsupported background-launch route explicitly.
• Preserve the native operation result when a concurrent human changes foreground/cursor state; retain diagnostic witnesses without cancelling or replaying the Agent action.
• Remove routine manual setup and duplicate message-composition confirmations from Computer Use and parent-agent guidance. Browser connection failures offer an available desktop-control route.
• Use the existing settings rows for the desktop-control toggle, Accessibility and Screen Recording. Missing-permission setup remains an explicit OS request; process diagnostics, Input Monitoring and manual-sharing controls are absent from this page.

User Stop, cancellation, OS permissions, explicit tool exclusions and native capability limits remain authoritative.

Verification

AI-assisted; tested locally on macOS with real native fixtures and a real Tauri settings window using an isolated profile.

• cargo test --locked -p openbitfun-core --no-default-features --features agent-runtime,git,tools-computer-use,tools-browser-web --lib computer_use_tool::tests: 51 passed; focused browser connection-hint tests: 2 passed.
• cargo test --locked -p openbitfun-desktop --lib computer_use: 163 passed, 9 native opt-in tests ignored. cargo build --locked -p openbitfun-desktop passed.
• cargo test --locked -p openbitfun-agent-content --test prompt_catalog_contracts: 8 passed. Focused control/permission contracts: 33 passed. node scripts/test-computer-use-context.mjs: 64 passed, 1 ignored.
• Real macOS test-macos-control-roundtrip.mjs and test-macos-input-controls.mjs --orchestration passed without manual-share setup: exact input counts, Unicode/OCR verification, background opening, non-AX canvas, shortcut/scroll, Stop, and preserved observer foreground/key/stacking/cursor.
• Frontend settings/control/sizing suites: 27 passed. check:web, build:web, focused ESLint, capabilities:check, check:repo-hygiene, check:core-boundaries and git diff --check passed.
• Windows production native modules cross-compiled with node scripts/test-windows-computer-use-native.mjs --check --target x86_64-pc-windows-msvc.

The broader minimal-feature Core filter also selects an existing registry test whose product-assembly expectation does not match the baseline tool plan; the owner suite passes. The Linux full-product CI suite passes all 2,573 tests, including registry_exposes_controlhub_and_computer_use (6 tests ignored). Linux Desktop library tests pass 431 tests (4 tests ignored, including the interactive portal and AT-SPI fixtures). Windows/Linux native delivery, cold application startup, fresh OS permission/restart flows and all four remote scenarios were not exercised by this correction. Historical evidence and exact limitations are recorded in docs/architecture/computer-use-control.md.

CI is complete on commit 2557c47371ece89f9ac7a4dd74bc23750c8f96d0: 11 checks pass, with 2 jobs skipped by the existing impact selector. Workflow run 38340300499 passes all three Rust/CLI OS jobs, frontend tests/builds and the aggregate validation gate. macOS Desktop library tests pass 497 tests (11 ignored). Windows compilation, CLI and independent platform contracts pass. Its Desktop test harness hits the existing Tauri #13419 pre-test loader signature (STATUS_ENTRYPOINT_NOT_FOUND); the unchanged CI policy tolerates that exact signature, so Windows Desktop library tests are not counted as executed.

Reviewer Notes

Native settings inspection shows only the standard desktop-control toggle and two permission rows. No new CSS tokens or persisted-format migration. Native fixtures use disposable applications and send no real messages. Background launch requests cannot guarantee every third-party application's activation behavior.

Checklist

• This PR is focused and does not include secrets, temporary prompts, generated scratch files, or unrelated artifacts.
• Relevant verification is recorded above, or skipped checks are explained.
• User-facing strings, docs, and locales are updated where applicable (existing localized settings labels reused).