Skip to content

feat(computer-use): add typed reminder, event and contact app commands behind a desktop capability - #10196

Merged
waleedlatif1 merged 12 commits into
stagingfrom
feat/computer-use-app-commands
Nov 19, 2026
Merged

waleedlatif1 merged 12 commits into
stagingfrom
feat/computer-use-app-commands

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • The helper gets an app_command method that runs allowlisted commands through EventKit, never Apple Events or scripts, so scripting apps stay blocked (fix(computer-use): refuse Script Editor, Automator and Shortcuts, which run shell commands outside the Terminal control #9990). It covers Reminders (list_reminder_lists, create_reminder), Calendar (list_calendars, create_event, with timed or all-day events up to 31 days and an optional alert) and Contacts (list_contact_groups, create_contact_group, which returns an existing group of the same name, and create_contact, with up to 5 emails and phones and an optional group). A date-time without an offset is the Mac's local time, as typing it into the app would be. Without the matching access a command fails with *_permission_required before touching anything. Status reports all three accesses without prompting.
  • The desktop service checks the app's usual Computer Use permission for Reminders, then asks the user to confirm every create in a dialog that shows the exact fields, with Cancel as the default. A declined create returns user_declined / not_started. Commands skip snapshot handling and the post-action observation.
  • The capability is advertised as computerUseAppCommands only by shells that support it, and only sent when computer use itself is allowed for the chat. The authorize route, transport and client tool executor accept the extended schema. The result message tells the model that a created item exists and must not be repeated.
  • Settings shows Reminders, Calendar and Contacts permission rows on shells that report them. Adds the usage strings and the personal-information.calendars and .addressbook entitlements for the hardened runtime.

Paired with mothership#983, which must land first (contract sync). Benchmarks show Reminders, Calendar and Contacts data entry costs about 18 GUI calls per item, and 60 to 130 calls per task.

Benchmark

Paired macOS VM runs (Reminders, Calendar and Contacts data-entry tasks, 2 runs each, same VM image, helper and worker build; the only difference is whether the desktop offers app commands):

GUI only App commands
Passed 6/6 5/6
Tool calls 367 137 (−63%)
Model cost $33.86 $8.57 (−75%)
Wall time 1630 s 539 s (−67%)

The miss read two people's values from the wrong duplicate rows of the source CSV; the commands saved exactly what they were given. Editing an existing contact in the same task still went through the GUI, so mixed tasks keep working.

Type of Change

  • New feature

Testing

  • service.test.ts: every write command (reminder, event, contact group, contact) asks for confirmation; a declined create never reaches the helper; a confirmed create runs once with no observation; app denial stops before the confirmation; list reads need no confirmation. Desktop ipc.test.ts and the desktop typecheck pass.
  • payload.test.ts: app commands are offered only where computer use is allowed. I confirmed this test fails with the guard removed.
  • Native tests cover date and event-boundary parsing, end-before-start, the 31-day cap, nameless contacts, malformed emails, unknown commands and invalid arguments; the helper typechecks against the macOS 14 target.
  • Targeted Sim vitest for settings, computer-tool, computer-use and payload: all pass. Full typecheck passed in the pre-push hook.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Companion: simstudioai/mothership#983

@vercel

vercel Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 19, 2026 6:15am UTC

Request Review

@waleedlatif1 waleedlatif1 changed the title feat(computer-use): add typed reminder app commands behind a desktop capability feat(computer-use): add typed reminder, event and contact app commands behind a desktop capability Nov 19, 2026
@github-actions github-actions Bot added the requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep label Nov 19, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Cross-repo companion check

One or more companion PRs aren't merged into staging yet. Merging this without them will leave copilot and sim out of sync — merge them in lockstep.

  • ❌ simstudioai/mothership#983 — OPEN, not merged (targets staging) — feat(computer-use): offer typed reminder, event and contact app commands to desk

@waleedlatif1
waleedlatif1 marked this pull request as ready for review November 19, 2026 05:30
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@waleedlatif1
waleedlatif1 force-pushed the feat/computer-use-app-commands branch from 7eb7430 to 612b864 Compare November 19, 2026 05:33
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@greptile-apps

greptile-apps Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Critical impact] The reviewed changes appear safe to merge.

Summary

Adds confirmed commands for Reminders, Calendar, and Contacts through macOS APIs. The latest change preserves seconds in reminder due times.

  • Chats can add reminders, events, and contacts with typed commands.
  • Every app-data create waits for the user’s confirmation.
  • App commands reach chats only when the desktop and chat allow them.
  • Settings show Reminders, Calendar, and Contacts access for Computer Use.

Reviews (9) · Last reviewed commit: "fix(computer-use): keep seconds in remin..." · Reviewed by Greptile

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed against the latest diff

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 23 files

Requires human review: Auto-approval blocked because this review re-detected 3 unresolved issues already reported by Cubic.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 25 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

Correction to my earlier replies: application/authorize.ts and lib/api/contracts/computer-use.ts were NOT switched before. The substitution I ran on them was a no-op, and I wrongly said they had been. Only repository.ts had the extended schema. That is fixed now in 48b77a0: the use case and the route contract both validate with ComputerUseWithAppCommandsSchema, authorize.test.ts returns a claimed create_reminder to the desktop (that test fails with the old schema), and a whole-word search finds no ComputerUseSchema outside generated files. Thanks for holding the score on this.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 28 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 28 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit e0b5ec1 into staging Nov 19, 2026
90 checks passed
@waleedlatif1
waleedlatif1 deleted the feat/computer-use-app-commands branch November 19, 2026 07:06

This branch was previously deployed

1 inactive deployment
Preview — ee79d323 Deployed Nov 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant