Skip to content

feat(computer-use): add typed reminder, event and contact app commands behind a desktop capability - #10196

Draft
waleedlatif1 wants to merge 2 commits into
stagingfrom
feat/computer-use-app-commands
Draft

waleedlatif1 wants to merge 2 commits into
stagingfrom
feat/computer-use-app-commands

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • The helper gets an app_command method that runs allowlisted commands through EventKit, never Apple Events or scripts, so scripting apps stay blocked (fix(computer-use): refuse Script Editor, Automator and Shortcuts, which run shell commands outside the Terminal control #9990). It covers Reminders (list_reminder_lists, create_reminder), Calendar (list_calendars, create_event, with timed or all-day events up to 31 days and an optional alert) and Contacts (list_contact_groups, create_contact_group, which returns an existing group of the same name, and create_contact, with up to 5 emails and phones and an optional group). Dates must be date-only or carry an explicit offset. Without the matching access a command fails with *_permission_required before touching anything. Status reports all three accesses without prompting.
  • The desktop service checks the app's usual Computer Use permission for Reminders, then asks the user to confirm every create in a dialog that shows the exact fields, with Cancel as the default. A declined create returns user_declined / not_started. Commands skip snapshot handling and the post-action observation.
  • The capability is advertised as computerUseAppCommands only by shells that support it, and only sent when computer use itself is allowed for the chat. The authorize route, transport and client tool executor accept the extended schema. The result message tells the model that a created item exists and must not be repeated.
  • Settings shows Reminders, Calendar and Contacts permission rows on shells that report them. Adds the usage strings and the personal-information.calendars and .addressbook entitlements for the hardened runtime.

Draft: a prototype paired with mothership#983, which must land first (contract sync). Benchmarks show Reminders, Calendar and Contacts data entry costs about 18 GUI calls per item, and 60 to 130 calls per task. I will compare call counts on the macOS benchmark before taking this out of draft.

Type of Change

  • New feature

Testing

  • service.test.ts: every write command (reminder, event, contact group, contact) asks for confirmation; a declined create never reaches the helper; a confirmed create runs once with no observation; app denial stops before the confirmation; list reads need no confirmation. Desktop ipc.test.ts and the desktop typecheck pass.
  • payload.test.ts: app commands are offered only where computer use is allowed. I confirmed this test fails with the guard removed.
  • Native tests cover date and event-boundary parsing, end-before-start, the 31-day cap, nameless contacts, malformed emails, unknown commands and invalid arguments; the helper typechecks against the macOS 14 target.
  • Targeted Sim vitest for settings, computer-tool, computer-use and payload: all pass. Full typecheck passed in the pre-push hook.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Companion: simstudioai/mothership#983

@vercel

vercel Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 19, 2026 1:12am UTC

Request Review

@waleedlatif1 waleedlatif1 changed the title feat(computer-use): add typed reminder app commands behind a desktop capability feat(computer-use): add typed reminder, event and contact app commands behind a desktop capability Nov 19, 2026
@github-actions github-actions Bot added the requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep label Nov 19, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Cross-repo companion check

One or more companion PRs aren't merged into staging yet. Merging this without them will leave copilot and sim out of sync — merge them in lockstep.

  • ❌ simstudioai/mothership#983 — OPEN, not merged (targets staging) — feat(computer-use): offer typed reminder, event and contact app commands to desk

This branch was previously deployed

1 inactive deployment
Preview — 50afcc3d Deployed Nov 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant