Skip to content

feat(workflow-code): workflows and workspace as code - #10129

Merged
waleedlatif1 merged 1255 commits into
stagingfrom
feat/workspace-code
Nov 19, 2026
Merged

waleedlatif1 merged 1255 commits into
stagingfrom
feat/workspace-code

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Supersedes #10128: this PR now carries workflows as code and workspace as code together, both already reviewed.

Summary

Workflows as code

  • A workflow can be read as typed TypeScript and applied back: sim workflows source get | create | apply with --dry-run, a generated .sim/AGENTS.md and cookbook, and stable exit codes.
  • @sim/workflow-code: compiler (TypeScript → canonical model), printer, materializer (canonical model → stored graph, keeping block ids through a lock file), diagnostics with file:line frames and hints, size and depth limits.
  • @sim/workflow-sdk: the typed SDK generated from the block catalog (agent, condition, table.queryRows, trigger.schedule, …) with workspace resources by stable key (table('tickets'), credential('slack'), skill('tone'), customTool('lookup')).
  • v2 API: GET/POST /api/v2/workflows/{id}/source, create, the source catalog and resources manifest. Every route goes through an authorized application use case; writes are rate limited, refuse workspace API keys, and answer 409 when the draft changed since the read.
  • Code tab: a read-only TypeScript view of the open workflow that refreshes as the canvas or a collaborator changes it.

Workspace as code

  • A directory describes a workspace — tables/, knowledge/<key>/ (settings + docs/), files/, tools/, skills/, workflows/<key>/ — and sim workspaces source get | apply --dir reads it back or applies it in one run, including a workflow and the table, knowledge base, tool or skill it uses.
  • @sim/workspace-code: one codec per kind, a statically parsed declaration reader (never executed), the base file, limits and diagnostics in the same shape. SDK subpath @sim/workflow-sdk/workspace (defineTable, column.*, defineKnowledgeBase, defineTool).
  • Server: GET/POST /api/v2/workspaces/{id}/source and a knowledge document download route; every write goes through the existing domain use case and its authorization.
  • Safety: a three-way rule (local / base / server) writes only what the directory changed and answers 409 when the server changed it too; nothing deleted on the server comes back; deletes need --allow-delete, show what they lose, re-check the resource right before writing, and are refused while a stored workflow still uses it; binary files and documents travel in bounded, resumable rounds with hashes checked before anything is written; a partial apply converges on re-run.

Changes outside the feature flag

  • References inside a JSON string of a JSON code field (e.g. {"summary": "<agent.content>"} in a Response block or table row data) are now JSON-escaped when the value would break the literal (a raw quote, newline or control character). A value that is already valid JSON string content, such as a PEM key with \n escapes, is spliced unchanged, as before.
  • Editing engine (Copilot edit_workflow and POST /api/v2/workflows/{id}/operations):
    • A new or rewired edge that would close a cycle is skipped as edge_cycle, matching the canvas. Cycles already stored are kept.
    • A new error edge from a trigger or a loop/parallel container is skipped (invalid_source_handle). Stored ones are kept.
    • Row-list normalization no longer replaces a field that merely shares a name with a table field (e.g. supabase params) with [].
    • Gates are the shared BLOCK_GATES, so nested children get the same placement checks as top-level blocks.
  • Switch and slider defaults are seeded as booleans and numbers (they were strings); every consumer accepts both.
  • A buffered knowledge document upload now hands its processing to the dispatcher right after the registering commit (the same backend choice the upload-session path makes), in the background; the outbox row stays the durable fallback.
  • The row-locked recheck before a source write compares edges as a set (the edges read has no ORDER BY), removing a spurious 409.
  • sim CLI: auth failures exit 13 and rate limits exit 9 (both were 1); GETs retry a 429 up to 6 times, waiting at most 60s each.

Everything else is behind the global workflow-code flag (off by default). Migration 0466 adds the new tables and a nullable document.file_sha256; additive only.

Companion: simstudioai/mothership#986 gates the workflows source / workspaces source commands on the workflow-code entitlement. This PR merges first.

Type of Change

  • New feature
  • Bug fix

Testing

  • Round-trip corpus over every block type at 100% on all properties; property and hostile-input tests for compile, canonicalize, materialize, print and every workspace codec.
  • Real-Postgres integration suites for workflow source (concurrency and staleness, locks, permissions, credential and env visibility, governance) and for every workspace kind plus a cross-kind suite (stale 409s, delete gates, in-use refusals, partial-failure convergence, blob rounds, permissions, limits).
  • HTTP e2e for workflow source read/apply, the workspace source CLI (get → edit → apply → get), and a two-page Code tab UI e2e.
  • Agent dogfooding: agents built the same backends through code vs. today's CLI — the workflow path with the cookbook used ~42% fewer tokens and finished ~5x faster; the workspace path used ~2.75x fewer calls in half the time, caught a column rename before it shipped, and lost no data on a destructive-instruction trap.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 19, 2026 8:41pm UTC

Request Review

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 152 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread packages/workspace-code/src/layout.ts Resolved
Comment thread apps/sim/package.json Resolved
@greptile-apps

greptile-apps Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge, with two non-blocking CLI fixes recommended.

Findings

  1. P2 Downloads erase child input types ▶
  2. P2 Expired logins still exit 1 ▶

Summary

Adds typed workflow and workspace source files, CLI editing commands, guarded API writes, and a read-only Code tab.

  • Workflow drafts can be read, edited as TypeScript, and applied back.
  • A workspace directory can describe resources and apply their changes together.
  • Knowledge documents can be downloaded and their uploads start processing sooner.

Earlier declaration-test feedback was withdrawn after waleedlatif1 explained that these tests cover generated SDK files outside the normal repository type-check.

Reviews (23) · Last reviewed commit: "Merge branch 'fix/wc-thr7' into feat/wor..." · Reviewed by Greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found and verified against the latest diff

Confidence score: 3/5

  • table.ts accepts declarations with more than 100 options, so they parse successfully but fail when applied. Enforce MAX_SELECT_OPTIONS during parsing.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/workspace-code/src/kinds/table.ts">

<violation number="1" location="packages/workspace-code/src/kinds/table.ts:168">
P2: The codec accepts more than the server's 100-option limit, so declarations with 101+ options parse successfully and only fail during apply. Enforce `MAX_SELECT_OPTIONS` while parsing the declaration.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 171 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 173 files

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 173 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic

…names

compileSource now returns the stored model it compiled against, re-keyed by the
source's renames, and keptCredentials pairs compiled blocks with it, so a member
who cannot see a credential can rename the block holding it.
…es and branches on bound blocks

The offline check compiles without the stored workflow, so it refused an
onError on a loop or parallel and a cycle that a get printed and the server
keeps, and told the agent to delete them. compileWorkflow takes storedUnread:
an edge between two lock-bound blocks may be stored, so it reports
W_ERROR_PORT_STORED / W_EDGE_CYCLE saying an apply keeps it only if the stored
workflow holds it, and the error stays for a new block. E_ERROR_PORT_UNSUPPORTED
hints no longer say to remove the edge. W_BRANCH_UNWIRED never fails the check
on a bound block, --strict included, matching its stored exemption.
# Conflicts:
#	packages/db/migrations/meta/0465_snapshot.json
#	packages/db/migrations/meta/_journal.json
…ough its lock ident

storedUnread now takes the lock, and compile counts a source block whose ident is one vacated
lock entry's as bound (lockIdentRenames, materialize's lock-ident rule, shared with
compileSource), so a renamed block's stored container error edge or stored cycle warns offline
as the server accepts it.
… a workflows check

A per-folder `sim workflows source check` in a workspace directory already
compiled every workflow's Start inputs for its child-call findings but left
workspace.d.ts on the stored inputs, so a parent mapping an input added to the
child since the get failed tsc with an excess-property error. It now refreshes
workspace.d.ts the way `sim workspaces source check` does.
…cycle skips

The edit engine (Copilot edits, v2 operations) could create an edge that
closes a cycle, which the canvas and realtime refuse and the executor
cannot run. createValidatedEdge now runs the shared wouldCreateCycle check
and reports a refused edge as an edge_cycle skipped item. Edges the
workflow already stored are exempt, so a stored cycle survives unrelated
edits and restated connections. Publishes edge_cycle in the v2 skipped-item
enum and regenerates the OpenAPI document and CLI types.
@github-actions github-actions Bot added the requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep label Nov 19, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Cross-repo companion check

One or more companion PRs aren't merged into staging yet. Merging this without them will leave copilot and sim out of sync — merge them in lockstep.

  • ❌ simstudioai/mothership#986 — OPEN, not merged (targets staging) — feat(worker): gate workflow-code CLI commands on the workflow-code entitlement

@waleedlatif1
waleedlatif1 merged commit bc8f09c into staging Nov 19, 2026
97 checks passed

This branch was previously deployed

1 inactive deployment
Preview — c6b6588d Deployed Nov 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant