diff --git a/.github/workflows/cloud-runtime-artifact.yml b/.github/workflows/cloud-runtime-artifact.yml index c81cf655c..eb213bb3d 100644 --- a/.github/workflows/cloud-runtime-artifact.yml +++ b/.github/workflows/cloud-runtime-artifact.yml @@ -42,9 +42,22 @@ jobs: working-directory: kernel run: cargo build --locked --release -p relayflowd + - name: Install SDK dependencies + run: npm ci --prefix sdk + + - name: Test SDK and type-level authoring contracts + working-directory: sdk + run: | + npm run build + npm run typecheck:tests + ./node_modules/.bin/vitest run \ + tests/typed-output.test.ts \ + tests/validate.test.ts \ + tests/spec-parity.test.ts \ + tests/deterministic-llm.test.ts + - name: Build standalone flows CLI run: | - npm ci --prefix sdk mkdir -p dist/cloud-artifact-input bun build sdk/src/cli-executable.ts \ --compile \ diff --git a/docs/SURFACE.md b/docs/SURFACE.md index 4ea7f3131..a14ca5b59 100644 --- a/docs/SURFACE.md +++ b/docs/SURFACE.md @@ -91,6 +91,31 @@ No process runs between events: the handler wakes, executes to its next await, p **Project-config discovery:** starting in the flow file's directory, `flows check` walks parent directories through the filesystem root and selects the first readable `flows.json`. That nearest file is the whole project config; it is not merged with outer files. Its schema is `{ "cli"?: , "executors"?: [] }`; unknown keys fail closed as `config_invalid`. A nearer config therefore defines a self-contained nested project boundary and prevents accidental inheritance of outer credentials or executors. The selected path is printed with project-level resolutions and named in an unresolved-CLI refusal; if it declares no `cli`, outer configs remain shadowed. At gate 1, a trigger executor is considered registered only when its name is present in this author-written `executors` array; `flows check` does not yet contact a registry, broker, or RelayCron, and absence is `no_executor`. 7. **Two dialects, one journal.** Declarative YAML — data, fully preflightable, sage's compile target, gate 9's self-authoring output. Imperative TS — journal-memoized function, maximum ergonomics. YAML is canonical; TS is the power tool. TS preflights its declared surface (agents, helpers, tools, identity), not arbitrary control flow — declared honestly per covenant 2. +### Structured output declarations + +Declarative `llm` and `agent` steps may declare an `output` JSON Schema. This +is authoring sugar for the existing kernel `json_schema` verification gate; the +compiler removes `output` before the journal boundary and emits the schema as +`verification.json_schema`. Authors must choose either `output` or an explicit +`verification` block. Declaring both is ambiguous and fails closed. + +```yaml +- id: extract + type: llm + prompt: Return the actionable request as JSON. + output: + type: object + required: [actionable, request] + properties: + actionable: { type: boolean } + request: { type: string } +``` + +The declaration does not add a kernel primitive and does not yet infer a +TypeScript result type from arbitrary JSON Schema. Typed parsed values belong +to the imperative `f.llm` / `f.agent` surface once that surface has a real +consumer; the spec SDK does not publish an unchecked phantom type in advance. + The authoring surface deliberately narrows `steps: []`: `flows check` refuses it as `invalid_spec`, while the kernel accepts it. This is a chosen authoring-time narrowing, not a kernel guarantee. diff --git a/ops/reviews/20260902-1620-pr133-history.md b/ops/reviews/20260902-1620-pr133-history.md new file mode 100644 index 000000000..5a2be39d2 --- /dev/null +++ b/ops/reviews/20260902-1620-pr133-history.md @@ -0,0 +1,444 @@ +# PR #133 fresh exact-head history / RFC review + +- Review lens: RFC and history fit, backwards compatibility, issue #132 acceptance scope, regression risk, evidence reproducibility +- Repository: `AgentWorkforce/flows` +- Base: `a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2` +- Reviewed head: `5ab0fee366362de33ea1bd7d88a96daef1cad18a` +- Date: 2026-09-02 +- Verdict: PASS + +## Findings + +No blocking findings. + +The change fits RFC-0001 decision 13: `output` is an authoring-surface field which is erased before the journal boundary and lowered to the existing kernel `json_schema` verification primitive. No kernel files, step verbs, or journal vocabulary change. The implementation fails closed on malformed or ambiguous declarations, including direct callers of the public `toKernelSpec` boundary. + +The repair commit removes the speculative `OutputFromSchema` / generic result-type API introduced at the first commit, and the test-only TypeScript project makes the negative `@ts-expect-error` assertion part of ordinary `npm test` and exact-head artifact CI. This matches the repository rule against speculative abstraction and provides a real type gate. + +Backwards-compatibility scope is preserved: + +- Existing explicit `verification: { type: json_schema, ... }` authoring still compiles. +- A direct comparison below shows the legacy declaration and new `output` sugar produce byte-identical kernel specs. +- The previous-generation `workflows/` tree is unchanged at base versus head. This review therefore establishes non-interference and continued acceptance of the legacy declaration; it does not claim to have executed every previous-generation v1 workflow through an external v1 runner. + +PR and issue scope are honest. Issue #132 remains open. PR #133 says `Refs #132`, not `Closes #132`, and its body explicitly limits the change to slice 1. The issue's full done condition still requires imperative typed `f.llm` / `f.agent` parsed values, a parsed-value gate in the research flow, the published surface package, direct-run input, parallel dispatch, and other listed slices. This PR adds declarative YAML/object `StepSpec.output` schema sugar only and documents that limitation. + +## Constitution read + +I read both required files completely before assessing code: + +```text +$ wc -l AGENTS.md docs/RFC-0001-everything-is-a-relayflow.md + 75 AGENTS.md + 244 docs/RFC-0001-everything-is-a-relayflow.md + 319 total + +$ shasum -a 256 AGENTS.md docs/RFC-0001-everything-is-a-relayflow.md +3e542e190bd4375a105612cf119b18e39b7732be181126c5bcf75c37010b65b8 AGENTS.md +cf8c0f41b12dc37699aa348d10c1907dae8b7e8550a34ad976aac1349639c7b7 docs/RFC-0001-everything-is-a-relayflow.md +``` + +The complete files were read with `sed -n '1,160p' AGENTS.md` and the three +non-overlapping RFC commands `sed -n '1,90p'`, `sed -n '91,180p'`, and +`sed -n '181,280p'`. The hashes above pin the exact contents read without +substituting a prose summary for file output. + +Relevant RFC fit observed from that literal read: + +- RFC §1: `llm` is distinct from `agent`, has value output, and verification is the rail. +- Covenant 2: malformed/ambiguous declarations must fail closed before a run. +- Decision 13: surface vocabulary may be open, but it must compile to the closed kernel vocabulary. +- The open versioning question requires compatibility rather than implicit deprecation. + +## Exact checkout and change inventory + +```text +$ pwd +/Users/khaliqgant/AgentWorkforce/flows-132-typed-output-wt +$ git rev-parse HEAD +5ab0fee366362de33ea1bd7d88a96daef1cad18a +$ git rev-parse a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2 +a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2 +$ git branch --show-current +feat/v2-typed-outputs +$ git log --oneline --decorate --no-merges a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2..5ab0fee366362de33ea1bd7d88a96daef1cad18a +5ab0fee (HEAD -> feat/v2-typed-outputs, origin/feat/v2-typed-outputs) fix(sdk): enforce structured output contracts +81c49df feat(sdk): compile typed outputs to json_schema +``` + +```text +$ git diff --stat a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2..5ab0fee366362de33ea1bd7d88a96daef1cad18a + .github/workflows/cloud-runtime-artifact.yml | 15 +- + docs/SURFACE.md | 25 ++++ + sdk/package.json | 3 +- + sdk/src/compile.ts | 19 ++- + sdk/src/index.ts | 1 + + sdk/src/output-schema.ts | 22 +++ + sdk/src/spec.ts | 13 ++ + sdk/src/validate.ts | 7 +- + sdk/tests/live-kernel.test.ts | 36 ++++- + sdk/tests/typed-output.test.ts | 209 +++++++++++++++++++++++++++ + sdk/tsconfig.tests.json | 10 ++ + testdata/hn-monitor.flow.yaml | 33 ++--- + 12 files changed, 365 insertions(+), 28 deletions(-) + +$ git diff --name-status a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2..5ab0fee366362de33ea1bd7d88a96daef1cad18a +M .github/workflows/cloud-runtime-artifact.yml +M docs/SURFACE.md +M sdk/package.json +M sdk/src/compile.ts +M sdk/src/index.ts +A sdk/src/output-schema.ts +M sdk/src/spec.ts +M sdk/src/validate.ts +M sdk/tests/live-kernel.test.ts +A sdk/tests/typed-output.test.ts +A sdk/tsconfig.tests.json +M testdata/hn-monitor.flow.yaml +``` + +I inspected the complete patch, including both commits' net diff and then the +repair-only diff. The exact patch is reproducibly pinned by its literal hash, +line count, and per-file numstat: + +```text +$ git diff --find-renames --find-copies --no-ext-diff a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2..5ab0fee366362de33ea1bd7d88a96daef1cad18a | shasum -a 256 +cb334c3a6d3ae3dc9ecc007d06a599920e8111b8eec6d66038ee5881ddb685f2 - + +$ git diff --find-renames --find-copies --no-ext-diff a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2..5ab0fee366362de33ea1bd7d88a96daef1cad18a | wc -l + 588 + +$ git diff --numstat a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2..5ab0fee366362de33ea1bd7d88a96daef1cad18a +14 1 .github/workflows/cloud-runtime-artifact.yml +25 0 docs/SURFACE.md +2 1 sdk/package.json +18 1 sdk/src/compile.ts +1 0 sdk/src/index.ts +22 0 sdk/src/output-schema.ts +13 0 sdk/src/spec.ts +5 2 sdk/src/validate.ts +30 6 sdk/tests/live-kernel.test.ts +209 0 sdk/tests/typed-output.test.ts +10 0 sdk/tsconfig.tests.json +16 17 testdata/hn-monitor.flow.yaml + +$ git show --stat --oneline 81c49df32aebcbb1b21fc3f87eeae55cb2ed6ff1 +81c49df feat(sdk): compile typed outputs to json_schema + sdk/src/compile.ts | 22 +++++- + sdk/src/index.ts | 2 + + sdk/src/output-schema.ts | 35 +++++++++ + sdk/src/spec.ts | 17 ++++- + sdk/src/validate.ts | 7 +- + sdk/tests/live-kernel.test.ts | 36 +++++++-- + sdk/tests/typed-output.test.ts | 170 +++++++++++++++++++++++++++++++++++++++++ + testdata/hn-monitor.flow.yaml | 33 ++++---- + 8 files changed, 294 insertions(+), 28 deletions(-) + +$ git show --stat --oneline 5ab0fee366362de33ea1bd7d88a96daef1cad18a +5ab0fee fix(sdk): enforce structured output contracts + .github/workflows/cloud-runtime-artifact.yml | 15 ++++- + docs/SURFACE.md | 25 ++++++++ + sdk/package.json | 3 +- + sdk/src/compile.ts | 13 ++--- + sdk/src/index.ts | 1 - + sdk/src/output-schema.ts | 17 +----- + sdk/src/spec.ts | 14 ++--- + sdk/tests/typed-output.test.ts | 85 ++++++++++++++++++++-------- + sdk/tsconfig.tests.json | 10 ++++ + 9 files changed, 127 insertions(+), 56 deletions(-) +``` + +The repair-only patch was also inspected in full. It removes `OutputFromSchema` and generic `TOutput`, adds the dedicated test tsconfig and CI invocation, centralizes validation in `validateOutputDeclaration`, adds internal-normalization validation, and pins raw `toKernelSpec` success/conflict/malformed arms. + +## Issue #132 and PR claim check + +```text +$ gh issue view 132 --repo AgentWorkforce/flows --json number,title,state,url +{"number":132,"state":"OPEN","title":"v2 authoring ergonomics: close the gaps found in the research-flow / v1 / Smithers comparison","url":"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/AgentWorkforce/flows/issues/132"} + +$ gh pr view 133 --repo AgentWorkforce/flows --json number,title,state,baseRefName,headRefName,headRefOid,url +{"baseRefName":"main","headRefName":"feat/v2-typed-outputs","headRefOid":"5ab0fee366362de33ea1bd7d88a96daef1cad18a","number":133,"state":"OPEN","title":"feat(sdk): compile structured outputs to json_schema","url":"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/AgentWorkforce/flows/pull/133"} +``` + +The literal issue comments establish the migration and slice boundaries: + +```text +$ gh issue view 132 --repo AgentWorkforce/flows --comments +author: kjgbot +association: member +edited: false +status: none +-- +Implementation ownership started on 2026-09-02. + +- Active first slice: typed `llm` / `agent` outputs on branch `feat/v2-typed-outputs`, owned by Agent Relay worker `flows-132-typed-output`, based on merged main `a0d42ff`. This must compile to the existing `json_schema` primitive; no kernel vocabulary expansion. +- Next slices, each as a separately reviewed PR: publish `@relayflows/surface`; direct-run input; journaled `agents[].model` surface plus strict typo/model linting; gate-3 parallel dispatch; settle replayable data gates versus code gates; verb-specific unknown-field refusal. +- Migration policy: v1 remains the default and supported runtime while these v2 authoring gaps close. Stored version remains authoritative for resume/schedules. Deprecation is a later explicit decision, not implied by this issue. + +The Cloud runtime artifact foundation merged separately in #131. A real Cloud v2 proof can proceed on the deterministic surface while this issue gates broader author migration and the research-flow completion criteria. +``` + +The current PR body was read in full. Its relevant scope language is literal: + +```text +## Scope boundary + +This is issue #132 slice 1 only. It does not publish +`@relayflows/surface`, implement imperative typed return values, or close #132. +v1 explicit `verification:` remains supported. +``` + +That description matches the patch: no `research/` implementation or local shim is changed, no imperative `f.llm` / `f.agent` surface is published here, and issue #132 remains open. + +## Backwards compatibility evidence + +The previous-generation v1 workflow tree is byte-identical between base and head: + +```text +$ if git diff --quiet a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2..5ab0fee366362de33ea1bd7d88a96daef1cad18a -- workflows; then echo V1_WORKFLOW_TREE_UNCHANGED=true; else echo V1_WORKFLOW_TREE_UNCHANGED=false; fi +V1_WORKFLOW_TREE_UNCHANGED=true + +$ for f in workflows/review-swarm.yaml workflows/watchdog.yaml workflows/bootstrap-gate1.yaml workflows/drive.yaml; do printf '%s ' "$f"; git show a0d42ffbdc7fb60b42c0b5bea4f58408249b08a2:"$f" | shasum -a 256 | awk '{print $1}'; done +workflows/review-swarm.yaml 6343d8c9f1505a1cbd3660c507730aa371a3d72b6a4103fff815741f43377ca6 +workflows/watchdog.yaml e04a14e20dd1c3562e66901994dbf4533224581bc45262114c0c9c7aa72aa589 +workflows/bootstrap-gate1.yaml f7f74dd595f1a68c3d819fe7e01f2a4a07bb3d0bcbd314777f30c9d07a02035e +workflows/drive.yaml b691305d132c2b2bacb5b4e44a4e6efa7b7872e9f52d4201bda1f00e09ae58c4 + +$ for f in workflows/review-swarm.yaml workflows/watchdog.yaml workflows/bootstrap-gate1.yaml workflows/drive.yaml; do printf '%s ' "$f"; git show 5ab0fee366362de33ea1bd7d88a96daef1cad18a:"$f" | shasum -a 256 | awk '{print $1}'; done +workflows/review-swarm.yaml 6343d8c9f1505a1cbd3660c507730aa371a3d72b6a4103fff815741f43377ca6 +workflows/watchdog.yaml e04a14e20dd1c3562e66901994dbf4533224581bc45262114c0c9c7aa72aa589 +workflows/bootstrap-gate1.yaml f7f74dd595f1a68c3d819fe7e01f2a4a07bb3d0bcbd314777f30c9d07a02035e +workflows/drive.yaml b691305d132c2b2bacb5b4e44a4e6efa7b7872e9f52d4201bda1f00e09ae58c4 +``` + +Legacy explicit verification and the new sugar lower identically: + +```text +$ PATH=/Users/khaliqgant/.local/share/mise/installs/node/22.23.2/bin:/usr/bin:/bin node --input-type=module -e '