No advisory deterministic step: v2 gates every deterministic step on exit code, so repair-before-failure flows all re-invent '|| true' - #521
Conversation
… not hidden
v2 gated every deterministic step on exit code with no opt-out, so every
repair-before-failure flow re-invented `<command> || true`. That workaround
discards the exit code: a later gate has nothing to read, and a forgotten
gate is indistinguishable from success, so a flow can ship red work silently.
A deterministic step can now declare `onNonZero: record`. The kernel keeps
the command's real exit code and output tails in the journal, passes the step
so dependents run, and labels the gate `exit_code:recorded` so no reader can
mistake it for a green command. `f.run(cmd, { onNonZero: 'record' })` resolves
to a `RunResult` read back from that journaled envelope, so the branch below it
reads the code and output the command actually produced. A later step asserts
freshness with `{ type: steps_green, ids: [...] }` rather than re-running.
The policy covers exit codes only. Timeouts, signals, the `-1` no-exit-status
sentinel, declared content and schema gates, and budgets all stay fatal under
either policy, and the default is normalized out of the canonical bytes so
every existing spec keeps its exact hash.
Also fixes a pre-existing hole this feature would have widened: an authored
`.gate()` lowers to its own kernel step that runs after the producer, and
`readCompletedStepOutput` read only the producer's entry — so a failed gate
resolved the operation as though the command had passed. It now checks the
run's verdict too, which is the same invisibility the recording policy exists
to remove, one layer up.
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…are-garden-b99b1be8 # Conflicts: # kernel/relayflowd-core/src/spec/tests.rs # packages/sdk/src/authored-flow-executor.ts # packages/sdk/src/authored-step-output.ts # packages/sdk/src/authored-worker-step.ts
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d036684. Configure here.
The merge hoisted runOperation for onNonZero but dropped the node
argument, so f.run steps never entered the step graph and status --json
reported after: undefined. Restore the {} node (a command is not a
display label) on both the record and fail paths, matching main.
…Options) Picks up the #547 authored-verdict recovery fix on main. Verified: surface+sdk typecheck clean; advisory-outcome 23, authored-step-graph-live, authored-advisory-run-live 16, authored-completion-recovery 5 green.

Advisory deterministic outcomes:
onNonZero: recordand thesteps_greengateCloses the "no advisory deterministic step" gap. v2 gated every
deterministicstep on exit code with no opt-out, so every repair-before-failure flow
re-invented
<command> || true— including this repo's own v1→v2 bridge(
flows/spec-builder.ts) andflows/diagnose/orchestration.spec.ts'sadvisoryGate.|| truediscards the exit code. A later gate has nothing to read, so the floweither re-runs the command or builds an evidence journal outside the kernel.
Worse, it is indistinguishable from success: a flow that forgets one gate ships
red work silently.
What an author writes now
tests.ok,tests.exitCode,tests.stdoutandtests.stderrare all read backfrom the step's journaled
{exit_code, stdout_tail, stderr_tail}envelope.Nothing is re-run and nothing is re-measured — that is the claim
|| truecannot make.
Acceptance
verify.rsa_recorded_nonzero_exit_passes_its_gate_and_names_the_code;relayflowd/tests/advisory_deterministic.rsauthored-advisory-run-live.test.ts"resolves to the journaled exit code instead of ending the flow" (asserts the repair branch received2 failing tests)steps-green.ts+advisory-outcome.test.ts"steps_green lowering"Design decisions worth reviewing
Recording is a policy about exit codes, and only about exit codes. A
timeout, a signal, the executor's
-1no-exit-status sentinel, a declaredcontent or schema gate, and a budget all stay fatal under either policy.
-1in particular is refused rather than handed back as a plausible code, mirroring
the kernel's own rule — otherwise a killed process would reach an author's
if (!result.ok)as a real red verdict.Red is allowed, not invisible. The kernel labels the gate
exit_code:recordedand names the code in the verification detail;flows checkannotates the step with[onNonZero: record]. Reporting it as anordinary
exit_codegate would have moved the|| trueambiguity into theinspection output.
The default is normalized away at both boundaries.
onNonZero: failisdropped in
compileStepand again intoKernelStep, andOnNonZero::is_defaultskips it on the Rust side — so every spec written before this field keeps its
exact canonical bytes and its exact
spec_hash. A parity test asserts thatdirectly.
steps_greenis compiler-lowered, not a kernel gate. It becomes a separatefatal deterministic step bound to the sources' envelopes, so the kernel never
sees the SDK spelling.
testdata/advisory-repair.*pins the lowered output,which is what proves the kernel parses what the SDK actually emits. The
generated gate never inherits its host's recording policy, or a red gate could
not fail anything.
Overload order on
f.run. The two literal overloads come first so anomitted or
'fail'policy keeps theStep<string>every existing body iswritten against; only the literal
'record'widens the result. A thirdunion-returning overload covers a policy held in a variable, where the author
narrows it themselves rather than having one branch guessed for them.
packages/surface/tests/run-on-non-zero.test-d.tspins all of this, includingthat a recorded result does not assign to
string.A misspelled policy is refused, never defaulted. Falling back to the fatal
default would delete the branch the author wrote below the call. Refused in
validateSpecfor declarative specs and before any ordinal is consumed inf.run.Bug found and fixed along the way
An authored
.gate()lowers to its own kernel step (<id>.gate) that runsafter the producer.
readCompletedStepOutputread only the producer'sstep.completedentry, so a failed gate on a successful command resolved theoperation as though it had passed — the child run was terminally
failedandthe author never heard about it.
This reproduces on the default policy too, so it predates this change, but
it is the same invisibility the recording policy exists to remove, one layer up,
and
recordwould have widened its reach.readCompletedStepOutputnow alsochecks the run's own terminal reason from the entries it already reads.
authored-advisory-run-live.test.tscovers it under both policies and assertsthe failed step is named as
run-1.gate, not the producer.Evidence
cd kernel && sh ../ops/cargo.sh test— all suites pass, 0 failures(includes 6 new
verify.rstests, 2 newspec/tests.rstests, 2 newrelayflowd/tests/advisory_deterministic.rsintegration tests, and the newspec_parity.rsfixture test).npm --prefix packages/schema test— 78 pass (the newadvisory-repairfixture is picked up by the existing parity walk).
npm --prefix packages/surface test— 46 pass, 9 files.npm --prefix packages/surface run typecheck:regressions— clean, includingthe new
run-on-non-zero.test-d.ts.npm --prefix packages/sdk test— 2401 pass. 7 files fail, all confirmedfailing identically at pristine HEAD (verified by stashing and rebuilding):
live-kernel,webhook-live,mcp,provider-trigger-executor,communication-mixed-resumeneedkernel/target/{debug,release}/relayflowd,which
ops/cargo.shdeliberately builds outside the repo;stuck-run-triageandauthored-node-runtimehit a vitest module-duplicationissue for flow files imported from outside the package root.
byte-identical.
Four suites needed updating for this change rather than being broken by it:
verb-field-lint(its fail-closed pin demands a sample value for every newstep field),
validateandgate-contract(theunknown_gate_kindenumeration), and
preflight(its converse test requires every declaredrefusal kind to be reachable through the public boundary — the three
steps_greenkinds now have scenarios).Not in scope
Retry policy (
maxIterations) and agent-step failure semantics, per the ticket.Note
Medium Risk
Touches kernel verification, spec hashing, and authored step outcome reading—core flow control paths—with broad SDK/surface API surface, though defaults and canonical hashing are preserved and coverage is heavy.
Overview
Adds repair-before-failure for deterministic steps:
onNonZero: 'record'(YAMLon_non_zero) lets a command finish with a nonzero exit while journaling code and output tails, so dependents can branch on real evidence instead of|| true. The kernel gains anOnNonZeropolicy (verify.rslabels satisfied redsexit_code:recorded); timeouts, signals,-1, and other gates stay fatal.f.rungains overloads: default still returns stdoutstring;recordreturnsRunResult(ok,exitCode, streams). The SDK/compiler/schema expose the field withfailomitted from canonical bytes for hash stability.Adds declarative
steps_green, compiler-lowered to a separate fatal gate step that reads journaled envelopes (not re-run).flows checkannotates recording steps; docs inSURFACE.mddescribe the pattern.Also fixes authored
.gate()child runs:readCompletedStepOutputnow fails when the run ends non-success, so a failed gate step is not treated as a passing producer.Reviewed by Cursor Bugbot for commit d036684. Bugbot is set up for automated code reviews on this repo. Configure here.
Summary by cubic
Deterministic steps previously failed the flow on any nonzero exit, forcing repair flows to hide failures with
|| true. They can now useonNonZero: 'record'to continue with the real exit code and output journaled, whilesteps_greenprovides an explicit later gate for final success.New Features
onNonZeroto specs andf.run; recorded runs returnok,exitCode,stdout, andstderrread back from the journal.steps_green, which checks journaled exit codes without rerunning commands.failas the default, rejects invalid policies, and preserves existing canonical hashes.Bug Fixes
.gate()now reports failures from its generated gate step instead of resolving the producer as successful.f.runstep registration in the authored step DAG sostatus --jsonreportsafterinstead ofundefined.Written for commit 7c018b4. Summary will update on new commits.
Fixes #509