Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions docs/SURFACE.md
Original file line number Diff line number Diff line change
Expand Up @@ -537,6 +537,128 @@ the kernel kills the command's process group and journals `completionReason: tim
`f.run` refuses with code `lease_exceeded`. The override applies only to that
invocation; calls without options retain the default.

### Repair before failure: `onNonZero: 'record'`

A deterministic step is gated on its exit code by default: a nonzero exit fails
the step and ends the flow. That is the right default, and it is the wrong one
for the single most common shape in practice — run a check, let it be red, and
hand its output to whatever is built to answer it.

`onNonZero: 'record'` is the opt-in for that shape. The command still runs, its
exit code and output tails are still journaled, and the step still completes;
what changes is that a positive exit code satisfies the exit check instead of
failing it, so dependents run. In TypeScript the result is a value rather than
a throw:

```ts
const tests = await f.run('npm test', { onNonZero: 'record' });
if (!tests.ok) {
await f.agent('fixer', { task: `Fix these failures:\n${tests.output}` });
}
```

`RunResult` is `{ ok, exitCode, stdout, stderr, output }`. Every field is read
back from the step's journaled outcome, never re-measured — `ok` is exactly
`exitCode === 0`. `stdout` and `stderr` are the journal's **bounded tails**, not
complete transcripts; `output` is `stdout` then `stderr` joined by a newline
when both are nonempty, a reading convenience rather than a reconstruction of
the real interleaving. Without the policy, `f.run` still resolves to the stdout
string, so existing bodies are unchanged.

This is what `<command> || true` cannot do. `|| true` throws the exit code
away, so nothing downstream can tell a passing check from a failing one, and a
flow that forgets a later gate ships red work silently. `onNonZero: 'record'`
keeps the code, which is the whole point: the branch below it is a fact, not a
guess.

**The policy covers exit codes only.** A timeout, a signal or a command that
never started produced no verdict to record, and still fails the step under
either policy. Declared `output_contains` and `json_schema` gates are still
enforced in record mode, as are budgets and journal-append failures. Recording
is not a retry policy and does not trigger semantic retries.

**Recording makes red allowed, not invisible.** `flows check` annotates a
recording step with `[onNonZero: record]`, and the kernel labels the satisfied
check `exit_code:recorded` with the numeric code in its detail, so a recorded
red outcome reads as red in the journal. A flow that records without ever
asserting green may still finish successfully — that outcome is inspectable,
not forbidden.

**Asserting green again.** To demand that recorded commands were green, use the
declarative `steps_green` gate, which reads the journaled outcomes and never
re-runs anything:

```yaml
version: '0.1.0'
steps:
- id: tests
type: deterministic
command: npm test
onNonZero: record
- id: lint
type: deterministic
command: npm run lint
onNonZero: record
- id: gate
type: deterministic
command: 'true'
verification:
type: steps_green
ids: [tests, lint]
```

`steps_green` is deterministic-hosts-only: a worker step records no exit code.
The ids it names must be deterministic steps that precede it; unknown, forward,
self and non-deterministic references are refused at compile time, as are empty
and duplicate id lists. The host keeps its own command and its own exit policy —
the assertion is lowered to a separate, always-fatal gate step that every
dependent of the host waits on.

A recorded outcome is immutable, so repair does not turn an old red step green.
A flow that repairs must produce **new** post-repair evidence and gate on that
distinct step:

```yaml
version: '0.1.0'
steps:
- id: tests
type: deterministic
command: npm test
onNonZero: record
# Declaring the envelope schema is what lets a later step bind the evidence.
verification:
type: json_schema
schema:
type: object
properties:
exit_code: { type: integer }
stdout_tail: { type: string }
stderr_tail: { type: string }
- id: repair
type: agent
dependsOn: [tests]
input:
failures: { step: tests }
instruction: Read input.failures and fix the failing tests.
- id: retest
type: deterministic
dependsOn: [repair]
command: npm test
- id: gate
type: deterministic
command: 'true'
verification:
type: steps_green
ids: [retest]
```

`retest` is an ordinary gated step, so the final `steps_green` over it is what
decides the run. Binding a recorded outcome as input requires the source to
declare an output schema, as every input binding does (see *Declarative output
binding*); an `output_contains` source is refused, because that gate cannot also
carry the envelope schema the binding reads — declare the constraint as a JSON
Schema instead.

### The authored operation lifecycle

An authored TypeScript body reaches `done()` only if every step it created was
Expand Down
30 changes: 29 additions & 1 deletion kernel/relayflowd-core/src/spec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ const STEP_COMMON_FIELDS: &[&str] = &[
"memory",
"requirements",
];
const STEP_DETERMINISTIC_FIELDS: &[&str] = &["command", "timeout_ms", "lease_ms"];
const STEP_DETERMINISTIC_FIELDS: &[&str] = &["command", "timeout_ms", "lease_ms", "on_non_zero"];
const STEP_LLM_FIELDS: &[&str] = &["prompt", "model", "cli"];
const STEP_AGENT_FIELDS: &[&str] = &[
"instruction",
Expand Down Expand Up @@ -406,6 +406,29 @@ impl StepSpec {
}
}

/// What a deterministic step's nonzero exit code means.
///
/// `Fail` is the kernel's long-standing implicit gate: `exit_code == 0` or the
/// step failed. `Record` is the repair-before-failure shape — the command is
/// allowed to be red, its exit code and output tails stay in the journal
/// exactly as executed, dependents run, and a later step reads the recorded
/// outcome instead of re-running the command. It is a policy for the step's
/// own exit code only; timeouts, worker errors, and declared content or schema
/// gates keep their existing fatal semantics.
#[derive(Debug, Clone, Copy, Default, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum OnNonZero {
#[default]
Fail,
Record,
}

impl OnNonZero {
fn is_default(&self) -> bool {
matches!(self, OnNonZero::Fail)
}
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum StepKind {
Expand All @@ -415,6 +438,11 @@ pub enum StepKind {
timeout_ms: Option<u64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
lease_ms: Option<u64>,
/// What a nonzero command exit means. Omitted on serialization when
/// it is the default, so every spec written before this field exists
/// keeps its exact canonical bytes and its exact hash.
#[serde(default, skip_serializing_if = "OnNonZero::is_default")]
on_non_zero: OnNonZero,
},
Llm {
prompt: String,
Expand Down
86 changes: 86 additions & 0 deletions kernel/relayflowd-core/src/spec/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,92 @@ fn preflight_data_is_fail_closed() {
);
}

/// `on_non_zero` is a declared policy, so it parses fail-closed like every
/// other spec field: an unrecognized value is a refusal, never a silent
/// fallback to the fatal default.
#[test]
fn on_non_zero_parses_the_two_declared_policies_and_refuses_anything_else() {
let recorded = RunSpec::parse(&json!({
"steps": [{
"id": "tests", "type": "deterministic", "command": "npm test",
"on_non_zero": "record"
}]
}))
.unwrap();
assert!(matches!(
recorded.steps[0].kind,
StepKind::Deterministic {
on_non_zero: OnNonZero::Record,
..
}
));
assert_eq!(recorded.validate(), Ok(()));

let explicit_fail = RunSpec::parse(&json!({
"steps": [{
"id": "tests", "type": "deterministic", "command": "npm test",
"on_non_zero": "fail"
}]
}))
.unwrap();
assert!(matches!(
explicit_fail.steps[0].kind,
StepKind::Deterministic {
on_non_zero: OnNonZero::Fail,
..
}
));

assert!(matches!(
RunSpec::parse(&json!({
"steps": [{
"id": "tests", "type": "deterministic", "command": "npm test",
"on_non_zero": "ignore"
}]
})),
Err(SpecError::Malformed(_))
));

// The policy belongs to the deterministic rung only; a worker step that
// declares it is an unknown field, not an inert decoration.
assert!(matches!(
RunSpec::parse(&json!({
"steps": [{
"id": "ask", "type": "llm", "prompt": "p", "on_non_zero": "record"
}]
})),
Err(SpecError::UnknownField { field, .. }) if field == "on_non_zero"
));
}

/// The boundary artifact is hashed. A step that does not declare the policy
/// must serialize to the exact bytes it did before the field existed, or every
/// committed canonical fixture and every memoized step hash moves at once.
#[test]
fn the_default_policy_is_absent_from_the_serialized_boundary_spec() {
let spec = RunSpec::parse(&json!({
"steps": [{"id": "a", "type": "deterministic", "command": "true"}]
}))
.unwrap();
let serialized = serde_json::to_value(&spec).unwrap();
assert!(
serialized["steps"][0].get("on_non_zero").is_none(),
"{serialized}"
);

let recorded = RunSpec::parse(&json!({
"steps": [{
"id": "a", "type": "deterministic", "command": "true",
"on_non_zero": "record"
}]
}))
.unwrap();
assert_eq!(
serde_json::to_value(&recorded).unwrap()["steps"][0]["on_non_zero"],
json!("record")
);
}

#[test]
fn agent_cwd_is_carried_and_must_be_absolute() {
let with_cwd = RunSpec::parse(&json!({
Expand Down
Loading
Loading