Skip to content

flows check never talks to the daemon, so it structurally cannot catch spec/daemon skew #502

Description

@khaliqgant

Context

flows check is a pure compile-and-preflight: it never talks to relayflowd. That is what makes it fast and worth having, and it also means the one validation tool we ship structurally cannot catch spec/daemon skew. Any field the npm packages gain before the runtime binary does behaves this way.

Reported in the Relayflows v2 Field Report (2026-09-18, Julian Fann), building a GitLab-sourced factory flow on CLI 2.0.19. Three of their four blockers were invisible to flows check, tsc --strict and 57 unit tests; only a live run surfaced them.

Evidence

#489 is the concrete instance: @relayflows/surface 2.0.19 and @relayflows/sdk 2.0.19 both declare cwd on an agent step, @relayflows/runtime-darwin-arm64 2.0.19 does not. tsc --strict accepted it, flows check returned exit 0, and the run then died:

FAILED [protocol_error] relayflowd could not complete the run request:
invalid_spec: unknown field "cwd" at steps[0] — refusing to guess (fail closed)

This issue is the class, not that instance: today a green check tells an author nothing about whether the daemon will accept the spec, which is the difference between a tool you trust and a tool you re-verify by running.

What to change

Give check a way to reach the daemon's validator. Either is acceptable:

  • Round-trip validation. flows check --against-daemon (or on by default when a daemon socket is reachable) submits the compiled spec to relayflowd for validation only — no run created, no journal written — and reports the daemon's refusal in the same diagnostic shape as a local one.
  • Version-lockstep packaging. Publish the runtime binary in lockstep with the npm packages so the three cannot disagree at the same version number. This closes the instance but not the class, so prefer it in addition to the round-trip, not instead of.

Acceptance

  • A spec containing a field the installed daemon rejects is refused by flows check, naming the same field and step the daemon names.
  • flows check with no daemon reachable keeps working exactly as today and says which mode it ran in, so a green check is never ambiguous about what it proved.
  • A test pins the skew case: a spec valid to the packages and invalid to the daemon must fail check.

Out of scope

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgarden-readyScoped and ready for an agent to pick up

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions