sdk: a stale lease error from the local LLM worker kills the whole run, though the kernel already retried the step

[agent-relay-code]

Update after rebase (cebdc0a): rebased onto main (2e2043f), which carries #558 / 2.0.30. onWorkerFailure now fails closed when an error arrives without its dispatch. The evidence was regenerated on the rebased head: all seven mutations reproduce, and the live-kernel suite passes 31/31 both at base 2e2043f and at this head. The sandbox full-suite and f6ece41 baseline figures below are the factory's original run, kept for the record.

Prevent stale local worker leases from aborting runs the kernel has already retried or completed.

• Classify local lease expiry with a typed error. Tag journal refusals with the worker verb, so run_terminal is dropped only for step.heartbeat, step.complete, and step.wait; lease_conflict is also dropped.
• Keep LLM, agent, communication, and HN-monitor subscribers connected after lease loss. Emit a stderr warning with run ID, step ID, attempt, and the refusal. The kernel's journal remains the authority for the outcome.
• Preserve fatal handling and original error identity for every other worker error, including malformed deadlines. Do not inspect error messages or traverse .cause.
• Add the missing authored-resume LLM listener and failure-cause handling in a separate commit.
• Give the executor five seconds after lease expiry for kernel reconciliation, matching the Rust CLI. A genuinely unresolved lease can therefore take five seconds longer to report.

Regression coverage includes expired dispatch → attempt 2 success for both worker types, actual kernel success followed by completion/heartbeat refusal, terminal refusals, direct-run/resume failure handling, the sweep grace and its bound, and non-lease fatal errors. The concurrency issue that creates stale dispatches remains outside this change.

Mutation evidence is captured in the evidence index. Seven checks each captured a failing mutation and passing byte-for-byte restoration; the ledger records the exact substitutions and restored hashes. The fatal-policy check removes the retained fatal callback as a negative control.

Literal command and captured output for the core restored regressions:

$ cd packages/sdk && npx vitest run tests/worker-lease-lost.test.ts
 Test Files  1 passed (1)
      Tests  16 passed (16)

$ cd packages/sdk && npx vitest run tests/worker-lease-lost-live.test.ts
 Test Files  1 passed (1)
      Tests  3 passed (3)

Full output: worker regressions,
real-kernel regressions.
Every other mutation command and both outputs are linked in the evidence index.

The new regression files also received a separate TypeScript check because the
existing test configuration enumerates selected files:

$ cd packages/sdk && npx tsc -p ../../evidence/worker-lease-lost/tsconfig.tests.json

Exit code: 0

The full SDK suite is not green. The final command was:

cd packages/sdk && PATH=/home/daytona/.cargo/bin:$PATH RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm test

Captured final output:

 Test Files  5 failed | 181 passed | 1 skipped (187)
      Tests  30 failed | 2866 passed | 17 skipped (2913)
     Errors  1 error
Exit code: 1

Full output includes successful
SDK typecheck, build, and the repository's test typecheck, followed by the test
failures. Remaining failures:

• Hosted isolation/protocol and native Babysitter tests: bubblewrap cannot
  configure loopback in this environment (Failed RTM_NEWADDR: Operation not permitted).
  The unhandled error is also from the hosted sandbox.
• Authored standalone runtime suite: installed Bun reports 1.3.6, while its
  prerequisite asserts 1.4.0.
• Live-kernel fixtures: eight failures, including missing real analyzer
  identification and agent-output/verification assertions.

Validation is therefore limited to the passing focused regressions and mutation
checks; this report does not claim a green full suite. No kernel or workflow
files were changed.

The unchanged live-kernel suite was also run after restoring the original SDK
source from f6ece41 and rebuilding:

$ cd packages/sdk && npx vitest run tests/live-kernel.test.ts
 Test Files  1 failed (1)
      Tests  8 failed | 23 passed (31)
Exit code: 1

Baseline output and
failed-name comparison
show the same eight failures. The implementation was restored byte-for-byte
and rebuilt successfully.
The comparison used the same Rust PATH and RELAYFLOWD_BIN as the final suite.

Checks

The checks fail on the base commit too, so these failures were not introduced by this change: they come from the repository itself or from the environment the checks ran in. This pull request is a draft until someone looks.

What ran (.relayflow/check.sh)

#!/bin/sh
# Reproduce this repository's CI checks on a fresh machine.
#
# Mirrors, in order:
#   .github/workflows/cloud-runtime-artifact.yml  the main PR gate: kernel
#                                                 build + tests, surface build,
#                                                 SDK typecheck/build/vitest,
#                                                 standalone CLI + artifact smoke
#   .github/workflows/schema-publish.yml          the `validate` job
#   .github/workflows/surface-package.yml         scripts/surface-package-gate.sh
#   .github/workflows/review-swarm-wrapper-guard.yml  its two offline parity checks
#
# Deliberately NOT run here, because each needs something this machine does not
# have:
#   * publish.yml — workflow_dispatch only; publishes to npm (NPM_TOKEN / OIDC),
#     writes git tags and builds the darwin-arm64 runtime on a macOS runner.
#   * schema-publish.yml's `npm` and `pages` jobs — NPM_TOKEN plus the
#     github-pages deployment environment.
#   * review-swarm.yml — launches a cloud review swarm with CLOUD_API_KEY and
#     RELAY_WORKSPACE_KEY. Its hermetic self-tests (swarm-gate.test.sh,
#     swarm-definition.test.sh) are run below only when `ruby` is installed;
#     they parse YAML with Psych and a GitHub runner ships Ruby, a bare machine
#     may not.
#   * review-swarm-wrapper-guard.yml's wrapper guard step — needs GH_TOKEN, the
#     `gh` CLI and a pull request number.
#   * actions/upload-artifact steps — no artifact store outside Actions.
#   * packages/ts-plugin's own `npm test` — no workflow runs it, so CI does not.
set -e

repo_root=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd -P)
cd "$repo_root"

step() { printf '\n=== %s ===\n' "$1"; }

# ---------------------------------------------------------------------------
# Toolchains. CI gets these from setup-node@v4 (node 22), setup-bun@v2 (bun
# 1.4.0) and dtolnay/rust-toolchain@stable; a fresh machine has to install them.
# ---------------------------------------------------------------------------
step 'Toolchains'
command -v node >/dev/null || { echo 'node is required (CI uses node 22)' >&2; exit 1; }
command -v npm >/dev/null || { echo 'npm is required' >&2; exit 1; }

# setup-bun@v2 pins bun-version "1.4.0", and the pin is load-bearing rather
# than cosmetic: under bun 1.3.6 a mutated `process.env.PATH` is not handed to
# `spawnSync`, so packages/schema/tests/parity.test.ts cannot shadow a real
# `claude` on PATH with its own stub and step-memory.flow.yaml fails preflight
# on the host's unauthenticated CLI. Install the pinned version whenever the
# machine's bun is anything else.
BUN_REQUIRED_VERSION=1.4.0
if [ "$(command -v bun >/dev/null 2>&1 && bun --version || echo none)" \
     != "$BUN_REQUIRED_VERSION" ]; then
  curl -fsSL https://bun.sh/install | bash -s "bun-v$BUN_REQUIRED_VERSION"
  PATH="${BUN_INSTALL:-$HOME/.bun}/bin:$PATH"
  export PATH
fi

# Plain rustup into the standard home, which is what dtolnay/rust-toolchain
# leaves behind — NOT ops/cargo.sh. That wrapper redirects RUSTUP_HOME to a
# private toolchain home for the cloud sandbox, and when a cargo shim is already
# on PATH the redirected home is empty, so rustup cannot choose a toolchain:
#   error: rustup could not choose a version of cargo to run
# cloud-runtime-artifact.yml calls plain cargo for the same reason.
if ! command -v cargo >/dev/null 2>&1; then
  if [ -x "$HOME/.cargo/bin/cargo" ]; then
    PATH="$HOME/.cargo/bin:$PATH"
  else
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
      | sh -s -- -y --default-toolchain stable --profile minimal --no-modify-path
    PATH="$HOME/.cargo/bin:$PATH"
  fi
  export PATH
fi
node --version
bun --version
cargo --version

# bubblewrap: packages/sdk/tests/hosted-extension-isolation.test.ts and
# babysitter-native-extension.test.ts launch the real /usr/bin/bwrap sandbox on
# linux. CI installs it with apt and then relaxes Ubuntu's AppArmor restriction
# on unprivileged user namespaces.
#
# This is a diagnostic here, not a hard gate: inside a container the sysctl is
# read-only, so `bwrap --unshare-all` can fail with
#   bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted
# and the sandbox cases fail for want of a kernel facility rather than for a
# defect. Warn loudly and keep going so the rest of the suite still runs.
if ! command -v bwrap >/dev/null 2>&1 \
  && command -v apt-get >/dev/null 2>&1 && command -v sudo >/dev/null 2>&1; then
  sudo apt-get update || true
  sudo apt-get install --yes --no-install-recommends bubblewrap || true
fi
if command -v sudo >/dev/null 2>&1 \
  && sysctl kernel.apparmor_restrict_unprivileged_userns >/dev/null 2>&1; then
  sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 || true
fi
if ! /usr/bin/bwrap --unshare-all --die-with-parent --new-session \
     --ro-bind / / /bin/true 2>&1; then
  echo 'WARNING: bubblewrap cannot create an unprivileged namespace here.' >&2
  echo '         The hosted-extension sandbox tests will fail for that reason.' >&2
fi

# ---------------------------------------------------------------------------
# cloud-runtime-artifact.yml
# ---------------------------------------------------------------------------
# testdata/preflight's agent-CLI fixtures are extensionless files with an ESM
# `import`. Node only runs them because module-syntax detection treats an
# extensionless file with no governing package.json as ESM. A GitHub runner
# checks the repo out under /home/runner/work with no package.json above it, so
# detection applies. This machine has /home/daytona/package.json declaring
#   "type": "commonjs"
# ABOVE the checkout, which turns detection off: node parses each fixture as
# CommonJS, the module body never runs, and the wrapper exits 0 having written
# nothing. Every agent step driven by one then fails its execution gate, which
# is the whole live-kernel agent surface (7 tests).
#
# Restore the CI condition for those fixtures only, and take the file back out
# on the way so the working tree is left as it was found. It is deliberately NOT
# committed: in CI it would be a no-op.
preflight_pkg="$repo_root/testdata/preflight/package.json"
if [ ! -e "$preflight_pkg" ]; then
  printf '{"type":"module"}\n' > "$preflight_pkg"
  trap 'rm -f "$preflight_pkg"' EXIT INT TERM
fi

step 'Artifact contract tests'
node --test scripts/cloud-artifact.test.mjs

step 'Kernel build and tests'
( cd kernel && cargo build --locked --release -p relayflowd )
( cd kernel && cargo test --workspace )

step 'Authoring surface build'
( cd packages/surface && bun install --frozen-lockfile --ignore-scripts && bun run build )

# --ignore-scripts because the surface is already built; without it npm runs the
# file: dependency's prepare before its own devDependencies exist. The SDK build
# below is explicit, so nothing is skipped.
step 'SDK dependencies'
npm ci --prefix packages/sdk --ignore-scripts

# `npm ci` installs @relayflows/surface from the REGISTRY, not the copy built
# above, so any SDK import of a symbol that has not been published yet fails
# typecheck. Override it with the local directory; --no-save keeps package.json
# and package-lock.json untouched. The `./` prefix is load-bearing — without it
# npm reads the string as a GitHub org/repo shorthand.
step 'Local surface override'
npm install ./packages/surface --prefix packages/sdk --no-save --ignore-scripts

# workflows/*.flow.ts and `flows check` resolve @relayflows/surface from the repo
# root, where nothing is installed. Link the same local surface there.
step 'Link the local surface at the repo root'
mkdir -p node_modules/@relayflows
ln -sfn ../../packages/sdk/node_modules/@relayflows/surface node_modules/@relayflows/surface
node -e "console.log(require.resolve('@relayflows/surface'))"

# ---------------------------------------------------------------------------
# schema-publish.yml (validate job). Needs the SDK's node_modules above: the
# schema tests require ajv, yaml and typescript through packages/sdk.
# ---------------------------------------------------------------------------
step 'Schema is regenerable, committed and deterministic'
node scripts/generate-json-schema.mjs
git diff --exit-code -- packages/schema/flows.schema.json
cp packages/schema/flows.schema.json /tmp/flows.schema.first.json
node scripts/generate-json-schema.mjs
diff -q /tmp/flows.schema.first.json packages/schema/flows.schema.json

step 'Schema parity and smoke'
( cd packages/schema && bun run test )

# ---------------------------------------------------------------------------
# cloud-runtime-artifact.yml, continued: the SDK suite.
#
# This is `npm test` expanded minus test:prep, which shells out to ops/cargo.sh
# (see the toolchain note above). test:prep's only other job is re-asserting the
# executable bit on the preflight CLI fixtures, kept below, and the relayflowd
# binary it would build is the release binary already built above — pointed at
# through RELAYFLOWD_BIN rather than compiled a second time.
#
# The whole suite, not a list of files: naming files means a test added anywhere
# else never runs.
# ---------------------------------------------------------------------------
step 'SDK typecheck, build and tests'
( [ ! -d testdata/preflight ] \
  || find testdata/preflight -name '*-cli' -type f -exec chmod +x {} + )
cd packages/sdk
npm run typecheck
npm run build
npm run typecheck:tests
# RELAYFLOWS_ALLOW_ANALYZER_SKIP: tests/live-kernel.test.ts runs one case against
# the real Claude analyzer and fails by default when it cannot reach a model.
# Setting it is the same explicit statement CI makes: this run is not gate-2
# acceptance evidence. Everything else in the suite still gates.
RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 \
  RELAYFLOWD_BIN="$repo_root/kernel/target/release/relayflowd" \
  ./node_modules/.bin/vitest run
cd "$repo_root"

step 'Standalone flows CLI, artifact assembly and smoke'
mkdir -p dist/cloud-artifact-input
node scripts/build-standalone-cli.mjs bun-linux-x64 dist/cloud-artifact-input/flows
node scripts/cloud-artifact.mjs build \
  --relayflowd kernel/target/release/relayflowd \
  --flows-executable dist/cloud-artifact-input/flows \
  --output-dir dist/cloud-artifact \
  --source-commit "$(git rev-parse HEAD)"
archive="$(find dist/cloud-artifact -name '*.tar.gz' -type f -print -quit)"
node scripts/cloud-artifact.mjs verify \
  --archive "$archive" \
  --sha256 "$(awk '{print $1}' "$archive.sha256")"
mkdir -p dist/cloud-artifact-smoke
tar -xzf "$archive" -C dist/cloud-artifact-smoke
dist/cloud-artifact-smoke/bin/relayflowd --help
flows_output="$(dist/cloud-artifact-smoke/bin/flows check --json testdata/hello-deterministic.flow.yaml)"
printf '%s\n' "$flows_output"
node -e '
  const report = JSON.parse(process.argv[1]);
  if (report.ok !== true) throw new Error("flows smoke report was not ok");
  if (report.path !== "testdata/hello-deterministic.flow.yaml") {
    throw new Error(`flows smoke reported unexpected path: ${report.path}`);
  }
' "$flows_output"

# ---------------------------------------------------------------------------
# review-swarm-wrapper-guard.yml, the two steps that need no token. Both are
# offline drift checks between ops/preswarm-check and workflows/review-swarm.yaml.
# PRESWARM_ALLOW_MISSING_CLI=1 exactly as the workflow sets it: the guard runs
# where claude/codex/opencode are not on PATH.
# ---------------------------------------------------------------------------
step 'Pre-swarm lens parity'
sh ops/preswarm-check/lens-parity-check.sh
PRESWARM_ALLOW_MISSING_CLI=1 sh ops/preswarm-check/lens-cli-parity-check.sh

# ---------------------------------------------------------------------------
# review-swarm.yml's hermetic gate self-tests. They stub agent-relay and gh and
# run offline, but they need Ruby's Psych YAML parser. A GitHub runner has Ruby;
# skip rather than install one here.
# ---------------------------------------------------------------------------
step 'Review gate self-tests'
if command -v ruby >/dev/null 2>&1; then
  bash .github/workflows/scripts/swarm-gate.test.sh
  bash .github/workflows/scripts/swarm-definition.test.sh
else
  echo 'SKIPPED: ruby is not installed, and these tests parse YAML with Psych.'
fi

# ---------------------------------------------------------------------------
# surface-package.yml. Last, because the gate runs its own `npm ci` in
# packages/sdk and swaps @relayflows/surface for a freshly packed tarball; doing
# that before the SDK suite would undo the directory override installed above.
# ---------------------------------------------------------------------------
step 'Surface package gate'
bash scripts/surface-package-gate.sh

printf '\n=== all checks passed ===\n'

Output on this branch (last 80 lines)

 ❯ refuse src/hosted-extension-protocol.ts:135:21
 ❯ ChildProcess.<anonymous> src/hosted-extension-protocol.ts:234:21

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { code: 'plugin_unsupported' }
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[12/22]⎯

 FAIL  tests/hosted-extension-protocol.test.ts > hosted extension hostile protocol > rejects an import-time different PR frame with zero adapter calls
 FAIL  tests/hosted-extension-protocol.test.ts > hosted extension hostile protocol > rejects an import-time different delivery frame with zero adapter calls
 FAIL  tests/hosted-extension-protocol.test.ts > hosted extension hostile protocol > rejects an import-time different event frame with zero adapter calls
AssertionError: expected Error: Hosted extension sandbox exited wi… { code: '…' } to match object { code: 'plugin_event_unroutable' }

- Expected
+ Received

- Object {
-   "code": "plugin_event_unroutable",
+ PluginError {
+   "code": "plugin_unsupported",
  }

 ❯ tests/hosted-extension-protocol.test.ts:430:5
    428|   ])('rejects an import-time %s frame with zero adapter calls', async …
    429|     let calls = 0;
    430|     await expect(runVerifiedNativeExtensionSandbox({
       |     ^
    431|       artifact: await artifact(hostileImport([frame, { type: 'error', …
    432|       manifest: validateFlowExtensionManifest(manifest()), dispatch: d…

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[13/22]⎯

 FAIL  tests/hosted-extension-protocol.test.ts > hosted extension hostile protocol > rejects two forged calls after the authoritative first outcome settles
 FAIL  tests/hosted-extension-protocol.test.ts > hosted extension hostile protocol > waits for a pending adapter to reject after a forged child error
 FAIL  tests/hosted-extension-protocol.test.ts > hosted extension hostile protocol > waits for a pending adapter to resolve after a forged child error
 FAIL  tests/hosted-extension-protocol.test.ts > hosted extension hostile protocol > returns a typed adapter rejection even when the hostile child hangs
Error: hostile child did not invoke the adapter
 ❯ Timeout._onTimeout tests/hosted-extension-protocol.test.ts:118:45
    116| async function waitForInvocation(invoked: Promise<void>): Promise<void…
    117|   await new Promise<void>((resolve, reject) => {
    118|     const timeout = setTimeout(() => reject(new Error('hostile child d…
       |                                             ^
    119|     void invoked.then(() => { clearTimeout(timeout); resolve(); }, rej…
    120|   });

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[14/22]⎯

⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯

Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.

⎯⎯⎯⎯ Unhandled Rejection ⎯⎯⎯⎯⎯
Error: Hosted extension sandbox exited without a valid completion (exit 1): bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted

 ❯ refuse src/hosted-extension-protocol.ts:135:21
    133|       : new PluginError('plugin_unsupported', 'Hosted capability rejec…
    134|     const refuse = (message: string) => {
    135|       const error = new PluginError('plugin_unsupported', message);
       |                     ^
    136|       CHILD_PROCESS_KILL(child, 'SIGKILL');
    137|       if (capabilityState === 'pending') {
 ❯ ChildProcess.<anonymous> src/hosted-extension-protocol.ts:234:21
 ❯ ChildProcess.emit node:events:520:22
 ❯ maybeClose node:internal/child_process:1084:16
 ❯ Process.ChildProcess._handle.onexit node:internal/child_process:304:5

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { code: 'plugin_unsupported' }
This error originated in "tests/hosted-extension-protocol.test.ts" test file. It doesn't mean the error was thrown inside the file itself, but while it was running.
The latest test that might've caused the error is "rejects two forged calls after the authoritative first outcome settles". It might mean one of the following:
- The error was thrown, while Vitest was running this test.
- If the error occurred after the test had been completed, this was the last documented test before it was thrown.
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

 Test Files  3 failed | 183 passed | 1 skipped (187)
      Tests  22 failed | 2887 passed | 4 skipped (2913)
     Errors  1 error
   Start at  06:23:01
   Duration  265.24s (transform 2.93s, setup 0ms, collect 49.76s, tests 700.21s, environment 24ms, prepare 7.60s)

Output on the base commit (last 80 lines)

=== Toolchains ===

######################################################################## 100.0%
bun was installed successfully to ~/.bun/bin/bun 
Run 'bun --help' to get started
v25.6.0
1.4.0
cargo 1.98.1 (797e8a9bc 2026-08-05)
sysctl: permission denied on key "kernel.apparmor_restrict_unprivileged_userns"
bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted
WARNING: bubblewrap cannot create an unprivileged namespace here.
         The hosted-extension sandbox tests will fail for that reason.
/tmp/relayflow-recipe.UP6Tr7: 121: cannot create //testdata/preflight/package.json: Directory nonexistent

What the repair agent found

Check repair notes

.relayflow/check.sh was failing on this branch. Three distinct causes, none of
them a defect in the branch's change. Two were missing setup and are fixed in
.relayflow/check.sh (not committed). One is a kernel facility this container
does not have and is left alone.

Fixed: bun was not pinned to the version CI uses (setup)

.github/workflows/cloud-runtime-artifact.yml pins setup-bun@v2 to
bun-version: "1.4.0". check.sh only installed bun when none was present, so
this machine's preinstalled bun 1.3.6 was used instead.

The pin is load-bearing, not cosmetic. Under bun 1.3.6 a mutated
process.env.PATH is not handed to spawnSync, so
packages/schema/tests/parity.test.ts could not shadow the machine's real
claude with the stub it writes, and step-memory.flow.yaml failed preflight
on the host CLI's unauthenticated auth status:

(fail) flows check fixture parity: step-memory.flow.yaml [979.00ms]

That aborted check.sh at the schema step, so the SDK suite never ran at all.
packages/sdk/tests/authored-node-runtime.test.ts:18 asserts the same version
directly and was failing for the same reason.

check.sh now installs bun-v1.4.0 whenever the machine's bun is anything
else. With it, schema parity is 77 pass, 0 fail.

Fixed: a package.json above the checkout disabled Node ESM detection (setup)

testdata/preflight's agent-CLI fixtures are extensionless files containing an
ESM import. Node runs them only because module-syntax detection applies when
no package.json governs the file. A GitHub runner checks out under
/home/runner/work with no package.json above it.

This machine has /home/daytona/package.json declaring "type": "commonjs"
ABOVE the checkout, which turns detection off. Each fixture is then parsed as
CommonJS, its module body never runs, and it exits 0 having written nothing:

$ printf '{"protocol":"relayflows-agent-cli-v1","instruction":"hi"}' \
    | ./testdata/preflight/analyze-story-stub-cli --relayflows-adapter-v1 | od -c
0000000

Every agent step driven by one then fails its execution gate with
output: null — the seven tests/live-kernel.test.ts failures. They are not
new: evidence/worker-lease-lost/baseline-comparison.txt records the same set
failing on the original SDK source at f6ece41.

check.sh now writes {"type":"module"} into testdata/preflight/package.json
before the run and removes it on exit, restoring the CI condition without
leaving anything in the tree. With it, tests/live-kernel.test.ts is
30 passed | 1 skipped. Full reproduction and captured output:
evidence/worker-lease-lost/live-kernel-module-type.txt.

A durable alternative, for a human to decide: commit
testdata/preflight/package.json as {"type":"module"}. It is a no-op in CI
and makes the fixtures' module kind explicit rather than inferred. Not done
here — it is outside this branch's scope.

Not fixed: bubblewrap cannot create a user namespace here (outside my control)

bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted

check.sh already probes for this and warns. The sysctl that would relax it,
kernel.apparmor_restrict_unprivileged_userns, is read-only in this container:

sysctl: permission denied on key "kernel.apparmor_restrict_unprivileged_userns"

22 tests in three files need the real sandbox and fail for want of that kernel
facility, not for a defect:

• tests/hosted-extension-isolation.test.ts — 13 failed
• tests/hosted-extension-protocol.test.ts — 8 failed
• tests/babysitter-native-extension.test.ts — 1 failed

Where the run ends up

With both setup fixes, the last full sh .relayflow/check.sh:

Test Files  3 failed | 183 passed | 1 skipped (187)
     Tests  22 failed | 2887 passed | 4 skipped (2913)

The 22 are exactly the bubblewrap set above. Every other step passes: kernel
build and tests, surface build, schema regeneration and parity, SDK typecheck,
build, test typecheck and vitest. The branch's own regression files all pass:

 ✓ tests/worker-lease-lost.test.ts (16 tests) 16ms
 ✓ tests/worker-lease-lost-live.test.ts (3 tests) 763ms
 ✓ tests/worker-lease-sweep.test.ts (2 tests) 6ms
 ✓ tests/resume-worker-lease.test.ts (2 tests) 4ms
 ✓ tests/direct-run-worker-lease.test.ts (2 tests) 6ms

check.sh stops at the first failing step, so the steps after the SDK suite
were never reached in that run. They were run by hand afterwards, with the same
commands and the same PATH, and all pass:

• standalone CLI build, artifact assembly, verify and smoke — flows check
  reported "ok":true for testdata/hello-deterministic.flow.yaml
• ops/preswarm-check/lens-parity-check.sh — PASS
• ops/preswarm-check/lens-cli-parity-check.sh — PASS
• scripts/surface-package-gate.sh — exit 0
  (PACKED_RUNTIME_OK, PACKED_RUNTIME_REFUSAL_OK, PACKED_TYPESCRIPT_OK)

The review gate self-tests skip here exactly as check.sh says they will:
ruby is not installed, and they parse YAML with Psych.

Fixes #560

---

Summary by cubic

Prevents a stale local worker lease error from aborting a run the kernel has already retried or completed.

Bug Fixes

• Classifies lease loss with WorkerLeaseLostError and drops journal refusals only for lease_conflict or run_terminal on step.heartbeat, step.complete, and step.wait.
• Keeps LLM, agent, communication, and HN-monitor workers connected after lease loss, warning to stderr instead of closing the client; requires the attempt's dispatch, and fails closed if one is missing.
• Waits up to five seconds past lease expiry for kernel reconciliation before reporting a failed step, matching the Rust CLI.
• Adds the missing LLM failure handling for authored resume and gates the new error cases in tsconfig.tests.json.

The mutation and baseline evidence under evidence/worker-lease-lost/ was regenerated on the rebased head: all seven mutations reproduce a failing test, baseline.py compares against the rebase's parent instead of a hard-coded commit, and the live-kernel suite passes at both base.

^{Written for commit cebdc0a. Summary will update on new commits.}

---

Note

Medium Risk
Changes how local workers react to lease-related journal errors on critical run/resume paths; incorrect classification could either abort good runs or mask real failures, though mutation tests and targeted regressions narrow that surface.

Overview
Fixes a bug where stale local worker lease errors (expired dispatches, post-retry lease_conflict / run_terminal refusals) closed the journal client and killed the whole run, even when the kernel had already retried or completed the step.

The SDK now classifies lease loss via WorkerLeaseLostError and journal refusals tagged with lease_conflict or run_terminal on step.heartbeat, step.complete, and step.wait. onWorkerFailure keeps workers attached for those cases: it emits a stderr warning with run/step/attempt context instead of calling fatal close. Other worker errors (including bad lease deadlines) still fail closed with the original error identity—no message or .cause heuristics.

CLI paths wire the same filter for direct run, authored resume LLM, HN monitor, and communication workers. flows run also adds a 5s post-expiry grace (LEASE_SWEEP_GRACE_MS) while polling, aligned with the Rust CLI, so the executor can wait for kernel reconciliation.

Regression coverage adds focused vitest suites (unit, live-kernel injection, sweep, direct/resume failure reporting) plus an evidence/worker-lease-lost/ bundle: mutation scripts, baseline comparison against f6ece41, and captured test transcripts documenting passing regressions and environment-specific full-suite failures (bubblewrap, Bun pin, preflight ESM).

^{Reviewed by Cursor Bugbot for commit 2dfb951. Bugbot is set up for automated code reviews on this repo. Configure here.}

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 5622684b-5cb3-4963-bf14-be6429c098c9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[khaliqgant]

Rebased onto main, which carries #558 and 2.0.30; the rebase was clean. Reviewed the source changes, and added one commit so onWorkerFailure fails closed if a worker error ever arrives without its dispatch.

Focused regressions run locally on the rebased head:

$ cd packages/sdk && npx vitest run tests/worker-lease-lost.test.ts tests/worker-lease-lost-live.test.ts tests/direct-run-worker-lease.test.ts tests/resume-worker-lease.test.ts tests/worker-lease-sweep.test.ts tests/worker-transcript.test.ts tests/authored-agent-artifacts.test.ts tests/cli.test.ts
 Test Files  8 passed (8)
      Tests  108 passed (108)

The factory's full-suite failures are the sandbox's (bubblewrap loopback, Bun version); it showed the same failures on the base commit. Marking ready so the review bots run. @codex review

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-23T07:23:06.569182Z	cebdc0a	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2dfb951950

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Update the mutation script for the final predicate

Running the documented python3 evidence/worker-lease-lost/mutate.py command at this commit fails immediately because worker-lease.ts now contains if (!isLeaseLost(error) || dispatch === undefined), so this exact needle is absent. The script also truncates mutations.txt before reaching the assertion, and the recorded restored hashes no longer match the final source. Update the mutations and recapture both mutant and restored outputs against the final head before retaining the mutation-verification claim.

AGENTS.md reference: AGENTS.md:L93-L96

Useful? React with 👍 / 👎.

[khaliqgant]

Fixed in cebdc0a. mutate.py now matches the fail-closed predicate (if (!isLeaseLost(error) || dispatch === undefined)) and no longer hard-codes the sandbox PATH. I reran it on the rebased head: all seven mutations reproduce, with mutant exit 1 and restored exit 0 each (mutations.txt, with the restored SHA-256 of each file).

[chatgpt-codex-connector]

Compare failures against the actual rebased parent

On this rebased change the parent is 2e2043f, but the baseline restores source from f6ece41; those revisions differ in two files this PR modifies (cli/run.ts and llm-worker.ts, by 58 lines), and the captured transcript identifies SDK 2.0.29 rather than the rebased 2.0.30. Consequently this comparison cannot establish that the full-suite failures also occur on the reviewed base; rerun it against the actual parent or narrow the claim.

AGENTS.md reference: AGENTS.md:L83-L86

Useful? React with 👍 / 👎.

[khaliqgant]

Fixed in cebdc0a. baseline.py now uses the rebased parent, git merge-base HEAD origin/main (BASE=<rev> overrides), and records it in each transcript. It takes RELAYFLOWD_BIN from the caller. I reran it: base 2e2043f gives Tests 31 passed (31), and the live-kernel suite at this head also passes 31/31 (live-kernel-head.txt). The old sandbox comparison is removed. The README explains that the sandbox's eight failures came from a stray /home/daytona/package.json, not from either revision.

[khaliqgant]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: cebdc0adb6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".