Skip to content

Prevent stale process state after failures - #13

Merged
pasunboneleve merged 1 commit into
mainfrom
codex/k8q0-process-state
Aug 26, 2026
Merged

Prevent stale process state after failures#13
pasunboneleve merged 1 commit into
mainfrom
codex/k8q0-process-state

Conversation

@pasunboneleve

Copy link
Copy Markdown
Owner

Summary

  • treat output-derived state as owned by the current managed process instance
  • clear stale values before start attempts and on stop or exit
  • reject readiness from stopped processes and reap exits through the supervisor
  • keep workflow failures nonfatal while logging their complete causal chain
  • release the bug fix as 0.10.2

Validation

  • cargo fmt --check
  • cargo test (167 unit tests, 6 integration tests)
  • cargo clippy --all-targets --all-features -- -D warnings
  • ./scripts/release-notes-from-changelog.sh 0.10.2 pasunboneleve/devloop

Review

  • Roborev job 211, Grok: cleanup and state-write failure findings fixed
  • Roborev verification job 212, Grok: exit-reaping and readiness race findings fixed; job canceled after repeated draft output stalled

Required CI gate

This PR must not merge unless both Linux / rust and macOS / rust are green.

Kata: k8q0

Context:
Managed process output could leave values such as tunnel URLs in session
state after an executable disappeared. A later workflow could accept that
persisted readiness and announce a URL owned by no running process, while the
top-level log hid the dependency error's root cause.

Decision:
Treat output-derived state as a lease owned by one managed process instance.
Invalidate it before every start attempt and when the process stops or exits;
require a live child before and after readiness succeeds; reap exits observed
during readiness through the normal supervisor policy. Preserve the existing
nonfatal workflow policy, but log the complete causal chain and name degraded
mode explicitly. Release the bug fix as 0.10.2.

Alternatives considered:
A required-process flag or unconditional runtime abort would prevent degraded
operation, which is useful when unrelated workflows can still run. A config-
specific cloudflared preflight would leave the stale-state bug available to
every other managed dependency.

Tradeoffs:
Session-state persistence failures remain fatal before a new process starts,
because starting without revoking old ownership is unsafe. During teardown,
invalidation failures are logged and cleanup continues so descendants cannot
leak and the runtime does not fail solely because an exited child could not
rewrite the state file.

Architectural impact:
ProcessManager now owns the lifecycle of process-derived state alongside child
supervision. Workflow orchestration still decides whether a failed workflow is
fatal; the runtime continues in degraded mode without publishing stale
process-derived values.

Validation:
- cargo fmt --check
- cargo test
- cargo clippy --all-targets --all-features -- -D warnings
- release-notes-from-changelog.sh 0.10.2 pasunboneleve/devloop
- Roborev job 211 findings resolved; verification job 212 findings resolved
  before the stalled job was canceled

Kata: k8q0
@pasunboneleve
pasunboneleve merged commit 315828f into main Aug 26, 2026
2 of 3 checks passed
@pasunboneleve
pasunboneleve deleted the codex/k8q0-process-state branch August 26, 2026 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant