Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph

⚠️ **A quiet pane is not always a pane that wants you.** A CLI can declare an optional `capabilities.workDetect.watchingLine` (a monitor, background shell or cloud hand-off it is still running); the idle probe reads it into `Session.watching` and `notePrompt()` opens that idle item ALREADY acknowledged, so no surface alerts. Only `idle` is eligible, and the label is pane-derived and prompt-injectable, so a pattern must anchor on chrome only that CLI draws. → [architecture-invariants#the-watching-signal-a-quiet-pane-that-is-not-waiting-for-you](docs/architecture-invariants.md#the-watching-signal-a-quiet-pane-that-is-not-waiting-for-you). Tests: `test/session-watching.test.ts`, `test/watching-no-alert.test.ts`.

**An exited agent in a live pane** (`paneExit`, #446): panes use `remain-on-exit on`, so `/exit` leaves a pane, session and pid that look alive; `TmuxManager.startPaneExitWatcher()` publishes `SessionState.paneExit` via `session:updated`. ⚠️ Never set `status: 'error'` or null the `pid` for it; the field is TRI-STATE (absent = UNKNOWN, never alive, scoped by `Session.paneExitApplies`); an absent `#{pane_dead_status}` is not 0; a path that starts a command in a pane must clear the record AND persist. → [architecture-invariants#an-exited-agent-in-a-live-pane-paneexit](docs/architecture-invariants.md#an-exited-agent-in-a-live-pane-paneexit)
**An exited agent in a live pane** (`paneExit`, #446): panes use `remain-on-exit on`, so `/exit` leaves a pane, session and pid that look alive; `TmuxManager.startPaneExitWatcher()` publishes `SessionState.paneExit` via `session:updated`. ⚠️ Never set `status: 'error'` or null the `pid` for it; the field is TRI-STATE (absent = UNKNOWN, never alive, scoped by `Session.paneExitApplies`); an absent `#{pane_dead_status}` is not 0; a path that starts a command in a pane must clear the record AND persist. A clean exit is CLOSED via `cleanupSession()` (`pane-exit-sweep.ts`): only an explicit numeric status 0 with no signal, confirmed by 2 reads, with no start/attach in flight (`paneLifecycleInFlight`) and not within 10 s of one (a startup error keeps its row); a crashed agent keeps its row. → [architecture-invariants#an-exited-agent-in-a-live-pane-paneexit](docs/architecture-invariants.md#an-exited-agent-in-a-live-pane-paneexit)

**Dead-pane respawn resume pin** (`_buildRespawnPaneOptionsWithResumePin()`, session.ts): recovering a dead pane, like a custom-model `restartCli()`, must pin the conversation or claude refuses the reused `--session-id`. The pin takes the first transcript-backed candidate (chain tail, launch seed, own id), never `_claudeSessionId`, adds nothing when none is backed, and is never applied to remote or docker sessions. → [architecture-invariants#dead-pane-respawn-the-resume-pin](docs/architecture-invariants.md#dead-pane-respawn-the-resume-pin)

Expand Down
9 changes: 9 additions & 0 deletions docs/architecture-invariants.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,15 @@ Further detail, closing: ⚠️ **Closing has the mirror-image race and one owne

**Codeman creates every pane with `remain-on-exit on`, so a session whose agent exited still looks alive.** `/exit` ends the CLI, tmux keeps the pane and the tmux session, and the `tmux attach-session` process Codeman records as `Session.pid` runs on, so no PTY exit handler fires and the record keeps its pid and `status: 'idle'` (Ark0N/Codeman#446). `SessionState.paneExit` (`{status?, signal?, at}`) is the fact tmux already knows, published through `toState()` so it rides `session:updated` and lands in `state.json` on the same persist — there is no SSE event for it. One batched `tmux list-panes -a` per tick fills it, from `TmuxManager.startPaneExitWatcher()`, which has its OWN always-on interval: the stats collector cannot carry it, because the browser arms and disarms that one with the Monitor panel (`panels-ui.js`) and boot skips it entirely when no session was recovered. ⚠️ **The field is TRI-STATE and its third state is absence**, meaning UNKNOWN, which renders as nothing and must NEVER read as alive; it covers a running pane, a session the read did not list, a failed probe, and every session shape a dead local pane does not describe. `Session.paneExitApplies` is the single place that scoping lives, and it fails closed for four shapes: a direct-PTY session (no pane), a remote SSH session (the local pane is the ssh client, whose death is a transport drop OR an exit — the whole of #355), a docker case (the local pane is a `docker exec` into the container's own tmux), and a session rebuilt from the socket (`MuxSession.discovered`: its synthetic `restored-<fragment>` id matches no `state.json` entry, so a remote session rediscovered after `mux-sessions.json` was lost would arrive looking local). ⚠️ **Never set `status: 'error'`** for an exited pane — that value is the PTY-exit breaker's and the browser answers it with a "restart it?" confirm — and **never null the `pid`**, which is what makes `selectSession()` re-attach and launch a fresh CLI. Local panes keep `remain-on-exit on`; flipping them to `failed` ends the tmux session, nulls the pid and reintroduces the auto-revive #355 removed. ⚠️ **An absent `#{pane_dead_status}` is not 0**: measured on tmux 3.2a a SIGKILLed pane reports neither a status nor a signal (`#{pane_dead_signal}` did not exist before tmux 3.4), so folding it into 0 would turn an unexplained death into a clean exit. A session answers only when the read listed EXACTLY ONE pane for it, since Codeman never splits a pane and a session the user split by hand has none that speaks for the agent. The three synchronous `isPaneDead()` callers (the `/wait` route, the TUI, the attach path) keep their own probes — this watcher is never fresh enough for them. ⚠️ **A path that starts a command in a pane must clear the record AND persist**, since the watcher's next tick sees the field already cleared and writes nothing. ⚠️ **The always-on timer gates the READ, never the tick.** `hasObservablePaneSession()` (`tmux-manager.ts`) skips the tmux exec while every session on the manager is one of the shapes `paneExitApplies` forces to UNKNOWN, so an instance running only remote or Docker work keeps ticking and costs nothing; the two predicates are two copies of one rule, and `test/session-pane-exit.test.ts` pins them against each other for the four session shapes that exist today — a FIFTH condition added to one and not the other still fails nothing, so change them together. Skipping retracts nothing, for the same reason a failed read does not. ⚠️ **The muted status dot is a specificity fight, and it is fought on three surfaces.** The tab renders `status` as before, and `tab-agent-exited` only quiets the dot, so the rule excludes three states BY HAND: `.tab-alert-action` and `.tab-alert-idle` on the tab, and `.tab-status.error` on the dot itself. Each of those colours means "this needs you" — the two alerts because a human is blocked, `error` because the browser answers it with a "restart it?" confirm — and each must survive the exit. The rich tab rail needs a SECOND copy of the rule, because its own `tab-state-*` dot rules are (0,9,1) against the strip's (0,5,0) — measured, an exited session on a detailed rail kept a full green dot and the working halo beside a badge reading "exited". Its twin matches that specificity exactly and therefore must stay BELOW those rules in source order. mobile.css needs a THIRD copy, with `!important`, because the phone block enlarges a `busy` dot and gives it a green glow that way, and `status` stays `busy` for a pane whose agent died mid-turn — without it a phone renders a grey dot still wearing the green halo. `test/session-pane-exit-ui.test.ts` resolves the real stylesheets in jsdom rather than matching selector text — styles.css for the desktop cases and both files for the phone ones — so the ordering, the hand-written exclusions and a missing phone rule all fail there. Tests: `test/session-pane-exit.test.ts`, `test/tmux-manager.test.ts`, `test/session-pane-exit-ui.test.ts`.

**A session whose agent exited cleanly is closed, and a crashed one is kept** (Ark0N/Codeman#446, part 2). After every pane read, `closeCleanlyExitedSessions()` (`server.ts`) closes each session that `shouldCloseCleanlyExitedSession()` (`pane-exit-sweep.ts`, pure) accepts, through `cleanupSession(id, true, CLEAN_EXIT_CLOSE_REASON)`. That is the X button's path, so an unpinned session is removed, a pinned one is demoted to `status: 'stopped'`, the lifecycle log records why, and the conversation stays resumable from the Resume list, which reads the lifecycle log and the transcripts rather than the pane. The rule has four parts, and each guards against a wrong close:

- ⚠️ **The status must be an explicit numeric 0 with no signal** (`isCleanPaneExit()`). An absent status is how a SIGKILL presents on tmux 3.2a, so `status ?? 0` would close an agent the OOM killer took. A non-zero status or any signal keeps the row, marked with the exit, as the crash evidence #210 was filed to keep.
- **At least `CLEAN_EXIT_CONFIRMING_READS` (2) authoritative reads must agree.** `TmuxManager.getPaneExitReadCount()` counts them. A repeat of the same pane pid, status and signal adds one, anything else starts again at 1, and a failed, empty or skipped read never reaches `applyPaneExits()`, so it neither confirms nor resets. A mux without the method never has a session closed.
- **No start, attach or relaunch may be in flight** (`Session.paneLifecycleInFlight`, raised for the whole of `_setupOrAttachMuxSession()` and `restartCli()`). The dead-pane respawn revives an exited pane on purpose, and the pane reads as dead until `clearPaneExitForNewPane()` runs after its startup delay.
- **The pane must have been up for `CLEAN_EXIT_MIN_PANE_LIFETIME_MS` (10 s)** since the last start, attach or relaunch finished (`Session.paneStartedAt`). A CLI that prints a startup error and exits 0 would otherwise lose its tab, and the error with it, seconds after launch; its row stays as `exited (0)` instead. An attach to an already running pane stamps it too, so an `/exit` within seconds of a server restart leaves a row to close by hand.

Scoping needs no check of its own here: `setPaneExit()` already forces `paneExit` to UNKNOWN for direct-PTY, remote, docker and discovered sessions. There is no setting, by the maintainer's decision on #446. ⚠️ Do not flip local panes to `remain-on-exit failed` to get the same effect: a destroyed pane ends the tmux session, the PTY exit nulls the pid, and the browser's `selectSession()` then launches a fresh CLI. `cleanupSession()` keeps `{workingDir}/.claude-images` while another session still uses that directory (`pasteImageDirInUseByOtherSession()`, `paste-image-gc.ts`), since the sweep would otherwise routinely delete a live sibling's pasted images. Paths are compared by `realpath`, and a detached session counts through its persisted record, because `killMux=false` removes it from the map while its pane keeps running; only a session being KILLED is exempt. Each exit gets ONE close attempt (keyed by session id and `at`), and a session being closed refuses `startInteractive()`/`startShell()` (`Session.markClosing()`), so a start that races the close cannot orphan a tmux session. `planRebootRestore()` refuses a record whose persisted `paneExit` is clean (`agent-exited`), which covers an agent that exited just before the power went, before the sweep reached it. Tests: `test/pane-exit-sweep.test.ts`, `test/paste-image-dir-shared.test.ts`, `test/reboot-restore.test.ts`, `test/tmux-manager.test.ts`.

### Dead-pane respawn: the resume pin

**The dead-pane respawn needs the same resume pin as a custom-model `restartCli()` and shares it** (`_buildRespawnPaneOptionsWithResumePin()` in `session.ts`, used by `restartCli()`, the dead-pane respawn in `_setupOrAttachMuxSession()`, and its create path when that path RELAUNCHES a CLI: after a failed respawn, or when tmux lost the whole session rather than the pane): a pane whose agent EXITED owns a transcript too, so recovering one with the bare launch line hit the same refusal and the conversation was stranded behind a tab that looked merely idle. The pin walks three candidates in order — the conversation chain's tail, the launch seed, then the session's own id — and takes the first one a transcript backs, never `_claudeSessionId` (which also holds history-correlated GUESSES keyed on the working directory, and launching from one would open and write to a conversation that was never this pane's). ⚠️ The create-path pin is written to `_resumeSessionId` as well, so unlike `restartCli()`'s one-respawn pin it PERSISTS through `toState()` as `resumeSessionId`: that field means "what the user asked to resume at creation, or what recovery pinned", and after a dead-pane respawn `_claudeSessionId` names whatever the walk actually pinned rather than the chain tail. ⚠️ A candidate no transcript backs is passed over, and falling off the end of the walk ADDS no pin (the options keep whatever launch seed they already carried): a divergent pin leaves `--session-id <this.id>` in the fallback branch, where a failed resume collides all over again, while pinning an id with no transcript prints claude's "No conversation found" into a brand-new session's scrollback and costs the running branch its `nice` priority (`wrapWithNice()` prefixes only the first branch of an `a || b`). ⚠️ Remote and docker sessions are never pinned: their pane commands are already self-healing, the conversation lives on the far side, and a local id resolves to nothing there.
Expand Down
8 changes: 8 additions & 0 deletions src/mux-interface.ts
Original file line number Diff line number Diff line change
Expand Up @@ -336,6 +336,14 @@ export interface TerminalMultiplexer extends EventEmitter {
*/
getPaneExit?(muxName: string): PaneExit | undefined;

/**
* How many authoritative pane reads have agreed on the exit `getPaneExit()`
* reports, or 0 when it reports none. The exited-agent sweep closes a session
* only once this reaches `CLEAN_EXIT_CONFIRMING_READS` (`pane-exit-sweep.ts`),
* and a multiplexer without this method never has a session closed by it.
*/
getPaneExitReadCount?(muxName: string): number;

/** Forget a session's exit observation, e.g. once its pane has been respawned. */
clearPaneExit?(muxName: string): void;

Expand Down
99 changes: 99 additions & 0 deletions src/pane-exit-sweep.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
/**
* @fileoverview The exited-agent sweep's decision rule (Ark0N/Codeman#446).
*
* Codeman creates every tmux pane with `remain-on-exit on`, so `/exit` ends the
* CLI while the pane, the tmux session and the `tmux attach-session` process
* all live on. Part 1 of #446 records that as `SessionState.paneExit`. This
* module decides when such a session is closed, the way the X button closes
* it, so finished sessions stop piling up on the board.
*
* The rule closes a session only on a POSITIVE observation of a clean exit:
*
* - The exit status must be an explicit numeric 0 with no signal. An absent
* status is UNKNOWN, never 0: on tmux 3.2a a SIGKILLed pane reports neither a
* status nor a signal, so reading absence as clean would sweep an agent the
* OOM killer took. A non-zero status or any signal keeps the row, marked with
* the exit, as the crash evidence #210 was filed to keep.
* - At least {@link CLEAN_EXIT_CONFIRMING_READS} authoritative pane reads must
* have agreed on that exit. A failed, empty or skipped read counts for
* nothing, because unknown never closes anything.
* - No start, attach or relaunch may be in flight for the session. The
* dead-pane branch of `Session._setupOrAttachMuxSession()` respawns an exited
* pane on purpose, and for a few seconds that pane still reads as dead.
* - The exit must land at least {@link CLEAN_EXIT_MIN_PANE_LIFETIME_MS} after
* the last start, attach or relaunch finished. A CLI that prints a startup
* error ("not logged in", a bad profile, a config error) and exits 0 would
* otherwise lose its tab, and the error with it, seconds after launch. Its
* row stays, marked `exited (0)`, for the user to read and close.
*
* Scoping to local mux-backed sessions happens before this rule runs:
* `Session.setPaneExit()` forces the field to UNKNOWN for direct-PTY, remote,
* docker and discovered sessions, so their `paneExit` never reaches here.
*
* Pure, so the rule is unit-tested without a server (test/pane-exit-sweep.test.ts).
*/
import type { PaneExit } from './types/index.js';

/**
* How many authoritative pane reads must agree on a clean exit before the
* session is closed. At the watcher's 2 s cadence two reads mean a finished
* session disappears within about four seconds of its agent exiting.
*/
export const CLEAN_EXIT_CONFIRMING_READS = 2;

/**
* How long a pane must have been up before a clean exit closes its session.
* An exit sooner than this after the last pane start is read as a startup
* failure rather than a user ending the agent, and the row is kept.
*/
export const CLEAN_EXIT_MIN_PANE_LIFETIME_MS = 10_000;

/** The lifecycle-log reason recorded when the sweep closes a session. */
export const CLEAN_EXIT_CLOSE_REASON = 'agent exited cleanly (status 0)';

/**
* Is this exit a clean one? True only for an explicit numeric status of 0 with
* no signal reported.
*
* ⚠ Never widen this to `(exit.status ?? 0) === 0` or to "no signal, so it was
* clean". An absent status is how a signal death presents on tmux 3.2a, and
* that shortcut would close crashed agents with nothing failing to warn you.
*/
export function isCleanPaneExit(exit: PaneExit | undefined): boolean {
if (!exit) return false;
if (exit.signal !== undefined) return false;
return exit.status === 0;
}

/** Everything the sweep needs to know about one session. */
export interface CleanExitSweepCandidate {
/** The session's published exit, already scoped by `Session.setPaneExit()`. */
paneExit: PaneExit | undefined;
/** Authoritative pane reads that agreed on that exit (`getPaneExitReadCount()`). */
confirmingReads: number;
/** A start, attach or relaunch is running for this session's pane. */
paneLifecycleInFlight: boolean;
/** The session is already being closed or detached. */
closing: boolean;
/**
* When the last start, attach or relaunch of this pane finished
* (`Session.paneStartedAt`), or 0 when none has run in this process.
*/
paneStartedAt: number;
}

/** Should the sweep close this session now? See the file overview for the rule. */
export function shouldCloseCleanlyExitedSession(candidate: CleanExitSweepCandidate): boolean {
if (candidate.closing) return false;
if (candidate.paneLifecycleInFlight) return false;
if (!isCleanPaneExit(candidate.paneExit)) return false;
// `at` is when this server first read the pane dead, so an exit during the
// start itself lands BEFORE `paneStartedAt` and is kept too.
if (
candidate.paneStartedAt > 0 &&
candidate.paneExit!.at - candidate.paneStartedAt < CLEAN_EXIT_MIN_PANE_LIFETIME_MS
) {
return false;
}
return candidate.confirmingReads >= CLEAN_EXIT_CONFIRMING_READS;
}
Loading
Loading