Fix --ignore-not-found and drive the corpus through both fakes #310

Activity

1. blooop

   added

   wayfinder:build

   on Aug 21, 2026
2. blooop

   added a parent issue on Aug 21, 2026

   • Close out the 2026-08 architecture review #299
3. blooop

   marked this as blocked on Aug 21, 2026

   • One conformance corpus for both devpod fakes #309
4. blooop

   self-assigned this

   on Aug 22, 2026
5. blooop commented on Aug 22, 2026

   

   blooop

   on Aug 22, 2026

   OwnerAuthor

   More actions

   breadcrumb: entering building — wf-tdd stage subagent launched (manager: wf-mid run working map #299).
6. blooop commented on Aug 22, 2026

   

   blooop

   on Aug 22, 2026

   OwnerAuthor

   More actions

   breadcrumb: seams, as the ticket and #309 pre-agreed them, plus what I established about real devpod before writing a single expectation.

   Three seams:

   1. The corpus file — a JSON argv→outcome table under the shared devpod fixture directory. Rows carry a name, seeded workspaces, argv, expected exit, and the expected workspace set afterwards. Deliberately not expected stdout: real devpod's own delete line is a timestamped, colourised log line, so pinning text would be pinning a fake's invention rather than reality. Exit code plus state effect is the whole contract.
   2. A Rust test over the in-process machine, driving every row.
   3. A pytest over the shim program, spawning it per row with a seeded state file, driving the same rows.

   Fidelity, established against real devpod v0.26.1 on this host rather than inferred from either fake:

   • delete <missing> --ignore-not-found exits 0 (and prints its usual "done deleted workspace" log line — so silence is not the real behaviour either, but text is out of the corpus's contract). Without the flag the same call exits 1. Flag before or after the positional makes no difference.
   • Real devpod's own help is authoritative on flag arity, and it says every flag the two fakes mis-handle takes a value: --init-env, --mount, --dotfiles-script-env and --ide-option/--provider-option are stringArray; --workspace-env-file, --prebuild-repository, --dotfiles-script-env-file are strings-plural; --dotfiles-script and ssh's --workdir are plain strings.
   • Real devpod accepts those flags before the positional source and still takes the positional as the source — measured: an up with --init-env FOO=bar --mount type=bind,... /tmp/nope-xyz-123 in front reached the clone step for /tmp/nope-xyz-123, so FOO=bar was never mistaken for a source. Both fakes would have taken FOO=bar. That is the shape that turns the flag-list gap from latent into observable, so it is what the corpus rows use.

   One row is annotated as unverified rather than guessed: delete of an existing workspace needs a real container to observe, so its expected outcome is inherited from both fakes agreeing, and the row says so.
7. blooop commented on Aug 22, 2026

   

   blooop

   on Aug 22, 2026

   OwnerAuthor

   More actions

   breadcrumb: built, PR #353. Three red→green slices.

   1. The reported defect: the in-process fake refused --ignore-not-found where real devpod exits 0. Red on both flag orders, then honoured on both, writing nothing — real devpod's line there is a timestamped log line no fake can spell.
   2. The shared corpus and its Rust driver. 24 rows, and driving them turned the known value-flag gap from latent into a failure: up with a value flag ahead of the positional made the flag's value the workspace source. The up value-flag list is now taken wholesale from real devpod's own help rather than grown a flag at a time, per verb, with the inherited globals in one place.
   3. The same corpus through the shim program, one spawn per row. Six rows went red — the five up value flags and ssh's --workdir — and the delete rows passed from the start, which is exactly the asymmetry that motivated the ticket: that behaviour was right in one fake and wrong in the other, with nothing comparing them.

   Two things worth carrying forward for whoever reviews or extends this:

   Provenance is part of a row, and enforced. Each row says whether its expectation was measured against the real binary on the host or is unverified and inherited, and a test fails a row that says neither. Collapsing those two kinds of claim is how the drift got in; anything needing a provisioned workspace to observe (delete/stop of a workspace that exists, up restarting one, ssh starting one) is on the unverified side and says what it is inherited from.

   stdout is outside the contract, on purpose. Real devpod answers a missing-and-ignored delete with a colourised, timestamped log line that nothing in this repo parses. A row asserts exit code and workspace state only — pinning text would pin a fake's invention. The output shapes that are parsed stay pinned against the recorded fixtures next to the corpus.

   Also measured and now relied on: real devpod accepts these flags before the positional and does not consume it. That is the shape the flag-ordering rows use, and it is what makes the gap observable rather than latent — in production ordering, source-first, a mis-parsed value flag happens to be harmless.
8. blooop commented on Aug 22, 2026

   

   blooop

   on Aug 22, 2026

   OwnerAuthor

   More actions

   breadcrumb: build → in review. PR #353 pushed; all 13 checks green.

   The fidelity half — the dangerous half, and the reason this ticket existed — was done by measurement against real devpod v0.26.1 on the host, not inferred from either fake: delete <missing> --ignore-not-found exits 0 on either side of the positional and 1 without the flag; stop/ssh/status on an unknown workspace exit 1; list --output json on a fresh home prints []. Flag arity came from each verb's own --help, and interspersal was measured directly rather than assumed.

   Three things worth carrying into review:

   • Rows that could not be measured say so, and a test enforces it — every row's verified line must open with measured or unverified. Four rows need a provisioned workspace and a container runtime, so they name what their expectation is inherited from instead of claiming measurement. That is the right shape for a corpus whose whole purpose is fidelity.
   • stdout is deliberately outside a row's contract. Real devpod answers a missing-and-ignored delete with a timestamped, colourised log line that nothing in this repo parses — which contradicts the shim's old comment claiming silence, so the corpus corrects a second inaccuracy beyond the one the ticket named. Exit code and state effect are the real contract; pinning text would pin a fake's invention. Parsed output shapes stay pinned against the recorded fixtures.
   • The value-flag gap was latent, not live, because production puts the source first — which is exactly why it survived unnoticed. The corpus rows put the flags where real devpod also accepts them.

   The delete rows passed against the shim from the start and failed against the in-process fake, which is precisely the asymmetry #309 was decided to close.
9. blooop commented on Aug 22, 2026

   

   blooop

   on Aug 22, 2026

   OwnerAuthor

   More actions

   wf-review: Approve (no blocking findings on either axis) — #353 (review). Two-axis report posted on the PR; five non-blocking follow-ups noted, the top one being the value-flag tables still duplicated across both fakes.
10. blooop

    marked this as blocking on Aug 22, 2026

    • The conformance corpus's guards can lose a row silently #357
11. blooop commented on Aug 22, 2026

    

    blooop

    on Aug 22, 2026

    OwnerAuthor

    More actions

    breadcrumb: in review → approved, awaiting human merge. Both axes pass, no blocking findings, and both axes genuinely ran as independent subagents.

    The Spec axis went past what the build claimed: it added a docker provider to a scratch home, provisioned an alpine devcontainer, and measured all seven rows the corpus marks unverified — every one came out exactly as written. So the corpus is faithful, but its provenance lines overstate the obstacle: measurement was inconvenient rather than impossible. The reviewer also mutation-tested both drivers rather than reading them — 10 mutants, all 10 killed, including pure state-effect divergence with the exit code unchanged, and drift in the lenient direction, which is the direction that caused the original defect.

    Non-blocking findings graduated to #357, because they are guard-decay rather than corpus-fidelity problems: a single row can be deleted with cargo test and pytest fully green (including the regression row for the very defect this PR fixes), since the guard checks flag mentions across all rows rather than row identity; the measured/unverified honesty guard is Rust-only, so a Python-side edit can drop it silently; and the five value-flag tables are now parallel in two languages with nothing diffing them, with roughly 14 of 44 flag names covered — so #309's defect class is half-closed rather than closed.

    A correction was also posted on #309 itself: its recorded resolution repeats the shim's claim that real devpod is silent on a missing-and-ignored delete, which measurement disproves. Fixed there so the map's decision record is not itself carrying an inaccuracy of the kind it was written to prevent.

    Side effects confirmed none: everything ran under scratch DEVPOD_HOME/XDG_CACHE_HOME, docker ps -a and docker volume ls empty afterwards, no destructive verb against unscoped state.
12. blooop

    closed this as completedin #353on Aug 23, 2026