Repository navigation
One conformance corpus for both devpod fakes #309
Description
Question
The two hand-maintained devpod fakes have drifted, in the dangerous direction (fake stricter than reality): DevpodMachine::delete ignores --ignore-not-found and refuses (rust/devlaunch-test-support/src/devpod.rs:269-278) where real devpod 0.26.1 succeeds and the Python shim was explicitly fixed for this ("the one thing a fake devpod must not do", test/fixtures/devpod_shim.py:224-235). Their value-flag lists also differ and both are incomplete against flags dl really passes.
Decide the mechanism that keeps them agreeing: one argv→outcome conformance table both suites drive (owned where, in what format), generating one fake from the other, or retiring one fake entirely.
Activity
blooop commentedon Aug 22, 2026
human-decided: One shared conformance corpus, as asked and as answered.
A JSON argv→outcome table lives in
test/fixtures/devpod/— the spot both trees already share recorded devpod output through (include_str!on the Rust side atclients/devpod.rs:1977-1985, plain reads on the Python side). Rows carry seeded state, argv, expected exit and expected state effect. One Rust test drivesDevpodMachineover every row; one pytest drivesdevpod_shim.pyover the same rows.Ruled out on evidence: retiring a fake — they serve different tiers, both are live in CI, and the Rust integration tests themselves run the Python shim as their whole devpod (11 call sites, ~320 tests). Generating one from the other — the repo has no codegen step at all (no build.rs anywhere), and a cross-language generator is a bigger standing cost than the drift it prevents.
The corpus must cover at minimum:
delete --ignore-not-foundon a missing workspace (exit 0, silence — the drift that motivated this), the up value-flags production really sends and both fakes currently mis-parse (--init-env,--dotfiles-script,--mount,--dotfiles-script-env,--workspace-env-file), and ssh's--workdir. Format details belong to #310, now unblocked.blooop commentedon Aug 22, 2026
Correction to this decision's record, from measurement during the build it authorised (PR #353).
The resolution above quotes the shim's comment that real devpod
delete --ignore-not-foundon a missing workspace "exits 0 with nothing to say". The second half is wrong. Measured against real devpod v0.26.1, it prints a timestamped, colourised log line (done deleted workspace delete.go:101). The exit code — the part the defect was about, and the part anything in this repo depends on — is 0 as recorded.So the shim's own comment carried an inaccuracy of the same kind it was written to warn about, and the corpus deliberately keeps stdout outside a row's contract for exactly this reason: nothing here parses that output, and pinning text would pin a fake's invention rather than the real tool's behaviour. Recorded here rather than only on the build ticket, because this comment is the decision anyone reading the map will land on.