Skip to content

Execute the Rust cutover: drive PR #260 to a merged 0.1.0 release #261

Closed

Description

@blooop

Execution ticket (wayfinder:build) for the follow-on effort that map #248 defers
to its own map with an execution override. The plan is decided (#248, tickets
#249–#255, docs/rust-rewrite-plan.md); this ticket is the doing: finish PR #260,
turn CI green, then merge and ship the first Rust release.

Where it stands

  • Branch: rust/big-bang-port — PR Rust port of devlaunch (big-bang cutover, 0.1.0) #260 (open, mergeable, not draft). 24 commits.
  • The port is functionally complete: four-layer Rust workspace under rust/
    (runner → tool clients → domain → flows), dl + aid binaries, ~1410 Rust tests
    incl. 108 binary-boundary tests against Python goldens, parity manifest empty,
    spec ledger 0 pending, compare.py 10/10 SAME.
  • A six-angle adversarial review ran against the finished tree; every material
    finding was reproduced and the fixes are committed (waves 1–2 + CI-fix commits).
  • A post-review tree audit re-confirmed the in-PR fixes actually landed: R9
    refresh-path fetch timeout (5s ls-remote bound; no unbounded fetch on any
    user-facing path), R7 prune lock-wait notice, F1 migration-under-lock+reload,
    the --force grammar, empty-id launch refusal, and the maturin wheel packaging.
    The one gap it found — aid's SIGINT handler was a bare _exit that skipped
    dl's cleanup, leaking the staged GitHub token and orphaning the up child on a
    Ctrl-C mid-aid-launch
    — is now fixed (both binaries share one
    dl::install_interrupt_handler; new aid/tests/interrupt.rs guards it).

Remaining work (the finish line)

  1. CI green. The rust-parity job was deterministically killing the GitHub
    runner at ~46m. Root cause: the lock_wait integration test drove dl --prune
    through .output() (blocks on stderr-pipe EOF that a detached refresh grandchild
    can hold open), and its later 30s "deadline kill" used kill -KILL -<pgid> with
    no --, which procps parses as a signal word and no-ops — so nothing was killed
    and an orphaned test binary held the step's log pipe open. Fixed in the last two
    commits (stderr→file + process-group + a deadline that kills via Child::kill
    plus kill -KILL -- -<pgid>; CI step routes cargo output through a file). The
    per-binary Test split in ci.yml was a diagnostic — consider collapsing it back
    to one bounded step once stable. The fix is confirmed: on the run before the
    aid commit, Test dl lock_wait and Test dl interrupt both passed (the runner
    no longer dies). aid's process-spawning interrupt test got the same log-pipe
    fence. Verify the latest run is fully green before merging.

  2. File the pre-existing security findings PRIVATELY. The review surfaced host
    RCE paths and path-containment issues that exist in the frozen Python too and
    are therefore NOT introduced by this PR. They must NOT be disclosed in a public
    issue or PR comment. The private write-up is with the owner; file via the repo's
    private security-advisory flow, not the public tracker. (Details deliberately
    omitted here — this is a public ticket.)

  3. Merge + release 0.1.0. Cutover ships as 0.1.0, version single-sourced in
    Cargo.toml; both channels (PyPI maturin bin-wheel + prefix.dev conda) publish
    from one tag on version change (publish.yml/conda-publish.yml), linux-64 only.
    The released dl/aid flip from Python to Rust here. Rollback = pin <0.1.
    Merge and release only on explicit owner go-ahead.

Constraints for the taker

Closes the execution half of #248.

Activity

  1. blooop commented on Aug 19, 2026

    @blooop
    OwnerAuthor

    CI status update (2026-08-19). Two distinct rust-parity blockers, in order of discovery:

    1. Runner hang at ~46m — FIXED & confirmed. The lock_wait integration test drove dl --prune through a pipe that a detached refresh child held open, and its deadline-kill used kill -KILL -<pgid> without -- (procps no-ops that), so an orphan held the step's log pipe until the runner died. Fixed (commits on branch); Test dl lock_wait/Test dl interrupt now pass in CI.

    2. e2e interactive PTY tests fail — FIX IN PROGRESS. Once the hang was fixed, rust-parity's tier-2 e2e reached the interactive suite for the first time in a while and 11 test_interactive_session.py tests time out (never saw 'TTY-STATUS:HAVE', round-trips hang). Root cause: a prior commit made ProcessRunner::passthrough always put its child in its own process group (setpgid(0,0)). That's right for devpod up (so a Ctrl-C can kill the build), but the interactive ssh -t exec also goes through passthrough — and a child in its own group is not the controlling terminal's foreground group, so it stops on SIGTTIN when it reads the PTY. Fix: make own-group a per-call choice on SpawnSpec — devpod up keeps it, ssh/git don't. The standalone Python e2e job passes throughout, which is consistent with this being a Rust-runner-only regression.

    Merge/release remain held for owner go-ahead.

  2. blooop commented on Aug 19, 2026

    @blooop
    OwnerAuthor

    CI is fully green (run 32279756786, commit f3a9e79). All jobs pass — rust-parity (both tiers incl. the tier-2 e2e interactive suite that was timing out), e2e, the py310–313 + default matrix, packaging, and gate. Both CI blockers are resolved: the runner hang and the interactive-ssh -t SIGTTIN regression.

    Remaining, both on owner go-ahead only:

    1. File the pre-existing security findings via the private security-advisory flow (never the public tracker). Write-up is prepared.
    2. Merge PR Rust port of devlaunch (big-bang cutover, 0.1.0) #260 and cut the 0.1.0 release.

    The PR is green and mergeable; nothing technical is outstanding.

  3. blooop commented on Aug 19, 2026

    @blooop
    OwnerAuthor

    Status update (2026-08-19 evening): an independent review of PR #260 re-confirmed the finish-line claims — CI green on head f3a9e79, full cargo suite green locally, parity manifest empty, ledger 0 pending, release wiring verified (note: merging #260 IS the 0.1.0 release — Auto-publish fires on the merge commit since Cargo.toml already says 0.1.0, and CHANGELOG has the required ## [0.1.0] section).

    The three non-security follow-up tickets (#262/#263/#264) are now implemented on PR #265, stacked on this branch, so #260 itself is unchanged. Remaining here: the private security filing (with the owner) and the merge+release on explicit owner go-ahead.

  4. blooop commented on Aug 19, 2026

    @blooop
    OwnerAuthor

    Owner-directed re-litigation (2026-08-19): merge + 0.1.0 is now gated on one more decision — #266.

    With the port implemented and green, the owner directed a re-examination of what the cutover actually buys. The load-bearing benefit — wf linking devlaunch-core in-process, killing the wire-skew bug class (#53's driver, #250's closure) — was re-measured against wf's actual code rather than the estimates:

    • The wire-tolerance layer is ~100–200 lines of serde parse in wf's src/reap.rs plus one version-floor error message, not ~300; the rest of reap.rs is deletion policy that survives linking unchanged.
    • wf's launch path deliberately execs into the dl binary for terminal handoff, so linking converts only the listing/rm calls — the binary stays in wf's world either way.
    • Skew moves rather than dies: from a loud runtime version-floor ("needs devlaunch 0.0.21+") to silent pin-time staleness of a git dependency.
    • The wire seam delivers dl fixes to wf for free via pixi global update; a crate pin costs re-pin → rebuild → wf release per devlaunch fix.

    Every other claimed benefit was either disclaimed at decision time (perf: one devpod round-trip ≈ 5× Python startup) or doesn't materialize (distribution: same channels, narrowed to linux-64). The port's genuine side-yield — the bug finds, the DEVLAUNCH_DL_CMD boundary-test harness, the grammar fixes — is language-independent and is harvested under either outcome.

    Consequence for this ticket: the "merge and release only on explicit owner go-ahead" constraint stands; the go-ahead now waits on #266 — whether wf-in-process actually beats the wire seam, settled by a ~1-day prototype on wayfinder's side (or by an explicit owner call for the all-Rust ecosystem, which #266 names as a legitimate resolution). Since merging #260 is the 0.1.0 release, the gate sits in front of the merge itself. Everything else here is unchanged: CI stays green, the private security-fix flow proceeds regardless of the outcome.

    Nothing about the port's execution quality is in question — #266 is about whether the premise pays, and both of its outcome branches (merge-with-retirement-conditions / harvest-and-close) are written down there.

  5. blooop commented on Aug 19, 2026

    @blooop
    OwnerAuthor

    Shipped (2026-08-19). The cutover is complete; this ticket's three remaining items all closed tonight, in an owner-directed session ("finish everything related to this rust port"):

    1. CI green — confirmed on the PR head (all jobs including rust-parity, 5.6 min) and again on main post-merge.
    2. Security findings filed privately — the three draft advisories are in the repo's security-advisory flow (path-containment/deletes/tar-header; fsmonitor host RCE; completion-cache injection). Publication timing stays the owner's call.
    3. Merge + release — the Decide: wf links devlaunch-core vs keeps the wire seam — prototype before cutover #266 gate resolved first (wf keeps the wire seam; the port proceeds on the owner's standing all-Rust direction — see Decide: wf links devlaunch-core vs keeps the wire seam — prototype before cutover #266 for the prototype evidence). PR Rust port of devlaunch (big-bang cutover, 0.1.0) #260 merged at 21:28 UTC; Auto-publish tagged v0.1.0, the maturin bin-wheel is live on PyPI and Conda Publish succeeded on prefix.dev.

    Pre-merge verification beyond CI:

    • Coexistence on a real cache (cutover checklist item 3): Python 0.0.29 and Rust 0.1.0 alternated read-side against a copy of this machine's production metadata (21 real workspaces, real devpod) — --ls --json byte-identical, metadata untouched.
    • Post-release smoke test: pixi global update devlaunch → dl 0.1.0 / aid 0.1.0 (static-pie ELF) on the host; --ls, --ls --json (21 rows), --repos all correct against the real cache; the release's first real-world write (the background refresh's last_fetched stamps) is format-identical to Python's spelling.

    Known non-blockers, tracked elsewhere: the Launch-benchmark workflow was already failing on main before the merge (bench-container pixi permissions — #193's territory, not a cutover regression); dev-loop names still run Python (#268); Python-tree + parity-harness retirement after soak (#267); API freeze finishing in the shrinking direction (#262, direction set by #266). Rollback remains pin devlaunch <0.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions