ci: bound the jobs, and stop rebuilding the sidecars every run - #213
Merged
Merged
Conversation
added 23 commits
September 18, 2026 09:36
The Rust job on master ran past twenty minutes with no end in sight. It was not wedged on anything this repository compiles: clippy had been running since 02:33 where the same step takes 65 seconds warm, because a cache entry never existed for it. Every cache in the repository is scoped to `refs/pull/211/merge` and keyed `Darwin-arm64`, which is the macOS panel job. Caches are branch scoped and a branch can only read its own and its base's, so nothing on master ever had one to read, and now that #211 is merged those entries are unreachable anyway. Neither job declared `timeout-minutes`, so the ceiling was GitHub's default of six hours. That is the part that turns a slow step into an invisible one: the run does not fail, it just never finishes, and the queue backs up behind it. Both jobs now have a bound sized to their warm time with room for a cold graph. The sidecars are the other two and a half minutes. Both build outside the workspace target directory, which is what `rust-cache` keys on, so neither was ever cached, and both are the least likely things in the tree to change: the proxy is a fixed-version crates.io install, and the reader is pinned to the v2-era WorkTable it must never move off. They are now cached as the produced binaries, keyed on the manifests, scripts, reader sources and GUI schema that decide them, since the reader's build script derives its schema copy from that. `runner.os` is in the key, so a Linux runner cannot pick up the host-target-suffixed macOS binaries sitting in a developer's checkout.
It was the longest step in the Rust job: 65 seconds warm, minutes cold. `--all-targets` type-checks the test and bench targets on top of the lib and bins, which is close to double the compile work, and `cargo test` on the next line then does nearly all of it again. A lint is worth most while the code is still open, which is the machine doing the writing, not one reporting six minutes later. It stays wired up here behind `workflow_dispatch` with a `clippy` input, so the machine's answer is one manual run away when someone wants it. `inputs.clippy` is undefined on push and pull_request, so the step is skipped there without needing a second condition. The exact local command is in the comment above the step, because it was documented nowhere: not in a gates file, not in CLAUDE instructions, and the only mention of clippy in `docs/` is one review from July.
A chat message read "and`WireMessage::Text`now takes`Utf8Bytes`": the spaces around every inline code span were gone. A paragraph is delimited by a blank line, so a single newline inside one is a wrap in the source rather than a break the author asked for. Between two plain words HTML collapses that newline to a space, which is why this went unnoticed for so long. Next to an inline element it does not: a text node ending in "and\n" followed by a <code> element has its trailing whitespace dropped at the element boundary, and the two run together. Normalising soft-wrap newlines to spaces before the inline split keeps the space inside the text node, where the boundary cannot eat it. In its own module rather than beside the renderer, so the test reaches it without importing MessageBody, which pulls in the component library and a motion entry point vitest cannot transform.
check_for_update went to the CDN with retry backoff even in a disposable QA profile. The round trip took 1 to 2s (2156ms in the failing run) against the 2s window of settings-update-check-completes, so that check's verdict was network latency. A QA process must also never be offered a published build to install over the checkout it is testing. With AZ_QA_WORKSPACE_ROOT set the command answers "no update" without the network. The check now completes in 77ms and the outcome suite is 312 of 312.
button-sweep.sh waited for target/blitz-control.json, which nothing writes any more, so every run ended in "ps-qa could not attach". The control server advertises <pid>-<instance>.json in a directory under $TMPDIR; the sweep now takes the one named for the pid it launched. The directory is matched by wildcard because its name is due to lose "tauri".
az-gui leaves /tmp/qa-profile-db.lock behind even after a clean SIGTERM drain, so the sweep refused to start on every second run. A lock whose pid is no longer alive is stale and is removed; one held by a live process still stops the sweep.
The chat-spacing change added the import out of order, and bun run lint fails the Frontend job on it.
tauri-runtime-blitz is published as izumo from 0.4.1, the release that answers agent-control requests instead of dropping the reply. The dependency, its three feature forwards, the runtime calls, the crates the owner build checks and the startup log all follow the new name. blitz-control-protocol moves to ^0.5.3, the version izumo is built against, so the two resolve to one copy.
The published 0.1.10 no longer compiles against endpoint-libs 3.2, which removed framed_json_with_max_frame, and the Rust job builds the sidecar from the resolved release, so every run failed there. 0.1.11 is built against 3.2.
The repository and its local checkout are now izumo; local-renderer.sh refused to run because it checked for the old directory.
The runtime is published as izumo and its repository is pathscale/izumo. The docs, the QA notes and a script comment said tauri-runtime-blitz or TRB; they say izumo, and docs/trb-consumer-history is docs/izumo-consumer-history. Two lines that name a version only published under the old name keep it with "(now izumo)".
blitz-preview still took tauri-runtime-blitz 0.3, the crate izumo was renamed from. It takes izumo ^0.4, and the ps-blitz crates move to ^0.4 with it: set_document_factory takes izumo's own ScriptDocument, so a second ps-blitz line would be a second, incompatible type. Every call maps one to one. The README and a comment that said the control socket was private and unfinished are brought up to date.
The shorter crate name let rustfmt fold the debug-options call onto one line.
cargo fmt --check failed on six spots: three the izumo rename shortened enough to fold, three older ones.
izumo, blitz-control-protocol, agent-experimental and promptsyntax were required at a patch floor. No lockfile is tracked, so the newest release resolves anyway; the floor only had to be chased on every release.
… client agent-abstraction 0.5 and agency-proxy 0.1.11 run on nagoya and take the reactor they register on as an argument rather than starting one of their own. The app creates the one Reactor during setup, hands its Handle to the AgencyProxy client and, through list_models, to model discovery, and keeps the Reactor in AppState for the life of the process: dropping it would stop the thread that reports those sockets and child processes. list_models takes the app state now; the catalogue itself is catalogues(), which the tests call with no reactor because nothing is discovered. The proxy's event receiver is agency-proxy-client's own broadcast type.
az-gui calls tokio::select! and tokio::join! but never asked for the macros feature; the old agent-abstraction's tokio switched it on for the whole graph. With that dependency gone the release build stopped compiling, while the tests kept passing because the dev-dependency enables macros.
The editor mounts under its row, so a row near the bottom opened it below the fold: in CI's 1344x900 window the URL field laid out at y=936, focused but off screen, and `item-issue-editor-opens` failed. Focus moves no scrollport. Reveal the block holding the row and its editor, then focus.
`sh -c 'kill -INT $$'` does not reliably die of the signal: the shell catches SIGINT and can reach exit first, and an inherited ignore cannot be undone. The watcher then saw an ordinary exit and the test failed about one run in seven. Send SIGINT from the parent to a sleep whose SIGINT disposition is reset to default across the exec.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Rust job on master ran past twenty minutes. Not wedged on anything we compile: clippy had been running since 02:33 where the same step takes 65s warm.
What was wrong
No cache for the Rust job. Every cache entry in the repo is scoped to
refs/pull/211/mergeand keyedDarwin-arm64(the macOS panel job). Caches are branch-scoped and a branch can only read its own and its base's, somasternever had one. With #211 merged, those entries are unreachable anyway. The Linux Rust job compiles the full dependency graph from scratch on every run.No
timeout-minuteson either job, so the ceiling was GitHub's six-hour default. That is what turns a slow step into an invisible one: the run does not fail, it just never finishes and the queue backs up behind it.The sidecars rebuilt every run, ~2m30s combined. Both build outside the workspace target directory, which is what
rust-cachekeys on, so neither was ever cached.What this does
timeout-minutes: 20on Rust (5m warm, ~15m cold),6on Frontend (1m20s warm).runner.osis in the key so a Linux runner cannot pick up host-target-suffixed macOS binaries.The sidecars are the safest things to cache: the proxy is a fixed-version crates.io install, and the reader is pinned to the v2-era WorkTable it must never move off.
Expected effect
First run on master populates
rust-cachefor the branch; subsequent runs read it. Steady state should be well under the 5m17s that the last warm branch run took.