Repository navigation
#872 stage A: Running a sweep section in the workspaces guide #890
Description
Lead agent, model claude-opus-5-5 via anthropic.
Stage A of #872, and the whole feature. Plan section: Design (Stage A) of plan v2, which stays authoritative. Where this body and the plan differ, follow the plan. Don approved v2 on 2026-10-14. His v1 answers still stand: Q1 no (rerun_job unchanged), Q2 the default note convention, Q3 WORKSPACES.md.
Reshaped for plan v2 (this stage's C-F469 bounce): item 5 now describes the latest-run-only step cache, the acceptance repeat is of the latest variant, and item 2 keeps the template's cost block. Scope is unchanged. The guide text for v2 is already on feat/872-A-sweep-guide @ 2aecdb5d.
Builds
A ## Running a sweep section in docs/WORKSPACES.md, so get_guide("workspaces", section="Running a sweep") serves it. It covers:
- One workspace per experiment; its workspace note holds the question and the factor list.
- A base workflow with every factor as a variable: a catalog template when it already exposes the factor, otherwise
save_workflowa copy withvariable:references, keeping the template'scostblock soplan.estimatestill has a basis. Note that{}keeps a dtype value a string. - Pin the seed in
arguments; change one factor per variant. Quotevalidate_workflow'splan.estimateper variant and get one go-ahead for the whole list. - One
run_workflowper variant, with anotewhose first line is the overrides (steps=20 attention=sage).wait_for_jobbefore submitting the next, because of host RAM (bug: two int8 H3 jobs on two cards OOM-kill the host - dispatch is VRAM-aware only, and the earlier job was the one killed #869); that sentence is dropped once bug: two int8 H3 jobs on two cards OOM-kill the host - dispatch is VRAM-aware only, and the earlier job was the one killed #869 ships. - Only the latest run of a step is cached. A variant identical to the latest run (the same variant submitted twice in a row) is
reused. Going back to an earlier variant after another has run re-renders it. Each card's worker has its own cache, and a restart empties it. A re-render is expected, not a failed variant. - The results table:
list_jobs(workspace=…)for job, status and note;get_jobfor timing and files. - Assessments with
set_noteon the output file. First linescore N/5 · judged_by <who>, then onekey: valueline per facet (faces, staging, speech, defects). - Then
keep_outputthe winner, and put the conclusion in the workspace note.
Owes
- A test asserting the section exists and names the tools it tells an agent to call, in the style of
tests/test_mcp_guides.py/tests/test_server_guides.py. - No change to
dw_mcp/, the REST API, orSURFACE_BUDGET.
Deploy
Server: the guide text ships with the server checkout get_guide reads, so lem pulls the new text.
Passes when
Its own repro works (plan v2, Acceptance intent: the section is served and names only real tools; following it on text-to-image with steps = 4 / 8 / 12, seed pinned, in a fresh qa-* workspace gives three noted jobs with the same seed and differing outputs; a fourth variant repeating the latest variant's overrides shows reused; a Q2-form set_note reads back in full in get_gallery_metadata and its first line in list_gallery), and every pending: #<this issue> case the tester writes passes.
No blockers. It does not wait on #869 or #865; if #865 is declined, step 2's "save a copy with variables" path covers the H3 knobs.
Activity
Merge feat/872-A-sweep-guide into develop (#890)
dkackman commentedon Oct 14, 2026
Lead agent, model
claude-opus-5-5viaanthropic.Shipped
776267a2ondevelop(merge offeat/872-A-sweep-guide).developto mini-ai before the tester runs. This is a docs-only stage:get_guidereads the server checkout, so the new text arrives with that deploy.Plan items (plan v1, Design (Stage A))
## Running a sweepis indocs/WORKSPACES.md, between Copying between workspaces and Where this is going.create_workspace+use_workspace. The question and factor list go inset_note("workspace:<name>", …).get_workflow(…, variables_only=true)); otherwiseget_workflow, add the variable, andsave_workflowa copy. The{}dtype note links Types and escaping.arguments, one factor per variant:validate_workfloweach variant, quote itsplan.estimate, and get one go-ahead for the whole list.run_workflowper variant: thenote's first line is the overrides (steps=8,steps=20 attention=sage). Callwait_for_jobbefore submitting the next variant. The bug: two int8 H3 jobs on two cards OOM-kill the host - dispatch is VRAM-aware only, and the earlier job was the one killed #869 host-RAM sentence is there, to be dropped once bug: two int8 H3 jobs on two cards OOM-kill the host - dispatch is VRAM-aware only, and the earlier job was the one killed #869 ships."reused": true), not a failed variant.list_jobs(workspace=…)for job, status and note;get_jobfor arguments,timingand files.set_noteon the output's gallery name, in the Q2 form (score N/5 · judged_by <who>, then onekey: valueline per facet). The full note is inget_gallery_metadata. Listings clip it at 200 characters.keep_outputit, and put the conclusion in the workspace note.Test:
tests/test_server_guides.py::…::test_the_sweep_section_names_the_loop_and_only_real_tools. It serves the real section throughguides.get_guide, asserts that each tool in the loop is named, and checks every backticked snake_case name in the section againstbuild_server(...).list_tools().No change to
dw_mcp/, the REST API orSURFACE_BUDGET.Deviation from the plan
text-to-imagedoes not expose steps as a variable. It hard-codesnum_inference_steps: 25. The acceptance repro (steps 4/8/12) therefore goes through step 2's "save a copy" path. The section uses that as its worked example: add"steps": 25undervariablesand set"num_inference_steps": "variable:steps". Following the guide literally means saving that copy first. Passingstepsstraight to the catalog template does nothing.Test state
ruffis clean.pytest tests/: 17 failures, which fail the same way ondevelopwithout this change. They are intest_examples.py(9),test_plugin_skills.py(5, Kandinsky 6) andtest_component_type_errors.py(3), and are unrelated to this stage.Nothing here is CUDA-specific: the repro runs on SD 1.5, which runs on mps.
dkackman commentedon Oct 14, 2026
Verify of #890 (stage A): FAIL on C-F469. Bouncing to the lead.
Tester: claude-opus-5-5 via anthropic. Server: mps on mini-ai.lan, develop @ 9e8f15d. I ran the stage's own repro plus the four
pending: #890cases in regression-suite-complete.md. Theirpending:lines stay in place.C-F469: the guide's step 5 doesn't hold for an earlier variant
Workspace
regression-sweep, workflowsweep-base(a copy of text-to-image withnum_inference_steps: "variable:steps",seeda variable). I ran three variants at seed 42:59b0c2eb6ede00bfd1dced41, run20261014-094347-cd1f1c832c7b87cc523bAll three succeeded.
run_workflow(sweep-base, {seed:42, steps:8}, wait_seconds=55)→ job1770448d66ec, run20261014-094410-cd1f1c83. Re-rendered: 9 passes, denoise_s 3.1, a new image file, noreusedanywhere. The guide's step 5 says "A variant whose step is unchanged from an earlier run with the same seed is served from the step cache". steps=8/seed 42 had already run (00bfd1dced41), but steps=12 had run since.validate_workflow(sweep-base, {seed:42, steps:8})right after →plan.cached_steps: 1.2069ea8093d0,reused: true. Its files weresweep-base/20261014-094410-cd1f1c83/final/test_image-0.0.jpg, which is call 1's run, not variant 2's20261014-094347-cd1f1c83.validate_workflow(sweep-base, {seed:42, steps:4})→plan.cached_steps: 0, though steps=4/seed 42 ran earlier.b3f837859bce, not reused (correct).What this shows from outside: the step cache holds only the most recent entry per workflow step. In a sweep, the variant you go back to is almost never the latest, so step 5's promise fails in the common case. Either the guide should say only the latest run of a step is cached (and C-F469 be amended to match), or the cache should keep one entry per input fingerprint. That is a design call for you. If the guide changes, the suite amendment goes through the approval route.
C-F470: one caveat
get_gallery_metadata. The listing showed its first line.score 3/5 · judged_by claude-opus-5-5 · xxx…was accepted.list_gallery(folder=sweep-base, version=3)clipped it to its first ~200 characters with…andnote_truncated: true.a note is at most 4000 characters, got 4099(andgot 4010,got 4101). The listing still showed the accepted note afterwards.keep_output(..., asset_name="sweep-winner.jpg")returnedasset:sweep-winner.jpg.set_note("workspace:regression-sweep", …)worked:list_workspacesshows the three-line note on the entry.Minor, not blocking
My copy of the template dropped its
costblock, sovalidate_workflow'splan.estimatecame back withminutes: null,basis: "unknown". The section's "save a copy" step doesn't say to carry the cost block over. That is worth a line if the section is revised.Cleanup:
delete_workspace("regression-sweep", acknowledged_cost=true)removed 1 workflow, 3 assets and 23 output files.feat(docs): #890 - sweep guide: only the latest run of a step is cach…
6 remaining items
dkackman commentedon Oct 14, 2026
Don approved plan v2 on #872 (2026-10-14). Back to the lead: re-spec C-F469 against v2 (repeat the latest variant), then merge
feat/872-A-sweep-guide@2aecdb5dand hand off to the tester.dkackman commentedon Oct 14, 2026
Body reshaped for #872 plan v2: item 5 is latest-run-only step caching, the acceptance repeat is of the latest variant, item 2 keeps the template's
costblock. Scope unchanged. The tester re-specs before the next build.Lead agent, model
claude-opus-5-5viaanthropic.Merge feat/872-A-sweep-guide into develop (#890)
dkackman commentedon Oct 14, 2026
Lead agent, model
claude-opus-5-5viaanthropic.Shipped to
developfor verify (plan v2)c243b994ondevelop(branchfeat/872-A-sweep-guide@2aecdb5d, plus a merge ofdevelop). This session doesn't deploy: the driver deploysdevelopto mini-ai before the tester runs.ruff checkis clean.pytest: 14358 passed, 18 failed. The same 18 fail onorigin/developwithout this branch (the kandinsky6 example workflows andtest_api_contract's committed document), so none are new.Plan v2 items → change
docs/WORKSPACES.md## Running a sweep, served byget_guide("workspaces", section="Running a sweep"):save_workflowa copy. The{}dtype note is there, and the copy keeps the template'scostblock (new in v2): step 2.plan.estimateper variant, one go-ahead: step 3.run_workflowper variant, with the note's first line holding the overrides, andwait_for_jobbetween variants because of bug: two int8 H3 jobs on two cards OOM-kill the host - dispatch is VRAM-aware only, and the earlier job was the one killed #869: step 4.reused. Going back to an earlier one re-renders it. Each card has its own cache, and a restart empties it: step 5.list_jobs(workspace=)andget_jobfor the table: step 6.set_noteon the output in the Q2 form (score N/5 · judged_by …, thenkey: valuefacets): step 7.keep_outputthe winner and put the conclusion in the workspace note: step 8.Test:
tests/test_server_guides.py::TestTheRealDocschecks that the section is served and names each tool of the loop. It checks that every backticked tool it names exists inlist_tools, and it pinsplan.estimate,reused,latest run,scoreandjudged_by.Deviations: none. No change to
dw_mcp/, the REST API orSURFACE_BUDGET, and no MCP shape changes. The repro runs on mini-ai, and nothing in it needs CUDA.dkackman commentedon Oct 14, 2026
Verified on mps on mini-ai.lan (develop @ c243b99). Tester: claude-opus-5-5 via anthropic.
I ran the stage's repro and all four
pending: #890cases exactly as written, in workspaceregression-sweep, all over MCP. All pass, so I removed thepending: #890line from each case.C-F467: PASS.
get_guide("workspaces", section="Running a sweep")serves the section, and so does a looser name.costblock; the bug: two int8 H3 jobs on two cards OOM-kill the host - dispatch is VRAM-aware only, and the earlier job was the one killed #869 wait reason.get_workflow'svariables_only.C-F468: PASS.
sweep-base, a text-to-image copy withnum_inference_steps: "variable:steps", cost block kept.other_device.8e92178908e6: steps=400d36241a08e: steps=8958aafaadca6: steps=12C-F469: PASS. Device was
mps Apple M5 Pro (MPS)throughout.0b849aad4991repeated steps=12:reused: true, and its files are variant 3's file.7e90dbfd5ffawent back to steps=8: not reused, rendered as a new run.0bccd3761055used seed 43: not reused.C-F470: PASS.
get_gallery_metadata.list_galleryshows it from the first line.·, first line startingscore 3/5 · judged_by):get_gallery_metadata;list_galleryclips it at ~200 chars withnote_truncated: true, starting with that first line.a note is at most 4000 characters, got 4001(andgot 4002). After the refusal, the stored note was still the 4000-char one.keep_output→asset:sweep-winner.jpgworked. Aworkspace:regression-sweepnote set with set_note shows up inlist_workspaces.delete_output(job_id=…)for all 6 jobs;delete_asset,delete_workflow;delete_workspace(acknowledged_cost=true).list_workspacesno longer listsregression-sweep.Note on the cases, not the code.
list_jobs(workspace="regression-sweep")returned 12 jobs. Six of them were"historical": true: leftovers from an earlier spec run, in a since-deleted workspace with the same name. The six non-historical ones were exactly what C-F468/C-F469 expect, newest first, with correct notes. A rerun in a reused workspace name will meet the same thing, so the cases should count non-historical jobs only. I'll propose that on the harness repo.