Skip to content

study_area_run.sh: record a run's groups, and carry on past a refused consolidate source (#358) - #359

Merged
NewGraphEnvironment merged 9 commits into
mainfrom
358-study-area-run-sh-a-run-that-stops-befor
Oct 19, 2026
Merged

NewGraphEnvironment merged 9 commits into
mainfrom
358-study-area-run-sh-a-run-that-stops-befor

Conversation

@NewGraphEnvironment

Copy link
Copy Markdown
Owner

Fixes #358.

What changes

The run records its groups before any spend. study_area_run.sh writes <TS>_run_wsgs.tsv
(wsg, host, skip, run_uid) as soon as the buckets resolve, and rewrites it after the per-host
accounting with each host's SKIPs. --recompute-only --run-uid=<uid> reads
<run_uid minus its suffix>_run_wsgs.tsv together with every group holding a log row for the
run_uid, so a group that failed on a cloud host, or before its first log row, is named. A run from
before this change has no file and falls back to the log alone; --wsgs= still wins.

A refused consolidate source no longer stops the run. schema_consolidate() already loops past
a source it refuses or fails on. The driver now writes one outcome per source to
<TS>_consolidate.tsv and, when every source has one, carries on to coverage, the recompute and
the compare:

  • the failed host's bucket adds no finish signal and is a new coverage class, not_consolidated,
    kept out of the recompute whatever the dispatcher's log says (a failure after the DELETE can leave
    log rows copied and streams not);
  • host_release keeps the host while its finished groups are left on it, with
    consolidate failed at stage <stage> as the reason in kept_hosts.tsv;
  • the run exits non-zero and names the groups.

When an outcome is missing for any source (R died), the old exit after consolidate stays.

A crash mid-copy still leaves a record. <TS>_consolidate_state.tsv lists every source's groups
in_flight before the copy and ok / failed:<stage> after. A bare --recompute-only names every
group not ok and keeps it out of the recompute until --wsgs= names it.

Acceptance

  • --recompute-only with no --wsgs= names a group that failed on a cloud host: tested on a
    scratch schema (GGGG, run on job1, no log row, named under "no rows").
  • A run with one refused source recomputes the other hosts' groups and exits non-zero naming the
    refused host: tested at the lib level (consolidate_outcomes, run_finish with the excluded set,
    host_release). The driver's full-run path needs cloud hosts; a structural test pins its wiring,
    and the driver's R writer is run against the reader.

Verification

  • test-study_area_finish.R and test-study_area_hosts.R: all pass, none skipped, local docker
    fwapg up. Full suite: 3293 pass, 3 fail, 0 skip; the three failures predate this branch and sit in files it does not touch (two lnk_wsg_resolve tests need the :63333 tunnel, which was down; the golden ADMS dams pin expects 8 where cabd.dams now gives 7).
  • Mutation check: 14 restored defects across the build and the review rounds, each turning its test
    red (the build's six in the archive's findings.md, the review rounds' eight in its progress.md).
  • /code-check branch: three rounds, five fragile findings, all fixed, none open. Rounds 2 and 3
    found defects inside the previous fix; round 3's enumeration of every set combination ended it.
    The shared mechanism: "which groups may not be safely on the dispatcher" was derived separately at
    each consumer from records covering different populations or times. Fixes from review: a failed
    host's finished groups stay expected; <TS>_consolidate_state.tsv is written in_flight before
    the copy so a crash fails closed; a bare --recompute-only keeps groups not consolidated ok out
    of the recompute until --wsgs= names them; the report's kept-host set uses the same union
    kept_hosts.tsv checks.

RUNBOOK §8d ("A missing group costs only itself") is updated.

Relates to NewGraphEnvironment/sred#24

🤖 Generated with Claude Code

https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx

NewGraphEnvironment and others added 9 commits October 18, 2026 22:52
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
…that host (#358)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
…-only keeps them out (#358)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
…ash fails closed (#358)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
…e, not re-modelled (#358)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
@NewGraphEnvironment
NewGraphEnvironment merged commit 7d9bee9 into main Oct 19, 2026
1 check passed
@NewGraphEnvironment
NewGraphEnvironment deleted the 358-study-area-run-sh-a-run-that-stops-befor branch October 19, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

study_area_run.sh: a run that stops before its tail leaves no record of which groups it was running

1 participant