You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
study_area_run.sh: a run that stops before its tail leaves no record of which groups it was running #358
#348 lets a run that loses groups still recompute and compare the rest. It also adds --recompute-only --run-uid=<uid> to finish a run once the missing groups have been re-modelled. Two gaps remain.
The expected set has no record of its own. Without --wsgs=, --recompute-only expects the groups that have a <schema>.log row for the run_uid on the dispatcher. Some groups have no such row:
a group that failed on a cloud host, since consolidate copies log rows only for groups that finished;
a group that failed before .lnk_log_run_start(), for example in lnk_load_overrides(), at the downstream guard's quit(1), or on a connect failure (wsg_run_one.R).
The driver knows the full set once the buckets resolve, but it writes that set nowhere.
Write <TS>_run_wsgs.tsv (group, host, SKIP) as soon as the buckets resolve. --recompute-only reads it by default (<run_uid minus its suffix>_run_wsgs.tsv, since a run_uid's prefix is its TS) and falls back to the log.
After a consolidate refusal, carry on to coverage with the sources that consolidated, keep the refused host, and exit non-zero as the kept-host path already does.
Acceptance
--recompute-only with no --wsgs= names a group that failed on a cloud host.
A run with one refused source recomputes the other hosts' groups and exits non-zero naming the refused host.
<TS>_run_wsgs.tsv is written before any spend and rewritten after per-host accounting with each host's SKIPs. --recompute-only reads the file together with the log, not instead of it.
A refused or failed source no longer ends the run when every source has a recorded outcome (<TS>_consolidate.tsv). Its groups get the new coverage class not_consolidated and are kept out of the recompute.
A crash mid-copy is covered too. <TS>_consolidate_state.tsv is written in_flight before the copy, and a later bare --recompute-only keeps any group not ok out of the recompute until --wsgs= names it.
Activity
Merge pull request #359 from NewGraphEnvironment/358-study-area-run-s…
CLAUDE.md: status bullet for #358 (run_wsgs record, consolidate carry…