Repository navigation
Promote with_mapping_code flag to stand-alone lnk_compare_mapping_code() export #175
Description
Context
The link R package (NewGraphEnvironment/link, currently v0.38.0) ships a lnk_compare_* family for parity comparisons against bcfishpass. In v0.37.0 (#168) the family split into:
lnk_pipeline_run(conn, aoi, cfg, loaded, ...)— modelling-only umbrella; writes<persist_schema>.streams+ per-species habitat + barriers.lnk_compare_rollup(conn, aoi, cfg, reference, conn_ref, species)— reference-agnostic comparison reader; reads persisted state + reference DB, returns long-format rollup tibble.lnk_compare_wsg(conn, aoi, cfg, loaded, reference, with_mapping_code, ...)— thin wrapper over both, plus an additional mapping_code lens whenwith_mapping_code = TRUE.
The mapping_code branch (existing helper .lnk_compare_wsg_mapping_code at R/lnk_compare_wsg.R:600-696) builds streams_mapping_code tokens per segment via lnk_pipeline_access + lnk_pipeline_mapping_code, then diffs against bcfishpass.streams_mapping_code. It's the segment-level QA artifact that complements the rollup-km diff.
Today the mapping_code is a flag (with_mapping_code = TRUE) inside lnk_compare_wsg, not its own family member. The flag's implementation operates on the per-WSG working schema and is fully coupled to lnk_compare_wsg's bundled pipeline-run flow (i.e. it cannot run against persisted state alone).
Problem
- Family-shape inconsistency. The
lnk_compare_*family currently haslnk_compare_rollup(its own export) andlnk_compare_wsg(with_mapping_code = TRUE)(a flag). A naming family should have stand-alone members per concern, not a bundled-with-flag pattern. Other family members (lnk_pipeline_*,lnk_barriers_*) already follow this rule. - Coupling to working schema. The current mapping_code build runs after
lnk_pipeline_*phases left a working schema populated. It cannot be re-run against an already-persisted WSG without re-running the entire pipeline. This blocks an operator workflow like "rebuild streams; later compute mapping_code on the same WSG without re-pipeline." - Resume gate doesn't apply. The v0.37.0 PG-state resume in
data-raw/wsgs_run_host.Ronly handles the rollup. A--with-mapping-coderun does its own full pipeline every time.
Goals
- Expose
lnk_compare_mapping_code(conn, aoi, cfg, reference = "bcfishpass", conn_ref, ...)as a stand-alone export, sibling tolnk_compare_rollup. - Read inputs from
<persist_schema>(streams + per-species habitat + barriers). No working-schema dependency on entry. - Rebuild the access/streams_mapping_code intermediate tables in a fresh per-WSG working schema (or accept a
schema=arg to control). Drop on exit unlesscleanup_working = FALSE. lnk_compare_wsg(with_mapping_code = TRUE)becomes a thin convenience wrapper that callslnk_pipeline_run() + lnk_compare_rollup() + lnk_compare_mapping_code(). Preserves operator-facing call surface; signals that the flag-mode is no longer the primary path.- Resume gate in the orchestrator extends to mapping_code: when PG state is canonical for streams/habitat/barriers AND a prior mapping_code RDS exists, skip; when PG state is canonical but mapping_code RDS is missing, run mapping_code only (~10-20s instead of full pipeline).
Acceptance
- New export
R/lnk_compare_mapping_code.Rwith the signature above; reads<persist_schema>only (noloadedarg). -
tests/testthat/test-lnk_compare_mapping_code.Rcovers arg validation + composition (working-schema rebuild + diff) viawith_mocked_bindings. Live-DB bit-identical test against the existinglnk_compare_wsg(with_mapping_code=TRUE)output for one WSG. - Refactor
.lnk_compare_wsg_mapping_codeintolnk_compare_mapping_codeproper; old helper becomes a@noRdwrapper for back-compat or is removed. -
lnk_compare_wsg(with_mapping_code = TRUE)still works (back-compat); now routes through the new exports. -
data-raw/wsgs_run_host.Rresume gate gains a third branch for "mapping_code missing but pipeline + rollup OK" → calls mapping_code only. -
data-raw/wsg_compare.Rcompanion — accept awith_mapping_codearg or expose a secondwsg_compare_mapping_codeper the data-raw split convention from Decouple bcfp compare from link pipeline run — separate compare-only step #168. - NEWS entry +
devtools::test()clean +/code-checkclean per phase commit.
References
- v0.37.0 NEWS entry (NewGraphEnvironment/link
NEWS.md) — describes the Decouple bcfp compare from link pipeline run — separate compare-only step #168 decouple and the mapping_code-stays-bundled-for-v1 decision. - Archived PWF for Decouple bcfp compare from link pipeline run — separate compare-only step #168:
planning/archive/2026-05-issue-168-decouple-pipeline-compare/{task_plan,findings,progress}.mddocuments why mapping_code was deferred. - Current mapping_code implementation:
R/lnk_compare_wsg.R:199-256(orchestration path) +R/lnk_compare_wsg.R:600-696(.lnk_compare_wsg_mapping_codehelper). - Sibling export pattern:
R/lnk_compare_rollup.R(added in v0.37.0). - Related future-issue (filed alongside): family-rename
lnk_compare_wsg → lnk_compare_run(pairs naturally with this one once all family members exist).
Out of scope
- The
lnk_compare_wsg → lnk_compare_runrename — separate issue. - Persist family rename — separate issue.
Update 2026-05-24 (post-#200): make the reference tunnel-free + add cross-WSG recompute
#200 (v0.40.4, merged) reproduced bcfp's per-species access so dam-downstream segments emit ;DAM, validated by a per-segment mapping_code match vs the local snapshot fresh.streams_vw_bcfp (PARS BT 98.95%, LFRA BT 97.77% / CO 97.90%). That sharpens this issue:
Reference is the local snapshot, not the tunnel. lnk_compare_mapping_code should diff link's <persist>.streams_mapping_code against fresh.streams_vw_bcfp (loaded tunnel-free by snapshot_bcfp.sh --with-bcfp-views), joined on blue_line_key + downstream_route_measure per species. Single local conn, no conn_ref, no :63333, no PG_PASS_SHARE. Supersedes the tunnel-coupled framing above and resolves #167 (no tunnel → no drops). bcfp build identity verified against s3://fresh-bc/bcfishpass/log.json (currently v0.7.15-14-ge12c1a5).
Orchestrator: compare on the dispatcher, post-consolidate (the #200 cross-WSG fix). In distributed runs each host computes mapping_code against only its local bucket's barriers, so cross-WSG ;DAM (e.g. PARS→PCEA/UPCE dams) is wrong across hosts. Move compare off the per-host path: after Step 9 consolidate (all barriers province-wide in one schema), recompute streams_access + streams_mapping_code for all WSGs, then run lnk_compare_mapping_code once. Net simplification: cyphers only run+persist (no tunnel, no compare); the dispatcher does recompute + one local snapshot-join compare.
Also in scope (orchestrator): M1-as-dispatcher wiring in wsgs_run_pipeline.sh / wsgs_dispatch.sh (drop the hardcoded ssh m1 self-ssh + the M4-centric host model).
Validation target: the 3 study areas — Peace (16 WSGs), Fraser (8), Skeena (5), drainage-closed to ~52 WSGs, 4-way (3 cyphers + M1). Relates to SRED NewGraphEnvironment/sred-2025-2026#24.
Activity
Shape-tolerant consolidate + cypher_prep species alignment (#204, #175)
Lean study-area runner: drainage-closed DS-first, tunnel-free, M1-dis…
PWF: Stage A pass (PARS ;DAM confirmed, 99% match); full 3-area run l…
study_area_run.sh: SSH-readiness wait before cypher prep + largest-ar…
study_area_run.sh: cyphers run dispatcher branch via CYPHER_PREP_BRAN…
study_area runner: species-presence filter + per-WSG soft-fail (#175, #…
PWF: data-loss incident root-caused + records-based fixes (#175)
docs: study-area run runbook + README entry (#175)
study_area_run: post-consolidate recompute = cross-WSG correctness gu…
PWF correction + 2026-05-25 parity doc (#175)
CLAUDE.md: refresh Status handoff to #175 (study-area parity, post-co…
#205: cheap post-consolidate recompute — lnk_access + 4 supporting fixes
Study-area mapping_code parity (tunnel-free) + cheap post-consolidate…
Archive planning files for issue #175
v0.42.0 (#215): PARS bull trout + Arctic grayling habitat-connectivit…