Repository navigation
lnk_persist_init: persist per-source dnstr flag columns in streams_access (mapping_code second-token NONE bug) #196
Description
Context
Post-#187 <persist_schema>.streams_access carries per-species access cols (has_barriers_<sp>_dnstr, access_<sp>) but drops the per-barrier-source flag cols that lnk_pipeline_mapping_code needs to classify the second mapping_code token:
has_barriers_anthropogenic_dnstrhas_barriers_pscis_dnstrhas_barriers_dams_dnstrhas_barriers_remediations_dnstrdam_dnstr_ind(sequence-aware dam-detection; resident flavor)remediated_dnstr_ind
These columns exist in working schema's streams_access (lnk_pipeline_access writes them when barrier_sources is passed) but #187 Phase 2 dropped them from cols_streams_access_base on (incorrect) "they're conditional on remediations/observations" reasoning. Actually conditional only on barrier_sources arg, which lnk_pipeline_run always passes.
Impact
lnk_mapping_code reads SELECT * FROM <persist_schema>.streams_access and finds the per-source columns absent → lnk_pipeline_mapping_code's has() helper returns FALSE → any_anth = any_pscis = any_dam = FALSE → second token defaults to NONE.
Empirically PARS BT 2026-05-19:
- link:
ACCESS;NONE4412 rows,ACCESS;NONE;INTERMITTENT7080 rows - bcfp:
ACCESS;DAM4411,ACCESS;DAM;INTERMITTENT5457,ACCESS;MODELLED1775,ACCESS;MODELLED;INTERMITTENT3113,ACCESS;ASSESSED900,ACCESS;ASSESSED;INTERMITTENT1100
Match_pct vs bcfp drops from historic 98%+ to ~30-50%.
Design choices
Column-naming inconsistency (pre-existing; not introduced by #187):
has_barriers_<source>_dnstr(boolean prefixhas_, suffix_dnstr)dam_dnstr_ind/remediated_dnstr_ind(nohas_prefix, suffix_indfor "indicator")
Two patterns coexist. Could rename to one convention (e.g., all has_<thing>_dnstr or all <thing>_dnstr_ind), but that's a separate cleanup. Scope of this fix: add the columns with existing names so lnk_pipeline_mapping_code's column probes find them.
Where to put the persist DDL:
- Option A: Extend
cols_streams_access_basedirectly with all 6 columns. Mirrors the bundle-wide DDL approach from lnk_persist_init: wide tables sized to active_species break on heterogeneous WSG runs #194. - Option B: New
.lnk_cols_streams_access_source_flags()generator (analogous to.lnk_cols_streams_access_per_sp(species)from mapping_code build decoupled from tunnel — persist streams_access + lnk_pipeline_run phase + rename with_mapping_code → mapping_code #187). Keeps per-source separated from base.
Picking B: future per-source extension (new barrier classes) lives in one place, mirrors the per-species helper pattern. Less coupling between "scalar base" and "source-driven".
Defaults: source classes are hardcoded today (anthropogenic/pscis/dams/remediations). #189-style data-driving deferred — would mean a parameters_barrier_sources.csv declaring which sources exist per bundle. Not in this hotfix.
Acceptance
-
R/lnk_persist_init.R: new.lnk_cols_streams_access_source_flags()helper returns named vector forhas_barriers_<source>_dnstr(boolean, 4 sources) +dam_dnstr_ind(boolean) +remediated_dnstr_ind(boolean) — 6 cols total. -
lnk_persist_initincludes those cols instreams_accessCREATE TABLE. -
lnk_pipeline_persist's INSERT projection picks them up automatically (iteratesnames(access_cols_v)). - Live smoke on PARS post-fix:
mapping_code_btdistribution showsACCESS;DAM/ACCESS;MODELLED/ACCESS;ASSESSEDtokens (not justACCESS;NONE). PARS BT match_pct vs bcfp returns to ~98%. - v0.40.3 patch release.
References
R/lnk_pipeline_mapping_code.R:174-194— column probes that consume the persisted flags.R/lnk_pipeline_mapping_code.R:196-210— hardcoded precedence chain (separate issue: rules engine).R/lnk_persist_init.Rcols_streams_access_base— currently missing the per-source columns.R/lnk_pipeline_access.Rlines ~250-260 — where the per-source flags get emitted.- Same bug class as lnk_persist_init: wide tables sized to active_species break on heterogeneous WSG runs #194 (caught by live data, not unit tests). Add unit tests for v0.40.0 surface (lnk_mapping_code, persist additions, pipeline_run mapping_code phase, deprecation shims) #191 tracks unit-test catch-up.
Activity
PWF #196: link filed issues #200 (access-set fix) + #201 (blocks_spec…
v0.40.3: persist per-source flags in streams_access (#196) (#199)
v0.40.4: per-species access reproduces bcfp barriers_<sp> so dam-down…